Does ChatGPT Plagiarize? Tested and Explained.

This year has seen the introduction of numerous AI assistants and tools, which seem to be revolutionizing the way professionals and students approach their work. You can now generate content, images, and more directly using an AI assistant, which has taken away the hassle of mundane tasks from a lot of workflows. Among these new AI assistants is the popular ChatGPT, an LLM chatbot that can pretty much do anything as long as it falls in the realm of its capabilities. You can generate articles, speeches, research papers, historical accounts, and much more using ChatGPT. However, this has led many users around the world to wonder if the chatbot actually produces new and original content. As it has been trained on tons of existing data on the internet, won’t ChatGPT plagiarize its outputs from available data? Let’s find out!

Does ChatGPT give the same responses?

Although ChatGPT has its limitations, the chatbot seems to be pretty good at its job. In the tests we did below, ChatGPT seems to give unique responses for the same prompt. This was first tested with the same prompt, then in the same conversation, from the same account, and then from a different user account. In all these instances, ChatGPT seemed to produce a unique response for the same prompt.

Prompt Comparision 1: Same prompt in different conversations

First Response in a new conversation
Response received from the first account

As you can see, the responses above are pretty similar to each other but not the same. A few words have been changed while keeping the context pretty similar to the original response.

Prompt Comparision 2: Same prompt in the same conversation

First response in the same conversation
Second response in the same conversation

We have a similar result when ChatGPT is presented with the same prompt in the same conversation. Although this time, it seems the text is pretty much the exact match to ChatGPT’s previous response, with only a few minor changes here and there.

Prompt Comparision 3: Same prompt from a different user account

Response received from the first account
Response received from the second account

This is where the difference between responses is quite prominent. Unlike the responses we got when using the same account, we got quite a different response for the same prompt from a different account. ChatGPT has changed the way it structures its answers while including different examples as well.

Related: How to Use Plugins in ChatGPT

Does ChatGPT plagiarize its output?

Now let’s address the elephant in the room. Does ChatGPT plagiarize its output? Plagiarism refers to the practice of presenting someone else’s work as your own. While not an issue for an AI chatbot, it can create numerous problems for professionals and students. So let’s do a few tests to check if ChatGPT plagiarizes its responses or not. Let’s get started.

Test 1: Prompt for published content

Let’s first test some of our own published content. Let’s start with a recently published post on our website. We will present ChatGPT with the title as the prompt and then check the results for plagiarism. Let’s use the post “How to Use Bing Chat on Chrome”.

As you can see, around 17% of the text from ChatGPT does indeed get flagged for plagiarism.

However, if you notice, the content gets flagged for plagiarism from random websites irrelevant to the topic. So depending on your client or your current needs, the content generated by ChatGPT can indeed be passable.

Related: How to Use ChatGPT on Discord

Test 2: Prompt for a published story

Next, let’s test for a popular, published story. This will give us the ability to check ChatGPT’s response against most website content, as the story will be quite popular and should be currently covered by a lot of websites. Let’s check out “Best iOS 15 features” in this example.

Note: We are using the free version of ChatGPT, which is using GPT 3.5 that has been trained on content published until 2021, which is why we have chosen an older trending story.

Now let’s check ChatGPT’s response for plagiarism.

This is the result when checking the content using Grammarly’s plagiarism checker. As you can see, the content seems to be pretty original. But let’s run it through Quetext as well, just to be safe.

Quetext seems to detect around 5% of the content as plagiarism which is quite true. But this is nothing that can not be fixed with a few edits. For now, it seems that ChatGPT is generating original content that can not be flagged for plagiarism.

Test 3: Prompt for a published news post

Now let’s try a published news story from 2020. This will help us ensure that we can test it against ChatGPT’s knowledge base, as it has only been trained up to content publically available until 2021. Let’s have ChatGPT generate a response for the “What is 1v1.lol game and why it’s special” post that was published by us back in 2020.

Now let’s check the content generated by ChatGPT for plagiarism.

This is where things tend to differ. 10% of the content generated by ChatGPT seems to be flagged as plagiarism by Grammarly. Let’s check Quetext and see how ChatGPT’s content fares for plagiarism.

Unfortunately, according to Quetext, around 15% of the content generated by ChatGPT is plagiarized. This can be an issue depending on your client; however, if you notice, the matching phrases seem to be quite generic. And the website where the matching content was found isn’t addressing 1v1 specifically.

Test 4: Prompt for a published product description

Another way to test for plagiarism is by testing for product descriptions. Product descriptions can be quite tough as the product specifications remain the same, which can make it harder for ChatGPT to generate original responses. Let’s try a product description for the iPhone 12 Pro.

Now let’s check the generated content for plagiarism with Grammarly.

This time ChatGPT seems to fair pretty well, with none of the content getting detected for plagiarism. But let’s check the content on Quetext as well, just to be safe.

This is where things get a bit complicated. Around 14% of the text seems to be plagiarized, according to Quetext. However, as with previous results, only generic statements seem to match the published content, with the published content not being about the iPhone 12 Pro at all. Now let’s also try technical specifications for the iPhone 12 Pro and see how ChatGPT fares with that.

Let’s run the content generated by ChatGPT through Grammarly’s plagiarism checker.

When it comes to technical specifications, around 14% of the content seems to be plagiarized. This is to be expected as features and technical specifications will end up being the same across posts for the same product. Let’s see how the content fares when checked with Quetext.

This is where things tend to get a bit more transparent. Around 61% of the content seems to be plagiarized when it comes to technical specifications. This is a lot and can land you in trouble depending on your client and their requirements.

Test 5: Testing ChatGPT’s response against its own plagiarism checker

This is a bit devious but why not? Let’s have ChatGPT analyze its own response for plagiarism, and then let’s check how it fares against Quetext, a popular plagiarism-checking tool. Let’s try a prompt for “How to use Google Duo on a Windows PC”.

Now let’s check the response using ChatGPT for plagiarism.

Finally, let’s check the response using Grammarly’s plagiarism checker.

Around 18% of the content is detected for plagiarism. This time the websites seem to be relevant as well as the phrases used by ChatGPT seem to be an exact match for the published content. Let’s see how the content fares when checked using Quetext.

Quetext also finds the content to be plagiarized. With a score of 42%, the content might be rejected by your client unless you decide to edit it or remove the matching phrases.

Does ChatGPT get flagged by AI detection tools?

Yes, ChatGPT will get flagged by AI detection tools, especially when using OpenAI’s own classifier. This is because these detection tools have been specifically made and generated to detect AI-generated content. AI-generated content has been a huge concern for organizations and educational institutions around the world, as many teachers wish to ensure that their students generate original content and reports while learning important lessons during the process.

How accurate are AI detection tools

This has been heavily debated in recent times, as many content creators and writers suggest that their content can be inappropriately flagged by AI detection tools. So is this true? Let’s find out in a few tests.

Test 1: Testing ChatGPT’s output

Let’s first test a ChatGPT output and see how it fares against popular AI detection tools. We will use “How to make a cake” as a prompt for this example.

This is the response generated by ChatGPT. Let’s now check it against the OpenAI classifier.

Now let’s check the generated response on CopyLeaks.

The results are quite surprising. While you would expect the OpenAI classifier to detect content generated by ChatGPT, that simply wasn’t the case. According to the classifier, the content was quite likely generated by a human. Copyleaks, on the other hand, was quite accurate and ended up detecting most of the content as AI-generated.

Test 2: Testing a post published before the release of ChatGPT

Now that we’re familiar with what happens with content generated by ChatGPT let’s check my originally published work from back in 2020. This was when none of these AI tools were available, so it will be interesting to see how this content fares against these popular AI detection tools. Let’s use this “Is ‘Fall Guys’ Free To Play?” post for this example. Let’s first check it with the OpenAI classifier.

Now let’s check it with CopyLeaks.

This is where the results seem to reflect our expectations. Both Copyleaks and OpenAI classifier find the content to be generated by a human instead of an AI.

Conclusion

If you’re a writer, then you might be aware of the issues with plagiarism when writing content professionally. Even if you write content from scratch, there can be times when some of your content is detected for plagiarism. The same seems to be the case for ChatGPT. When generating content about newer topics, ChatGPT seems to generate mostly original content. While generating content for older or niche topics leads to more plagiarism. So yes, the content generated by ChatGPT can be detected for plagiarism, and it’s always a good idea to check it through trustable sources in case that is a factor for you.

We hope this post helped you easily understand if ChatGPT’s content is plagiarized or not. If you face any issues or have any more questions, feel free to reach out to us using the comments below.