Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text

From GPT-2 [17] to GPT-3 [2], the emergence of Large Language Models (LLM) has made it possible that machines can generate a variety of high-quality texts that are quite similar to human language, making it hard to distinguish between human-generated and AI-generated texts. The release of ChatGPT [16] contributed to the widespread use of AI-generated text tools while leading to concerns that the abuse of ChatGPT may bring misleading information, phishing, and academic dishonesty [19]. Therefore, some studies [4] attempt to build detectors based on the pre-trained language model for ChatGPT-generated texts to prevent the abuse of the powerful ChatGPT.

However, detectors that perform well in intra-domain scenarios have poor robustness due to being easily attacked by carefully designed strategies such as paraphrasing and polishing [20], which refers to the process of revising and improving the original text. Although retrieval [7] can help mitigate this problem, it still poses significant challenges, such as high costs and lack of explanation. Moreover, as pointed out by the Association for Computational Linguistics (ACL), it is necessary to establish a grading system for the degree to which large models participate in the paper generation process.¹

In order to identify ChatGPT-polished texts and provide users with more intuitive explanations, we create a novel dataset called HPPT (ChatGPT-polished academic abstracts instead of fully generated ones) for training a detector and also propose the Polish Ratio method which measures the degree of modification made by ChatGPT compared to the original human-written text. Through experiments, we have demonstrated the effectiveness of our model in accurately identifying polished texts trained on our novel dataset. Moreover, our explanation method, the Polish Ratio, has shown promising results on both our own dataset and other datasets that have not been seen before: there are significant distinct distributions in the predicted Polish Ratio of human-written, ChatGPT-polished, and ChatGPT-generated texts. Our contributions are: a) we build a Human-ChatGPT Polished Paired abstracT (HPPT) dataset, containing polished and original paired abstracts with similarity measurements as the degree of polishing. b) We propose the Polish Ratio to help explain the detection model indicating the modification degree of the text by ChatGPT. c) The experimental results show that our model performs better than other baselines on three datasets. The code and dataset are available: https://github.com/Clement1290/ChatGPT-Detection-PR-HPPT.

Along with the appearance of large language models such as ChatGPT, some detection algorithms are proposed to prevent the abuse of such powerful AI-generated text models. Recent detection methods can be roughly grouped into two categories [22]: white-box and black-box detectors. The white-box detector needs to access the distributed probability or vocabulary of the target language model, while the black-box detector only checks the output text of the target model. Table 1 summarizes recent algorithms that detect ChatGPT-generated texts.

Gehrmann et al. [3] is a popular white-box detector, which takes advantage of the probability and entropy of the linguistic patterns, while DetectGPT [13] utilizes the observation that ChatGPT texts tend to lie in areas where the log probability function has negative curvature to conduct zero-shot detection. Watermark-based detection methods [6, 12] need to divide the vocabulary into a randomized set of words that is softly prompted to use during sampling (“green” tokens) and the remaining words (“red” tokens), and modify the logits embedding when the model generates the results.

Training another classifier as the detector with labeled data [4, 14, 15] is a mainstream black-box method. Guo et al. [4] finetunes the Roberta on the HC3 (Human ChatGPT Comparison Corpus) dataset to obtain an effective detector. However, the simple ChatGPT-generated texts in the HC3 dataset make the model trained on it vulnerable to being attacked using the polishing strategy, and the robustness is not ensured. Moreover, some effective detectors are not open-source and are directly used for commercial operations, such as GPTZero² and OriginalityAI.³

On the other hand, the existing black-box detectors rarely provide explanations for the prediction. Ribeiro et al. [18] proposed Local Interpretable Model-agnostic Explanations (LIME) to explain the predictions of any classifier in an interpretable and faithful manner by learning an interpretable model locally around the prediction. Lundberg and Lee [11] proposed the SHapley Additive exPlanations (SHAP) method to assign each feature an importance value for a particular prediction. Although they perform effectively on basic NLP tasks such as sentiment analysis [5], they do not provide a convincing explanation to users in ChatGPT detection task [14], compelling us to step forward and address this gap.

To facilitate detecting ChatGPT-polished texts and offer more intuitive explanations to assist final judgment, we first collect human-written abstracts and polish all of them using ChatGPT forming Human-ChatGPT Polished Paired abstracT (HPPT) dataset. Additionally, for each abstract pair in the dataset, we furnish three different similarity metrics (Jaccard Distance, Levenshtein Distance, and BERT semantic similarity) between the human-written abstract and the corresponding abstract polished by ChatGPT. Based on the data, we train the Roberta model as the detector to conduct the detection task. Meanwhile, we also train the Polish Ratio model to explain the detection result to reveal the degree of ChatGPT involvement. The overall detection process is shown in Figure 1.

Since the existing datasets are specifically designed for purely ChatGPT-generated detection, such as HC3 [4] and ignore the ChatGPT-polished texts, we first construct the Human-ChatGPT Polished Paired abstracT (HPPT) dataset. Specifically, we collect human-written abstracts of accepted papers from several popular NLP academic conferences and polish all of them using ChatGPT.⁴ The texts in our dataset are paired, making it easy to observe the difference between human-written and ChatGPT-polished texts. Overall, we collect 6050 pairs of abstracts and corresponding polished versions from the ACL anthology (ACL, EMNLP, COLING, and NAACL) in the past five years (2018-2022): 2525 are from ACL, 914 are from EMNLP, 1572 are from COLING, and 1039 are from NAACL.

To uncover the distinctions between human-written and ChatGPT-polished texts, we compute their similarities using three metrics: BERT semantic similarity,⁵ Jaccard Distance, and Levenshtein Distance (details are presented in Section 3.3). As shown in Figure 2, the paired texts are semantically similar in BERT semantic similarity, which shows it is difficult to distinguish human-written and ChatGPT-Polished abstracts from BERT-based semantics. But Jaccard Distance and Levenshtein Distance provide a better way to distinguish them as they align closely with the Gaussian distribution, making them suitable for measuring the degree of ChatGPT involvement.

Following the previous work [4], we treat the detection as a binary classification task and build a black-box detector utilizing the Roberta model [10] because the model of Chat GPT is not accessible, as discussed in the Section 2. Unlike them, we regard the original abstract without polishing as human-written, and its corresponding ChatGPT polished abstract is regarded as Chat GPT involved.

Due to the black box nature of many existing detectors, they are unable to provide explanations that are crucial for differentiating ChatGPT-Polished text. Therefore, we employ two independent explanation methods GLTR (Giant Language model Test Room) and Polish Ratio (PR). This approach is favored over posterior explanation methods such as SHAP, ensuring more comprehensive justifications for our final judgments.

• GLTR: Giant Language model Test Room [3] is a statistical method used for AI-generated text detection (mainly for GPT-2). It assumes that human-written text uses a wider subset of the distribution under a model. Although it was initially designed for GPT-2-generated text detection, we formulate a hypothesis that the distribution of GPT-2 and ChatGPT-generated texts is similar in some way since both are AI-generated texts. It uses three criteria to help detection: (1) The probability of the word, as shown in Equation 1; (2) The absolute rank of a word that is the rank of the Equation 1; (3) The entropy of the predicted distribution, as shown in Equation 2.

Specifically, if a text is written by a human, a word should have a low probability, which leads to a higher top rank and the entropy also should be large.

We consider the GLTR as our baseline for the explanation method as we have discovered that the method is effective in explaining the distinction between human-written and entirely ChatGPT-generated texts. However, it fails to provide adequate explanations when ChatGPT-polished texts are present (refer to Section 4.3 for further details). To address this issue, we adopt a new method called the Polish Ratio.

* PR: Polish Ratio is a new metric we propose to measure the degree of ChatGPT involvement for a text. It measures the degree of text modification before and after revision using ChatGPT by calculating the similarity score between original human-written texts and the corresponding polished texts. In our dataset HPPT, we take two metrics Jaccard Distance and Levenshtein Distance (Levenshtein Distance is normalized by the maximum length of the two sequences) as the Polish Ratio. Since ChatGPT generates the text using tokens, we evaluate the distances based on white-space tokenized words. As shown in Equation 3, Jaccard distance measures the dissimilarity between sets by comparing the size of their intersection and union. Levenshtein distance measures the difference between two sequences by counting the minimum number of edits (deletion, insertion, and substitution), as shown in Equation 4.

Where:

• - A, B are the set of words in the document text strings.

• - | · | represents the cardinality (number of elements) of the set.

Where:

• - a, b represent a list of white-space tokenized words in the document.

• - | · | represents the length of the word list.

• - lev(a[i :],b[j :]) represents the Levenshtein distance between the sublist of a starting at index i and b starting at index j.

Therefore, we regard the PR model as the regression model where either the Jaccard distance or normalized Levenshtein distance of the polished texts is the target value of the Polish Ratio. In ideal conditions, the predicted PR value of an abstract should approach 0 for a human-written one and should be close to 1 when ChatGPT revises a majority of words in the abstract. We use the Roberta-based model to extract feature x_i ∈ ℝ⁷⁶⁸ of each sentence i and use an MLP to conduct the final regression task. The sigmoid function is chosen as the activation function to ensure the range of the regression result is in [0, 1]. Compared to other explanation indices like confidence level, our PR method takes advantage of the paired abstracts before and after polishing to measure how much the ChatGPT involvement is, which can give a more independent and convincing explanation.

We conduct experiments on the following three datasets to demonstrate the effectiveness of our model.

• HPPT

  It is our built ChatGPT-polished dataset, which consists of 6050 pairs of recent abstracts and corresponding polished versions. Meanwhile, to measure the degree of ChatGPT involvement in the text, we also provide the Levenshtein distance and Jaccard distance of the polished abstracts compared with their corresponding human-written ones as the labeled PR value and label 0 as the PR value of those human-written abstracts. We randomly partition the HPPT into the train, test, and validation sets by 6:3:1 to train and test our model (Roberta-HPPT).

• HC3 [4]

  The HC3 (Human ChatGPT Comparison Corpus) dataset⁶ is one of the most popular ChatGPT detection datasets containing question-answering QA pairs. The human answers are collected from publicly available question-answering datasets and wiki text, while the answers provided by ChatGPT are obtained from its preview website through manual input of questions for each interaction. We choose its English version corpus, which consists of 85,449 QA pairs (24,322 questions, 58, 546 human answers, and 26,903 ChatGPT answers). We randomly partition the HC3 into the train, test, and validation sets by 6:3:1 and regard the answer text as the input of our detection model to ensure the detector’s versatility. In addition, it is also used to train our baseline model (Roberta-HC3).

• CDB [9]

The ChatGPT-Detector-Bias Dataset⁷ (CDB) is a mixed dataset consisting of 749 text samples derived from human, ChatGPT and GPT-4. The human data is from TOEFL essays in a Chinese educational forum, US College Admission essays, and scientific abstracts from Stanford’s course CS224n. The GPT data consists of samples polished by both ChatGPT and GPT-4, as well as essays fully generated by ChatGPT using different prompts. We also take it as the harder dataset to test the detectors’ generalization ability because it not only contains the GPT-4-generated or GPT-4-polished text but also contains well-designed prompt engineering ChatGPT-generated text and the human writing samples from both native and non-native English writers.

During training our detection model, we use the batch size 16, learning rate 2e – 5 and the maximum epoch is set to 10. The model is chosen as the best one on the validation set. While training our PR model, we use batch size 4, learning rate 1e – 5, and the maximum epoch is set to 15.

Table 2 shows several baselines and our model performance on three datasets. Our model performs well on both in-domain dataset HPPT and out-of-domain datasets (HC3 and CDB), suggesting that our model trained on the polished HPPT dataset is more robust than other models. Although the Roberta-HPPT model is only trained on HPPT, it achieves comparable performance compared to the SOTA model in HC3, with only a 3% difference and better than DetectGPT. On the more difficult dataset (CBD), our model significantly outperforms other baselines with 88.15% accuracy. Specifically, our model only drops 6% on the out-of-domain dataset while the Roberta-HC3 and the DetectGPT drop by nearly 40%, demonstrating the strong robustness of our model. The reason is that our model is trained on the ChatGPT-polished text instead of ChatGPT-generated text, which can tackle more difficult samples such as GPT-4-generated, GPT-4-polished, and well-designed prompt engineering ChatGPT-generated texts.

Although our model achieves high accuracy in detecting ChatGPT-polished texts, it still needs an explanation of the degree of ChatGPT involvement in the text. Motivated by this, we adopt two explanation methods (GLTR and Polish Ratio) to measure them.

Figure 3 shows the visualization of the probability, absolute rank, and the distribution’s entropy of two pairs of texts from HC3 and HPPT. A higher concentration of green tokens indicates that the words are selected from the top-ranked probability generated by the LLM system. Comparing samples 1 and 2, we find that the difference between human-written and ChatGPT-generated texts is noticeable. Texts entirely generated by ChatGPT prefer to use those top-rank probability words, which validates our hypothesis in Section 3.3. However, it is relatively hard to distinguish human-written and ChatGPT-polished texts, as illustrated in samples 3 and 4. The reason is that the fraction of top-rank probability words employed in polished texts decreases significantly, approaching that of human-written texts. As a result, the GLTR explanation mechanism proves challenging in this scenario.

Therefore, we propose Polish Ratio to explain the detection result of both entirely ChatGPT-generated and ChatGPT-polished texts. As shown in

Table 3, we try Jaccard distance and Levenshtein distance as the true label of the Polish Ratio respectively. Considering the influence of outliers and the unbalanced samples because of 0 PR ones (human-written abstracts), we try different loss functions to test their effectiveness. Suppose the target value for the Polish Ratio of each text sample i is y_i and the predicted value is $y^i$⁠, the different losses l_i for sample i are defined as:

where ℒ₁ is Ll-loss, Smoothℒ₁ is smooth Ll-loss and ℒ_MSE is the mean square error while the loss for all N samples is calculated as $ℓ(y,y^)=1N∑i=1Nℓi$⁠. Through the experiment, we find that MSE is the best choice.

The different distributions of PR values given by our model on test data in Figure 4 show that our PR explanation model demonstrates strong generalization capabilities. It not only effectively distinguishes between human-written and polished texts but also successfully discriminates ChatGPT-generated texts that are not encountered during the training stage. While both Jaccard distance and Levenshtein distance serve as suitable PR metrics to differentiate the CP and CG groups, the use of Levenshtein distance proves to be more effective, where the median Polish Ratio of ChatGPT-polished texts is around 0.5, while that of ChatGPT-generated texts is around 0.65. According to the violin plot shown in Figure 4b, we suggest a Polish Ratio greater than 0.2 indicates ChatGPT involvement, and a value greater than 0.6 indicates that ChatGPT generates most words.

Our Polish Ratio is also accurate in the case study. Table 4 is the case study of the Polish Ratio regression on the index Levenshtein Distance. Indeed, it can effectively differentiate the extent of modification introduced by ChatGPT.

For instance, the degree of text modification by ChatGPT varies significantly between text (1b) and text (2b), as highlighted in red, leading to a notable difference in their PR values, which proves the effectiveness of our PR method.

We conduct an error analysis to gain insights into our model’s learning process and assess the efficacy of our explanation methods. We focus on understanding the reasons behind the mistakes the baseline model and our model made in the HPPT dataset, shown in Table 5.

The result shows that the baseline model only trained on the purely ChatGPT-generated text exhibited confusion when encountering a polishing attack, as evidenced by the classification of nearly all polished samples as human-written. In contrast, our model (Roberta-HPPT) tends to make mistakes for originally human-written texts, as shown in Table 5. By exploring the Polish Ratio of these misclassified samples, we find misclassified samples have a relatively high PR which originally should be close to 0 PR for human-written samples, as shown in Figure 5. This discovery indicates that our PR method can explain the reasons why the detection model makes mistakes: the writing style of original human-written texts may be similar to that of ChatGPT.

The framework of our ChatGPT involvement detection method also works effectively on Chinese texts. We collect 1,232 abstracts from the CSL: a large-scale Chinese Scientific Literature dataset [8] that contains 396,209 Chinese papers’ meta-information including title, abstract, keywords, academic category and discipline. Then we use ChatGPT to polish⁸ the selected 1,232 Chinese abstracts and calculate the Jaccard Distance and Levenshtein Distance (normalized) between the polished and the corresponding human-written abstract based on Chinese characters. The 1,232 pairs of abstracts are randomly partitioned into the train, test, and validation sets by 6:3:1. Following the framework mentioned in Section 3, we train the detection and PR model⁹ to investigate whether our proposed method is generalizable to the Chinese language. Through experiments, we discover that the detection accuracy (98.24%) significantly outperforms DetectGPT (53.25%) on our collected Chinese test sets, and our PR model can also distinguish human-written, ChatGPT-polished, and ChatGPT-generated texts¹⁰ effectively as shown in Figure 6.

Our proposed “Polish Ratio” method can also be generalizable to other LLMs like Llama2 [23]. Following the framework shown in Figure 1, we replaced the ChatGPT with the Llama2-7b model and polished the same abstracts in HPPT. Under the same experiment settings as reported in section 4.1.2, our method can effectively distinguish human-written (HW), Llama-polished (LP), and Llama-generated (LG) texts, as shown in Figure 7. Both results suggest that our proposed method can be generalized to different languages and work on different kinds of powerful LLMs.

In this paper, we focus on building a more robust ChatGPT detector with independent explanations by detecting the ChatGPT-polished text. We first develop a dataset called Human-ChatGPT Polished Paired abstracTs (HPPT). Then we train a Roberta-based model that can distinguish human-written texts and ChatGPT-polished texts. We also propose a Polish Ratio method to indicate the degree of ChatGPT involvement in the text. Experiments demonstrate that our model achieves better robustness than the baselines. Equipped with the help of the robust detector and Polish Ratio explanation we proposed, users can make an accurate and convincing judgment of the suspected texts. In the future, we will focus on investigating the specific patterns ChatGPT prefers to use from the perspective of some dynamic features of the text.

This research is supported by the internal project of Shenzhen Science and Technology Research Fund (Fundamental Research Key Project Grant No. JCYJ20220818103001002), and the Internal Project Fund from Shenzhen

Research Institute of Big Data under Grant No. T00120-220002.