As the field of artificial intelligence (AI) continues to advance, there is an increasing concern about the quality and originality of research documents generated by language models (LLMs). In a recent study, 13 experts were asked to evaluate a set of 50 research documents generated by LLMs, with the goal of identifying similarities between these documents and existing work. The results were alarming: 24% of the evaluated documents were found to be either paraphrased or significantly borrowed from existing work, without proper citation or acknowledgement of the original sources.

To better understand the scope of this problem, the authors of the source papers were cross-verified the identified instances of plagiarism. The remaining 76% of documents showed varying degrees of similarity with existing work, with only a small fraction appearing completely novel. These findings highlight the need for more rigorous evaluation and citation practices in AI-generated research documents.

The study also demonstrated that automated plagiarism detectors are inadequate at catching plagiarized ideas from LLMs. This is concerning, as these tools are often relied upon to identify potential instances of plagiarism in academic work. The findings suggest that more sophisticated methods may be needed to effectively address the issue of plagiarism in AI-generated research documents.

The implications of this study are far-reaching, as it highlights the need for greater attention to originality and citation practices in AI-generated research. It also raises questions about the reliability of automated plagiarism detectors and the need for more robust evaluation methods in the assessment of academic work generated by LLMs.

The study underscores the importance of rigorous evaluation and citation practices in AI-generated research documents, and highlights the limitations of current plagiarism detection tools. As the use of LLMs continues to grow, it is crucial that we address these concerns to ensure the integrity and quality of academic work in this field.