A newly released study conducted in collaboration with New York University Abu Dhabi (NYUAD) shows that when answering assessment questions, ChatGPT’s performance was either similar to or surpassed that of university students in 9 out of 32 courses. While approximately 74% of students from five different countries would consider using ChatGPT for academic aid, a significant 70% of educators regard such usage as constituting academic dishonesty.
The study, published in the journal Scientific Reports, examined ChatGPT’s capabilities in answering questions for academic assessments in various subjects, including computer science, political science, engineering, and psychology. It found that the AI’s performance may be equivalent to, or even exceed, that of the average college student. Interestingly, nearly three-quarters of the surveyed student population expressed a willingness to use ChatGPT for academic tasks, despite the prevailing educational viewpoint that using the technology constitutes plagiarism.
To undertake this investigation, researchers Talal Rahwan and Yasir Zaki engaged faculty members from diverse disciplines at NYUAD. They collected three student-written answers for each of ten exam questions created by the faculty. ChatGPT was then prompted to produce three separate sets of responses to these ten questions. These responses were evaluated alongside the student submissions by three independent graders who were not informed about the origin of the answers. The outcomes revealed that ChatGPT received an equal or higher average grade than students in 9 out of 32 courses examined. Only in courses related to mathematics and economics did students consistently outscore ChatGPT. The software performed most notably in the course titled “Introduction to Public Policy,” securing an average grade of 9.56, which was substantially higher than the student average of 4.39.
Moreover, the researchers polled 1,601 individuals across Brazil, India, Japan, the United States, and the United Kingdom to gather perspectives on the acceptability of employing ChatGPT for academic assignments. This sample included a minimum of 200 students and 100 educators from each country. While 74% of the student respondents indicated a willingness to use ChatGPT, educators across all countries consistently underestimated this percentage. Additionally, 70% of the educators stated they would categorize the use of ChatGPT as plagiarism.
The study also assessed the effectiveness of two tools specifically designed to identify AI-generated text: GPTZero and an AI text classifier. These tools erroneously classified ChatGPT’s responses as human-written 32% and 49% of the time, respectively.
Collectively, these findings could serve as valuable data for shaping policies regarding the utilization of AI technologies in educational contexts.
Reference: “Perception, performance, and detectability of conversational artificial intelligence across 32 university courses” by an extensive team of researchers, published on August 24, 2023, in Scientific Reports.
DOI: 10.1038/s41598-023-38964-3
Table of Contents
Frequently Asked Questions (FAQs) about ChatGPT Academic Performance
What does the recent research collaboration with NYUAD reveal about ChatGPT’s performance?
The collaboration demonstrates that ChatGPT’s performance is comparable to or better than university students in 9 out of 32 courses when responding to academic assessments.
How do students and educators view the use of ChatGPT for academic tasks?
Approximately 74% of students express willingness to use ChatGPT for academic assignments. In contrast, 70% of educators perceive its usage as constituting plagiarism.
What were the key findings of the study regarding ChatGPT’s performance in specific courses?
ChatGPT’s performance excelled in 9 of the 32 courses examined, achieving average grades similar to or higher than students. Notably, it stood out in the ‘Introduction to Public Policy’ course.
What was the approach taken to compare ChatGPT’s responses with student-written answers?
Faculty members provided three student submissions for ten assessment questions, and ChatGPT generated responses to the same questions. Independent graders evaluated both sets of answers, unaware of their source.
How did the study assess the detectability of ChatGPT’s answers as AI-generated text?
Two tools, GPTZero and an AI text classifier, were used to identify AI-generated text. They misclassified ChatGPT-generated answers as human-written 32% and 49% of the time respectively.
What geographical areas were covered in the study’s participant pool?
The study gathered perspectives from individuals in Brazil, India, Japan, the United States, and the United Kingdom. The sample included students and educators from each of these countries.
What implications can be drawn from these findings?
The research provides valuable insights for shaping policies concerning the integration of AI tools in educational settings, considering the performance of ChatGPT and the differing viewpoints of students and educators.
More about ChatGPT Academic Performance
- Scientific Reports: “Perception, performance, and detectability of conversational artificial intelligence across 32 university courses”
- New York University Abu Dhabi (NYUAD)
- GPTZero: Identifying AI-generated text
- AI text classifier for identifying AI-generated text
