Paper written using ChatGPT demonstrates opportunities and challenges of AI in academia

Released: 24.03.23

ChatGPT has the potential to create increasing and exciting opportunities – but also poses significant challenges – for the academic community, according to an innovative study written in large part using the software.

Launched in November 2022, ChatGPT is the latest chatbot and artificial intelligence (AI) platform touted as having the potential to revolutionise research and education.

However, as it becomes ever more advanced, the technology has also prompted concerns across the education sector about academic honesty and plagiarism.

To address some of these, the new study directly uses ChatGPT to demonstrate how sophisticated Large Language Machines (LLMs) have become but also the steps that can be taken to ensure its influence remains a positive one.

Published in the peer-reviewed journal Innovations in Education and Teaching International, the research was conceived by academics from Plymouth Marjon University and the University of Plymouth.

For the majority of the paper, they used a series of prompts and questions to encourage ChatGPT to produce content in an academic style. These included:

• Write an original academic paper, with references, describing the implications of GPT-3 for assessment in higher education;
• How can academics prevent students plagiarising using GPT-3?
• Are there any technologies which will check if work has been written by a chatbot?
• Produce several witty and intelligent titles for an academic research paper on the challenges universities face in ChatGPT and plagiarism.

Once the text was generated, they copied and pasted the output into the manuscript, ordered it broadly following the structure suggested by ChatGPT, and then inserted genuine references throughout.

This process was only revealed to readers in the paper’s discussion section, which was written directly by the researchers without the software’s input.

In that section, the study’s authors highlight that the text produced by ChatGPT – while much more sophisticated than previous innovations in this area – can be relatively formulaic, and that a number of existing AI-detection tools would pick up on that.

However, they say their findings should serve as a wake-up call to university staff to think very carefully about the design of their assessments and ways to ensure that academic dishonesty is clearly explained to students and minimised.

Professor Debby Cotton, Director of Academic Practice and Professor of Higher Education at Plymouth Marjon University, is the study’s lead author. She said:

“This latest AI development obviously brings huge challenges for universities, not least in testing student knowledge and teaching writing skills – but looking positively it is an opportunity for us to rethink what we want students to learn and why. I’d like to think that AI would enable us to automate some of the more administrative tasks academics do, allowing more time to be spent working with students”

Corresponding author Dr Peter Cotton, Associate Professor in Ecology at the University of Plymouth, added:

“Banning ChatGPT, as was done within New York schools, can only be a short-term solution while we think how to address the issues. AI is already widely accessible to students outside their institutions, and companies like Microsoft and Google are rapidly incorporating it into search engines and Office suites. The chat (sic) is already out of the bag, and the challenge for universities will be to adapt to a paradigm where the use of AI is the expected norm.”

Dr Reuben Shipway, Lecturer in Marine Biology at the University of Plymouth, said:

“With any new revolutionary technology – and this is a revolutionary technology – there will be winners and losers. The losers will be those that fail to adapt to a rapidly changing landscape. The winners will take a pragmatic approach and leverage this technology to their advantage.”

 

Find their full study here.