More and more scientific articles are being written with the support of generative artificial intelligence. Language models such as ChatGPT are becoming a tool not only in the lab, but also… at the keyboard. In computer science, almost one in five abstracts and introductions in 2024 publications have already been written by AI. Does this revolution signify an acceleration in the pace of innovation, or does it herald the devaluation of scientific writing as a measure of intellectual value?
Phenomenon: science written by machines
A team of Stanford University researchers analysed more than 1.1 million scientific publications and preprints from three main sources: arXiv, bioRxiv and Nature. The aim was to investigate the presence of linguistic generative models (LLMs) in scientific content. The results leave no illusions: in less than two years, the share of AI-generated texts has increased by leaps and bounds.
In computer science in 2024, as many as 22.5% of abstracts and 19.5% of introductions were generated or significantly modified by LLM. This compares with only 2.4% in November 2022. A similar phenomenon is also observed in electrical and systems sciences (18-18.4%) and, to a lesser extent, in mathematics (8.9-9.4%).
It is in the opening sections of articles – the abstracts and introductions – that the greatest LLM activity is concentrated. These are the sections where language models do best: they require a synthetic account of the problem, an efficient summary of the content and linguistic fluency, and not necessarily an original conceptual contribution. In the methodological and results sections, the presence of AI is noticeably less.
Motivations: speed and pressure
Behind the increase in the use of AI in writing are not just technological capabilities, but also systemic factors. Authors who publish preprints more frequently are much more likely to use LLM – probably because of the increasing pressure on publishing speed.
In the scientific world, especially in technological disciplines, the number of publications increasingly translates into prestige, funding and promotion. Generative models are therefore becoming a way to increase ‘throughput’ – they speed up the editing of texts, improve the English language, and help to build a compelling message. In texts under 5,000 words, their presence is particularly evident.
At the same time, there are clear geographical differences. Articles from China and continental Europe are more likely to contain passages written by AI than those from North America or the UK. Non-English-speaking authors are more likely to use LLM as a linguistic support, which, the researchers suggest, may be due to limited access to language editors or the need to optimise the scientific message quickly.
The problem: quality versus originality
However, the automation of scientific writing raises questions deeper than just those related to efficiency. The main concerns relate to the quality, authenticity and originality of scientific texts.
Does the AI-generated abstract still represent the author’s unique contribution? Does an introduction written by an LLM meet the criteria of scientific creativity, or is it just a linguistically correct compilation of existing concepts? And what about responsibility for the content – does the scientist signing off on the machine-generated text still remain its sole author?
Also of concern is the issue of ‘taming’ models – academics may unconsciously adapt the language and structure of publications to the patterns favoured by LLMs, thus making academic style standardised, automated and vulnerable to manipulation.
Risks: content inflation and dilution of reviews
In the long term, this can lead to so-called content inflation – a flood of correct but secondary publications. Language models facilitate the production of a large number of articles that meet formal criteria but do not contribute substantive value. If the pace picks up and the tools become more widespread, the scientific review system will face a huge challenge.
Reviewers may find it difficult to judge whether a text has been written by a human, by AI or jointly. Furthermore, blurring authorship may make it difficult to identify a researcher’s actual contribution, which in turn undermines trust in the publication process as a mechanism for qualitative selection.
New standards and transparency
Currently, few journals or scientific platforms clearly regulate the use of AI for writing. However, there are calls for transparency rules – similar to declaring conflicts of interest or sources of research funding.
Some in the academic community are calling for the use of LLM to be openly declared: whether in a footnote or in a special ‘authors and tools’ section. There are also ideas of classifying the LLM as a form of assistant, similar to a language editor – albeit with much more influence on the content. How quickly and in what form the new standards are developed will determine whether AI becomes an ally of quality or a catalyst for its erosion.
