Background: Large language models (LLMs) may help to diversify authorship in scientific journals by supporting non-English speaking researchers in writing, revising, and editing scientific papers.

Objectives: To quantify the frequency of AI-generated text in the dermatology literature and to relate these results to the geographic diversity of authorship.

Methods: We extracted abstracts of 4573 articles published in 21 dermatology journals in March 2024 from 2017 to 2024. We identified AI-generated content using an AI-detector and adjusted the raw rates to account for false positives. Additionally, we computed diversity indices to quantify temporal trends in the geographic distribution of the affiliations of first authors.

Results: We found that the raw rate of AI-generated abstracts remained relatively stable from 2018 to 2023 but exhibited a significant increase in March 2024. In March 2024, the raw rate of AI-generated abstracts was 33.8% (95% CI: 30.0% to 37.8%) and significantly higher than in any preceding year. After adjusting for false positives, the proportion of AI-generated abstracts remained relatively stable below 5% from 2018 to 2023 but jumped to 17.9% (95% CI: 14.9%–21.3%) in 2024. There was a positive correlation between the rate of AI-generated abstracts and the journal’s impact factor for articles published before 2024, with a correlation coefficient of 0.42 (95% CI: -0.01 to +0.72, P-value = 0.06). Regarding the geographic distribution of the affiliations of first authors, both the Shannon and Simpson diversity indices showed a decrease in 2024 compared to the baseline year of 2017.

Conclusions: Our data suggest that the year 2024 indicates a turning point in the use of AI-generated text in the dermatology literature, with the occurrence of AI-generated text increasing significantly compared to previous years. However, the increasing adoption of AI tools alone is not sufficient to enhance the diversity of scientific output in specialized fields such as dermatology.

Why was the study undertaken? This study was performed to quantify the frequency of AI-generated text in the dermatology literature and to relate these results to the geographic diversity of authorship.

What does this study add? This study shows that the prevalence of AI-generated texts in dermatological literature has increased significantly in 2024 but that this increase has not been accompanied by a greater geographic diversity of scientific contributions in the field.

What are the implications of this study for the understanding of skin physiology and pathology and/or disease management? The influence of scientific literature on funding priorities and drug development is profound and a lack of diversity impacts disease understanding and clinical care. Despite a historical peak in its usage, LLMs did not improve the visibility of countries traditionally underrepresented in the dermatology literature.

Artificial intelligence, dermatology, large language models

Over the last few years, the use of large language models (LLMs) such as GPT-4, LLama, and BERT has significantly influenced various aspects of academic medicine and scientific writing. These models understand and generate human-like text, assisting researchers in drafting research papers and summarizing complex data [1]. In the current scientific landscape, English dominates as the primary language of communication. Consequently, most scientific papers are written in English, leading to challenges for non-native speakers who may struggle with clarity and complexity in their writing. This situation contributes to global inequalities, particularly in countries with limited English education, resulting in these researchers having fewer opportunities to get their papers accepted in top-tier journals [2]. LLMs can help diversify authorship in scientific journals by supporting researchers for whom English is not the first language, aiding in writing, revising, proofreading, and editing to improve readability [2–4].

However, the use of these tools in academic work raises ethical concerns [1], and risks such as errors, AI-inherent bias, and incorrect quotations are significant [5]. Additional limitations include a lack of transparency and proper disclosure [6]. Current policies of major dermatology journals like the Journal of the American Academy of Dermatology (JAAD), the Journal of the American Medical Association Dermatology (JAMA), the British Journal for Dermatology (BJD), the Journal of the European Academy of Dermatology (JEADV) [7], and the Journal of Investigative Dermatology (JID) [8] concur that AI tools cannot be named as authors [9–11]. These tools do not meet the ICMJE authorship criteria, as they cannot disclose conflicts of interest or handle copyright and licensing agreements. Human involvement remains crucial due to the unique responsibilities involved in authorship [12]. Nevertheless, the use of AI and AI-assisted technologies is permitted in the writing process, provided their use is disclosed.

Understanding the dynamics of language barriers and their impact on scientific publishing in dermatology is essential for promoting inclusivity in the field. Although the application of LLMs in scientific writing is widely acknowledged, literature describing the prevalence of AI-generated text and its influence on the diversity of authorship remains absent. This study aims to quantify the prevalence of AI-generated text in the dermatology literature over time and examine its relationship with authorship diversity.

Our data suggest that 2024 marks a turning point in the use of AI-generated text within the dermatology literature. This year, we observed a sharp increase in the proportion of AI-generated abstracts in our field. Both the raw and adjusted counts showed significant rises compared to previous years. After accounting for false positives, we estimated that the probability of encountering AI-generated abstracts in the dermatology literature is around 18%, a significant increase from the rate of well below 5% observed before 2024. Several developments in 2023 likely contributed to this increase in the use of LLMs in academic writing. Notably, in 2023 there was a high density of new releases and updates regarding LLMs. In February 2023, Meta released LLaMA (Large Language Model Meta AI), OpenAI launched GPT-4, and in May, Google released PALM-2 (Pathways Language Model 2). Not only has the range and quality of LLMs improved, but the threshold for using these models among researchers may have decreased. This shift can be attributed to the increasing familiarity with LLMs and the decision by most academic journals to provide clear guidelines for their use rather than imposing outright bans.

Despite the increased usage of LLMs, there are only a limited number of studies quantifying their usage in medical scientific literature. Bisi et al. compared the rate of AI-generated text before and after the release of ChatGPT in November 2022 in a single orthopaedic journal. They found an increase in AI-generated text from 10.3% to 15.6%. However, they also noted that AI-generated text was primarily detected in abstracts. This finding led us to focus our analysis on abstracts only [13].

The potential advantages of LLMS in medical writing are evident. They may assist researchers by drafting sections of their papers and improving productivity, and they may enhance language quality, grammar, and style, especially for non-native English speakers. Therefore, LLMs can play a crucial role in diversifying authorship in scientific journals and may help to address global inequalities in scientific publishing. As scientific literature typically reflects the predominant research focus in the field, a lack of diversity among regions can skew our understanding of skin diseases and affect clinical care. This issue is particularly pronounced in the field of dermatology, where certain skin diseases that are more prevalent in underrepresented countries do not receive the attention they deserve. Consequently, research on these underrepresented skin diseases often faces lower levels of funding and interest. This lack of attention can delay or even prevent the development of new treatments and drugs, exacerbating health disparities. Our analysis suggests that the use of LLMs alone is not sufficient to enhance the diversity of scientific output in dermatology. Instead, it may even widen the gap between well-funded, technologically advanced countries and those with fewer resources. Instead of an increase in diversity, we found a shift in the landscape of publication dominance. Originally dominated by the US, there is now a bipartite dominance that includes China, which may have benefited from LLMs supporting non-English speaking authors. However, the share of publications from other countries has remained stable and the overall diversity did not increase but rather decline. Interestingly, we observed an increase in diversity in scientific publications during the COVID-19 pandemic, reflecting the global perspective and widespread impact of this disease. This trend highlights how a global health crisis can bring attention to diverse perspectives and regions that are typically underrepresented. However, our data suggest that the use of LLMs did not play a significant role during the COVID-19 pandemic.

It is important to note that LLMs may also introduce certain negative effects. Currently, AI systems primarily work with freely accessible databases, which excludes subscription journals. This limitation can lead to an overemphasis on publicly accessible abstracts and data, potentially skewing the representation of available research. Moreover, LLMs have limited ability to verify the accuracy of sources, which can result in the inclusion of incorrect or fabricated data. Therefore, it is crucial to ensure that LLMs do not misinterpret or misrepresent scientific data when utilizing these tools. Despite their potential to diversify scientific publishing and attract a broader range of researchers, LLMs might also negatively impact scientific writing. The concern is that scientific writing could become sterile and uniform, lacking the idiosyncratic features that make reading enjoyable.

Our study has several limitations. First, we used an AI-detector to identify AI-generated text, which may produce false positives. To mitigate this effect, we used the year 2017, a period prior to the widespread availability of LLMs, to set a baseline for false positives and adjusted the rate of detected AI-generated text accordingly. We did not use human reviewers because it has been shown that researchers themselves are often unable to distinguish between LLM-generated and human-written texts. Second, we analyzed abstracts from dermatology literature only, rather than scanning entire papers. The proportion of AI-generated text may be much lower in full articles compared to abstracts. Third, our analysis focused on abstracts from March of each year, which excluded some quarterly dermatologic journals. Finally, it is important to emphasize that the detection of AI-generated text does not disqualify the scientific content of an article or the originality of its authors. LLMs may have been used solely for text polishing and improvement of grammar and style, which, in our opinion, is entirely acceptable.