Page 61 - AIH-1-3

Artificial Intelligence in Health ChatGPT in writing scientific articles

(cardiology, oncology, and remote therapy). We evaluated • It is necessary to indicate the exact number of references
the result of ChatGPT’s work on two aspects: (i) The • It is necessary to specify that ChatGPT can use the
quality of the content obtained and (ii) the reliability of the Internet to search for references. Although only
literature references cited by ChatGPT. version 4 has this feature, to achieve equal initial
conditions for both versions of ChatGPT, a similar
2. Data and methods addition was used with version 3.

2.1. ChatGPT versions in use It is also important to add that a new chat was created to
In this article, versions of ChatGPT 3.5, released on generate a response to each prompt so that ChatGPT would
May 28, 2020, and ChatGPT 4, released on March 14, not be tied to its past responses. No custom instructions
2023, were used. Compared to the 45 terabytes of data or other custom settings were used with ChatGPT. Third-
32
for GPT-3, the latest iteration of OpenAI’s GPT-4 has party plugins and other experimental new features were
1 petabyte of training data. 33,34 As a result, GPT-4 can disabled for ChatGPT 4.
produce significantly more accurate results than GPT-3. Table 1 presents examples of prompts for two versions
In additionally the latest version of the neural network of ChatGPT, using the case of writing a scientific article on
has about 100 trillion parameters, compared to 175 billion the topic of biotelemetry in cardiology. To study the impact
for GPT-3. This feature has enabled the creation of more of prompt formulation on the sources obtained, three types
35
precise formulations and coherent text, which is one of of prompts were formulated, each of which complements
the key parameters for evaluating the performance of the and refines the previous type.
neural network.
2.4. Parameters for the evaluation of articles
2.2. Branches of medicine
Literature sources for the generated articles were considered
According to 2022 statistics, the primary causes of according to the following criteria:
mortality among the population in the United States of • Reality of the source: This criterion assesses the
America (USA) are cardiovascular diseases, cancer, and correspondence of the title, author, and journal to the
COVID-19 infections. Specifically, about 699,000 people real scientific publication. The search was conducted
36
died from conditions related to the cardiovascular system. on the largest academic databases
Similar statistics were reported for 2020 in Europe, where • Impact factor (IF) of the journal: The value of
out of 5.18 million deaths, 33% were due to circulatory this parameter was obtained using the platform
diseases. 37,38 For this reason, fields such as cardiology, scijournal.org. 39
oncology, and clinical medicine, particularly in therapy,
were chosen to explore the potential of writing scientific The assessment of the reality of the source requires
publications using a neural network. The following specific a separate explanation. We categorized all the sources
topics were studied: obtained into three major groups:
• “Biotelemetry in cardiology” (i) Reliable sources (“blue”): These sources are fully
• “Biotelemetry in oncology” reliable sources with correct authors, titles, and
• “Remote medical examination.”
Table 1. Prompts on the topic of biotelemetry in cardiology
2.3. Prompts used and their engineering features for the two versions of ChatGPT
The primary interface for interacting with the ChatGPT No Prompt for GPT-3 and GPT-4
chatbot involves submitting a prompt that entails a detailed 1 Write on your own a short research paper on “biotelemetry in
description of the requirements for the neural network. cardiology.” Do not send me an outline, I need a paper. Be sure
In the study, specific prompts were selected for each of to add references to sources as you write. The total number of
the chosen fields of medicine. Each of those prompts was sources should be at least 10.
tailored for the ChatGPT versions being used. To generate a 2 Write on your own a short research paper on “biotelemetry in
prompt for ChatGPT, the following points were considered: cardiology.” Do not send me an outline, I need a paper. Be sure
• It is necessary to write a research paper to add references to sources as you write. The total number of
sources should be at least 10. Use only real existing sources
• It is necessary to write a research paper “on your own” (you can search for Internet resources).
• It is necessary to write a research paper that is not an 3 Write on your own a short research paper on “biotelemetry in
outline for this paper, and the paper should be short; cardiology.” Do not send me an outline, I need a paper. Be sure
otherwise, ChatGPT outputs only a part of a paper to add references to sources as you write. The total number of
• It is necessary to include references – without specifying sources should be at least 10. Use only real existing sources
this phrase, ChatGPT might not include them (you can search for Internet resources) with a high-impact factor.

Volume 1 Issue 3 (2024) 55 doi: 10.36922/aih.2592