Page 101 - AIH-2-4

Artificial Intelligence in Health AI vs humans in clinical code conversion

of codes that could be processed simultaneously without 2.3. Phase 3: Conversion of codes from the
increasing the likelihood of errors, such as incorrect Australian extension of the Systematized
conversions, fabricated inputs, or skipped entries. To Nomenclature of Medicine Clinical Terms using
manage this, codes were processed in batches of five, with Claude 3.5 Sonnet
five batches (25 codes in total) converted at a time. At the time of the study (September 2024), Claude

The following final prompt was used: 3.5 Sonnet was unable to import or export Microsoft
26
Excel files. Therefore, data were copied and pasted directly
Please manually and sequentially convert the into the chat as a means of input. However, there were
SNOMED-CT-AU codes into ICD-10-CM codes and limitations on the maximum allowable message length.
names which have not been converted yet. If you need As a result, the data were entered in three batches of 500,
to, complete this task in batches of 5. Take as long as you followed by a final batch of 476.
need. Do not hallucinate, and only convert codes which
have been provided to you. Do not create new codes to The prompt was developed in a similar manner to
convert. Provide an update after 5 batches have been Phase 2, using an iterative approach. Claude 3.5 Sonnet
completed. Provide a.xlsx file at the end. required fewer instructions to accurately perform the
task, allowing the prompt to be developed relatively
Figure 1 presents the final prompt and the corresponding quickly. However, the limits of output message length
output from ChatGPT-4o. restricted Claude 3.5 Sonnet to processing only 50 codes
at a time.
Following each conversion of 25 codes, ChatGPT-4o
needed to be prompted to continue (Figure 2): During prompt testing, it was noted that Claude
3.5 Sonnet needed to be explicitly instructed not to skip
Please continue. lines, as reflected in the final prompt (Figure 3):

Following conversion, ChatGPT-4o generated the Please convert these SNOMED CT-AU codes and names
requested Microsoft Excel file containing the original input into ICD-10-CM codes and names. Convert them in
data and the corresponding final output data. sequential order starting from the top and ensuring you
do not skip any. I have provided 500 codes in total and
need 500 responses at the end. Start with the first 50. If
you are unable to convert a code, please state this.

Figure 2. ChatGPT-4o is prompted to continue with the next batch of
Figure 1. ChatGPT-4o prompt and output conversions
Abbreviations: ICD-10-CM: International Classification of Diseases, Abbreviations: ICD-10-CM: International Classification of Diseases,
th
10 Revision, Clinical Modification; SNOMED CT-AU: Australian 10 Revision, Clinical Modification; SNOMED CT-AU: Australian
th
extension of the Systematized Nomenclature of Medicine Clinical Terms. extension of the Systematized Nomenclature of Medicine Clinical Terms.

Volume 2 Issue 4 (2025) 95 doi: 10.36922/AIH025200045