Page 102 - AIH-2-4

Artificial Intelligence in Health AI vs humans in clinical code conversion

As Claude 3.5 Sonnet was unable to directly export a code that could be run to generate an output file (Figure 4).
Microsoft Excel file, it was instead instructed to produce R The following request was made: “Can you write the code
for R to create an Excel file of this data. Write it out in
full so it exports all entries 1 – 100.” Due to limitations
in output message length, this process was completed in
batches of 100. The generated code was then run using R
Studio (Posit, USA) to produce the final output file.

2.4. Statistical analysis
For the purpose of analysis, the ICD codes were split into
three components: (i) letter code (“Level 1”); (ii) major
numeric code (before the decimal point: “Level 2”);
and (iii) minor numeric code (after the decimal point:
“Level 3”) (Table 1).
A pattern-matching program was developed using the
C programming language to identify partial and perfect
matches among the 1,970 cases between: (i) manual
coding and ChatGPT-4o; (ii) manual coding and Claude
Figure 3. Claude 3.5 Sonnet prompt and output 3.5 Sonnet; and (iii) ChatGPT-4o and Claude 3.5 Sonnet.
Abbreviations: ICD-10-CM: International Classification of Diseases,
10 Revision, Clinical Modification; SNOMED CT-AU: Australian The program converted the codes from each method into
th
extension of the Systematized Nomenclature of Medicine Clinical Terms. their component parts. Some manual editing was necessary

Figure 4. Claude 3.5 Sonnet generating R code to create a Microsoft Excel file
Abbreviations: ICD-10-CM: International Classification of Diseases, 10 Revision, Clinical Modification; SNOMED CT-AU: Australian extension of the
th
Systematized Nomenclature of Medicine Clinical Terms.

Volume 2 Issue 4 (2025) 96 doi: 10.36922/AIH025200045