Page 79 - AIH-2-2

Artificial Intelligence in Health

ORIGINAL RESEARCH ARTICLE
Optimizing knowledge distillation for efficient

breast ultrasound image segmentation: Insights
and performance enhancement

Bahareh Behboodi * , Rupert Brooks 2 , and Hassan Rivaz 1,3
1
1 Department of Electrical and Computer Engineering, Concordia University, Montreal, Quebec,
Canada
2 Microsoft Canada, Montreal, Quebec, Canada
3 School of Health, Concordia University, Montreal, Quebec, Canada

Abstract
Most modern models designed for ultrasound (US) image segmentation are
characterized by high computational and memory requirements, limiting their
practical utility in point-of-care US settings. Consequently, researchers have devised
innovative approaches to compress these large models, enabling the training of
smaller networks capable of achieving comparable generalization performance.
Among these strategies, knowledge distillation (KD) has emerged as particularly
suitable for scenarios involving small datasets or where significant efficiency
*Corresponding author:
Bahareh Behboodi improvements are desired. While previous KD-based methods have focused on
(b_behboo@encs.concordia.ca) extracting comprehensive information from diverse levels of teacher representation,
Citation: Behboodi B, Brooks R, they often overlook the identification of the most effective representation level.
Rivaz H. Optimizing knowledge Additionally, many existing techniques propose intricate strategies that present
distillation for efficient breast implementation challenges. To address this gap, our study concentrates on selecting
ultrasound image segmentation:
Insights and performance optimal teacher representations from various levels. Through an exhaustive analysis
enhancement. Artif Intell Health. of KD pathways, loss functions, and the impact of augmentation, we offer valuable
2025;2(2):73-86. insights into the mechanisms underlying knowledge transfer from the teacher to
doi: 10.36922/aih.3509
the student networks. Our proposed methodology significantly enhances student
Received: April 26, 2024 performance, elevating the Dice similarity score from 73% to 80%, while the teacher
1st revised: July 31, 2024 model achieves 81%. Notably, our student model achieves this improvement with
only 0.82 million parameters, compared to the teacher model’s 96 million parameters.
2nd revised: August 13, 2024
Accepted: September 4, 2024
Keywords: Ultrasound; Image segmentation; Model compression; Knowledge distillation
Published online: December 20,
2024
Copyright: © 2024 Author(s).
This is an Open-Access article 1. Introduction
distributed under the terms of the
Creative Commons Attribution Ultrasound (US) imaging is one of the most widely used medical imaging modalities,
License, permitting distribution,
and reproduction in any medium, with benefits that include cost-effectiveness, non-invasiveness, portability, and real-
provided the original work is timeliness. However, because of its low quality, the interpretation of US images
properly cited. necessitates professional competence, which must evolve with the variety of imaging
Publisher’s Note: AccScience techniques available. The delineation of an organ or region of interest (ROI) in a US
Publishing remains neutral with image where the pixels inside the intended ROI share certain characteristics is known
regard to jurisdictional claims in
published maps and institutional as US image segmentation. Image segmentation, often known as an important step in
affiliations. many computer-aided detection (CAD) pipelines, aids further quantitative analysis of

Volume 2 Issue 2 (2025) 73 doi: 10.36922/aih.3509