Page 58 - GTM-1-2

Global Translational Medicine A Taxonomy of AI Assisted Medical Robots

possesses the ability to generate candidate structures to endoscope camera provides useful functions, such as
sense different surfaces and materials. The signals collected magnified stereo imaging of the ventricle internal structure,
from the sensing system can be sent to processing neural dynamic segmentation of ventricle parts, and automatic
networks to extract the valid tactile sensing patterns. annotation on the screen for surgical decision-making.
Another challenge to accurate segmentation and annotation
3.2. Multi-sensor fusion is the deformable tissue and organs at the operation

Recognizing and understanding the various behaviors of area. The 2D shape silhouettes of the tissue or organs are
surgeons under different scenarios may improve the cognition extracted from images from a monocular camera to assist
and assistive ability of medical robots. The traditional data the 3D deformable registration models through several
fed to assistive medical robots are often single modal, such as projective constraints of multiview geometry [53,57,76] . The
robotic imaging information from CT or MRI . One of the allocation of communication channels by AI-ART, such as
[16]
focuses of current research on human-robot interaction is 5G, ensures real-time video transfer of the surgery process.
sensor fusion based on multi-modality information collected This lays a solid foundation for telemanipulated medical
during or after the surgery process. For instance, in a surgery robots. Simulated sensing is obtained through the tactile
with the assistance of a laparoscopic robot, the robot must sensing technique equipped on these robots. AI-ART also
be able to comprehend the term “hemostasis” spoken by the enables multi-views of a patient through cameras mounted
surgeon, segment the images collected by the camera sensors at different positions. The fusion of these multi-views, in
mounted on the robot and obtain semantic understanding which the images are stitched together, forms a panoramic
of organs or tissues of the patient, recognize the bleeding view by convolutional neural network (CNN) [50,78] .
vessels, and use the correct size of hemostatic forceps to 4. Challenges and directions
implement the action within a short period of time. The
construction of a multimodal information framework is 4.1. Future challenges
necessary for flexible interfaces of various sensor types, The past decades have witnessed the tremendous progress
accuracy improvement, haptic sensation, diverse vital signs, of AI-ART in the fabrication of various medical robots, such
surgeons’ spoken words and gestures, and decoupling of as laparoscopy surgical robots, single-port laparoscopy
internal modules and external software interfaces. Concrete robots, naked-eye imaging laparoscopy, capsule robots,
gesture recognition incorporates the kinematics of grippers, wearable medical robots, and so on, as shown in Figure 7.
grasping different tissues, with proper fine-grained tissue or Due to the breakthrough of parallel computation
organ segmentation under different surgical types [77,78] . Multi- chips, such as graphics processing unit (GPU) and field
sensor fusion has greatly contributed to the commercial programmable gate array (FPGA), as well as artificial
translation from AI-ART, such as the combination of CT intelligence algorithms like multi-layer perceptron (MLP),
and MRI to obtain more precise spatial scanning results in convolutional neural network (CNN), deep learning (DL),
relation to the location of lesions. However, CT and MRI and the latest knowledge distillation techniques, various
images require diverse configuration parameters under medical therapies and applications have been uncovered.
different conditions to ensure a better fusion. Therefore, more
emphasis should be placed on adaptive fusion by AI-ART. New surgical tool manipulation modeling and
navigation methods are made feasible with more powerful
3.3. Augmented reality for telemanipulated medical artificial intelligence algorithms and computation hardware.
robot However, the main obstacle to surgical robot automation
In the past 3 years, COVID-19 has posed unprecedented is the interaction between surgeons and robots. Another
challenges to both, patients who have underlying diseases challenge lies in in vivo microrobots, which have shown
and surgeons with long-distance or safety isolation potential for target drug and cell delivery, bacteria killing,
concerns. As a result of the considerable advancements in vascular cleanup, and other therapeutic applications. The
both, AI-ART and hardware computing power, medical 6-DoF motion control and navigation of the microrobots
applications integrated with augmented reality are growing in vivo are pushed forward by the rotating magnetic field
exponentially . In cardiovascular surgery, complicated technique. The challenges are listed below.
[57]
anatomical structures make the surgery more challenging. i. Vision segmentation accuracy and robustness: Is the
For instance, in obstructive hypertrophic cardiomyopathy, medical robot’s vision algorithm capable of precisely
the surgeon needs to open a small slot without damaging and robustly segmenting dynamic tissue or organs
the upper cardiac aorta, which is only a few millimeters from complex backgrounds with the given required
away from it. Therefore, in cases such as this, the interactive metrics? This is crucial in automatic surgery and
augmented reality technique supported by robotic human-robot interaction.

Volume 1 Issue 2 (2022) 9 https://doi.org/10.36922/gtm.v1i2.176