The CVIG department at the CCG/ZGDV Institute has been joining research in the health sector for several years, acquiring, in particular, the ability to process medical images by applying new computer vision tools and Deep Learning algorithms.

Image segmentation divides it into multiple regions that may or may not belong to the same class. Typically, the criteria for this separation include color, texture, shape or other more complex factors. In recent decades, there has been significant evolution in this area, mainly with the advent of tools based on artificial intelligence, such as Deep Neural Networks and Transformers. These technologies have replaced traditional segmentation methods based on form factor, Bounding Box orientation, and Hu Moments. Such progress is directly linked to the evolution of hardware, with the emergence of more powerful and affordable graphics cards.

Segmentation of medical images using Deep Learning algorithms—a sub-area of artificial intelligence based on artificial neural networks—has become essential for analysis and diagnosis. Models such as U-Net and its variants stand out for their precise segmentation capabilities.

In medicine, segmentation is used to identify organs or lesions in computed tomography (CAT), magnetic resonance or microscope images. Specialized pathologists traditionally carry out this highly complex process. Traditional methods face challenges such as the lack of professionals and the need for high precision in interpretation, often requiring the collaboration of more than one specialist.

• Use cases in medicine

Tumor Segmentation

 

Figure 1: Tumor segmentation with MedSAM (https://github.com/bowang-lab/MedSAM)

Accurate tumor segmentation is crucial for planning and monitoring cancer treatment. Deep Learning models, such as convolutional networks with attention mechanisms, increase accuracy when delineating tumor boundaries, even in heterogeneous tissues. In figure 1, a segmented tumor region can be seen.

 

Organ Segmentation

 

Figure 2: Organ segmentation using U-Net (https://blog.dsacademy.com.br/segmentacao-de-imagens-medicas-com-deep-learning/)

Organ segmentation is fundamental in clinical applications such as volumetric analysis and disease assessment. Models trained on large datasets show high efficiency. Figure 2 shows the segmentation of organs in detail.

 

Vascular Segmentation

 

Figure 3: Vascular Segmentation (https://www.sciencedirect.com/science/article/pii/S1746809424003318)

Blood vessel segmentation is vital for understanding vascular diseases. Specialized architectures like U-Net efficiently capture complex structures. Figure 3 provides examples.

 

Lung Segmentation

 

Figure 4: Segmentation of regions where lung cancer is present (https://www.sciencedirect.com/science/article/pii/S1877050924009852).

In the context of diseases such as lung cancer and emphysema, advances in U-Nets have made it possible to locate nodules and determine the severity of lung conditions with precision, as shown in figure 4.

 

Dermatological Segmentation

 

Figure 5: Dermatological lesion segmentation (https://www.sciencedirect.com/science/article/pii/S0010482522010290).

In dermatology, Deep Learning has improved the segmentation of skin images, enabling more effective diagnoses. Deep networks like ResNets allow for high accuracy even with limited data. Figure 5 illustrates a segmented lesion.

• Challenges and perspectives

Despite advances, segmentation faces challenges such as noise, inadequate contrast, and the need for large datasets annotated by experts. Furthermore, ensuring the protection of sensitive data is crucial. To move forward, it is essential to invest in energy-efficient, interpretable, and generalizable models and promote interdisciplinary collaborations. Human resource development also plays a crucial role in overcoming these barriers.

By: André Silva
CVIG Department - Researcher in computer vision, image processing and artificial intelligence.