Deep learning in digital pathology: what, why, and how?

Deep learning (DL) is a subset of machine learning (ML) that falls under the category of artificial intelligence (AI). DL is based on artificial neural networks (ANNs), a layered and connected system of algorithms that receive and process information.

Traditional machine learning can only be taught information extracted and coded by humans. Deep learning, on the other hand, is highly flexible and scalable. It can learn features by itself, making it an extremely powerful tool for image analysis. What brings the deep to deep learning is the addition of more neural network layers, from tens to hundreds. Deep learning models with multiple hidden layers can achieve impressive performance on various tasks, from image recognition to natural language processing.

Artificial intelligence (AI)
Techniques that enable computers to mimic human intelligence

Machine learning (ML)
Use of data and algorithms to imitate the way that humans learn 

Deep learning (DL)
A subset of ML, often referred to as the next generation of machine learning, as DL learns from data without external feature extraction and thus does not have the bias and limitation of feature extraction

How do neural networks work? 

Deep learning is based on convolutional neural networks (CNN), which are computing systems designed to find patterns that are too complex to be manually taught to machines to recognize. CNNs are named after convolutions, a type of mathematical operation that they use to assess input data to extract information.

Neural networks see images as a grid of numbers represented by the pixels in an image. To use DL for image recognition, the networks must first be trained with, for example, images of a specific type of tumor. They then develop an idea of what an image of that tumor contains and learn to recognize it.  

CNNs can use different learning methods. Some of the more common ones are supervised, unsupervised, semi-supervised, and reinforcement learning. Aiforia’s AI, and thereby its convolutional neural networks, are, in most cases, trained with the help of supervised learning, where the neural networks learn on a labeled dataset. However, they can also be used in semi-supervised approaches, where the neural networks are used to find image patterns that may explain certain outcomes. For example, a research team from the University of Helsinki developed a deep learning AI model to predict patient outcomes in a complex ovarian cancer. Read the full case study here →

The training process

The first step in training is to select the input data. In the case of image analysis, this is the training set of images labeled for certain parameters. The training set should represent the possible variations in the whole material used in the actual image analysis workflow.

With Aiforia® Create, Aiforia’s AI development tool, this training process is easy for the end user, as no data science or software programming expertise is needed. Thanks to its intuitive interface, the parameters are taught to the neural networks simply by annotating and categorizing certain features in the image, for example, liver tissue versus background, such as in this image:

Aiforia® Create allows users to train AI models to learn many different image features. During AI training, the artificial neural networks can be taught to recognize different tissue patterns, such as liver tissue, parenchyma, and portal areas, by drawing small example areas of each category. The possibilities are limitless, as the neural networks can learn whatever tissue pattern users are able to identify and teach them.

Why deep learning is essential in digital pathology?

Deep learning sets new records and high standards in fields such as image recognition, a significant technological advancement in healthcare. The recognition and discovery of patterns lie at the heart of scientific progress, and deep learning AI is by far the best tool for this. Let’s review three clear benefits of deep learning in pathology.

1. Improving accuracy and saving time

For the first time, deep learning AI models can mimic humans in learning to recognize complex visual features in image data. However, DL is faster and often more accurate and, thus, can surpass human capability. It can be deployed in various applications, from object quantification to tissue classification based on morphology, and gives accurate, quantitative information from biological samples. 

Therefore, pathologists can automate manual and time-consuming image analysis work and focus on more critical tasks like decision-making and collaborating with peers. With the convenience of a remote connection, the pathologist can automate analysis, review the data, and examine the specimen anywhere, at any time.

2. Increasing efficiency in research and discovery

Deep learning AI can find what the human eye sometimes cannot see. Features that are either too small, heterogeneous in their expression, or found in a vast quantity spread over a large area are easily recognizable by AI. The discovery process can be enhanced, especially when considering the development of novel drug molecules, as deep learning excels at detecting subtle differences between study groups.

Even the smallest of changes, which cannot be manually visualized, can be detected by the AI models. Furthermore, you can train them with external ground truths not readily identifiable from the images provided, such as response to treatment. Predictive models can be trained with outcome data. These then learn to extract and visualize the features associated with the outcome. 

3. Increasing consistency

Inter- and intra-observer subjectivities are critical issues in image analysis across diagnostic settings, research fields, and sample types. Whether a specific feature is scored can vary from one person to another and even within the same pathologist from one day to the next. 

Deep learning AI, however, doesn’t get tired, and its performance stays consistent. It does not stray away from what it has been taught to do or find. The algorithms classify results and solve problems with concordance, always according to the ground truth they were given.

"If a person started counting these cells one by one, they would lose their health and age. And what’s more, the results are completely consistent. AI doesn’t have bad days like humans do. Subjectivity is wiped away, while my work is significantly sped up. This is hugely important in pathology but also in research in general." – Nelli Sjöblom, MD specializing as a pathologist at the Helsinki University Hospital

Explore case studies where deep learning has been used in digital pathology →