New AI model draws treasure maps to diagnose disease (2024)

Mark Anastasio (right) and Sourya Sengupta. Credit: Jenna Kurtzweil, Beckman Institute Communications Office.

Medical diagnostics expert, doctor’s assistant, and cartographer are all fair titles for anartificial intelligence model developed by researchers at the Beckman Institutefor Advanced Science and Technology.

Their new model accurately identifies tumors and diseases in medical images and is programmed to explaineach diagnosis with a visual map. The tool’s unique transparency allows doctorsto easily follow its line of reasoning, double-check for accuracy, and explainthe results to patients.

"The idea is to help catch cancer and disease in its earliest stages — like an X ona map — and understand how the decision was made. Our model will helpstreamline that process and make it easier on doctors and patients alike,” saidSourya Sengupta, the study’s lead author and a graduate research assistant at the Beckman Institute.

This research appeared in IEEE Transactions on Medical Imaging.

Cats and dogs and onions and ogres

First conceptualized in the 1950s, artificial intelligence — the concept thatcomputers can learn to adapt, analyze, and problem-solve like humans do — has reachedhousehold recognition, due in part to ChatGPT and its extended family ofeasy-to-use tools.

Machine learning, or ML, is one of many methods researchers use to create artificiallyintelligent systems. ML is to AI what driver’s education is to a 15-year-old: a controlled, supervised environment to practicedecision-making, calibrating to new environments, and rerouting after a mistakeor wrong turn.

Deep learning — machine learning’s wiser and worldlier relative — can digest larger quantities of information to makemore nuanced decisions. Deep learning models derive their decisive power from theclosest computer simulations we have to the human brain: deep neural networks.

These networks — just like humans, onions, and ogres — have layers, which makes themtricky to navigate. The more thickly layered, or nonlinear, a network’sintellectual thicket, the better it performs complex, human-like tasks.

Consider a neural network trained to differentiate between pictures of cats and picturesof dogs. The model learns by reviewing images in each category and filing awaytheir distinguishing features (like size, color, and anatomy) for futurereference. Eventually, the model learns to watch out for whiskers and cryDoberman at the first sign of a floppy tongue.

But deep neural networks are not infallible — much like overzealous toddlers, saidSengupta, who studies biomedical imaging in the University of IllinoisUrbana-Champaign Department of Electrical and Computer Engineering.

“They get it right sometimes, maybe even most of the time, but it might not always be for the right reasons,” he said. “I’m sure everyone knows a childwho saw a brown, four-legged dog once and then thought that every brown,four-legged animal was a dog.”

Sengupta’s gripe? If you ask a toddler how they decided, they will probably tell you.

“But you can’t ask a deep neural network how it arrived at an answer,” he said.

The black box problem

Sleek, skilled, and speedy as they may be, deep neural networks struggle to master theseminal skill drilled into high school calculus students: showing their work.This is referred to as the black box problem of artificial intelligence, and ithas baffled scientists for years.

On the surface, coaxing a confession from the reluctant network that mistook aPomeranian for a cat does not seem unbelievably crucial. But the gravity of theblack box sharpens as the images in question become more life-altering. Forexample: X-ray images from a mammogram that may indicate early signs of breastcancer.

The process of decoding medical images looks different in different regions of theworld.

“In many developing countries, there is a scarcity of doctors and a long line ofpatients. AI can be helpful in these scenarios,” Sengupta said.

When time and talents are in high demand, automated medical image screening can bedeployed as an assistive tool — in no way replacing the skill and expertise ofdoctors, Sengupta said. Instead, an AI model can pre-scan medical images andflag those containing something unusual — like a tumor or early sign of disease,called a biomarker — for a doctor’s review. This method saves time and can evenimprove the performance of the person tasked with reading the scan.

These models work well, but their bedside manner leaves much to be desired when, forexample, a patient asks why an AI system flagged an image as containing (or notcontaining) a tumor.

Historically, researchers have answered questions like this with a slew of tools designed todecipher the black box from the outside in. Unfortunately, the researchersusing them are often faced with a similar plight as the unfortunateeavesdropper, leaning against a locked door with an empty glass to their ear.

“It would be so much easier to simply open the door, walk inside the room, and listen to the conversation firsthand,” Sengupta said.

To further complicate the matter, many variations of these interpretation toolsexist. This means that any given black box may be interpreted in “plausible butdifferent” ways, Sengupta said.

“And now the question is: which interpretation do you believe?” he said. “There is a chance that your choicewill be influenced by your subjective bias, and therein lies the main problemwith traditional methods.”

Sengupta’s solution? An entirely new type of AI model that interprets itself every time —that explains each decision instead of blandly reporting the binary of “tumorversus non-tumor,” Sengupta said.

No water glass needed, in other words, because the door has disappeared.

Mapping the model

A yogi learning a new posture must practice it repeatedly. An AI model trained totell cats from dogs studying countless images of both quadrupeds.

An AI model functioning as doctor’s assistant is raised on a diet of thousands ofmedical images, some with abnormalities and some without. When faced with somethingnever-before-seen, it runs a quick analysis and spits out a number between 0and 1. If the number is less than .5, the image is not assumed to contain atumor; a numeral greater than .5 warrants a closer look.

Sengupta’s new AI model mimics this setup with a twist: the model produces a value plus a visual map explaining its decision.

The map— referred to by the researchers as an equivalency map, or E-map for short— is essentially a transformed version of the original X-ray, mammogram, orother medical image medium. Like a paint-by-numbers canvas, each region of the E-mapis assigned a number. The greater the value, the more medically interesting theregion is for predicting the presence of an anomaly. The model sums up thevalues to arrive at its final figure, which then informs the diagnosis.

“For example, if the total sum is 1, and you have three values represented on themap — .5, .3, and .2 — a doctor can see exactly which areas on the mapcontributed more to that conclusion and investigate those more fully,” Senguptasaid.

This way, doctors can double-check how well the deep neural network is working — likea teacher checking the work on a student’s math problem — and respond to patients’ questions about the process.

“The result is a more transparent, trustable system between doctor and patient,” Sengupta said.

X marks the spot

The researchers trained their model on three different disease diagnosis tasksincluding more than 20,000 total images.

First, the model reviewed simulated mammograms and learned to flag early signs oftumors. Second, it analyzed optical coherence tomography images of the retina, whereit practiced identifying a buildup called Drusen that may be an early sign ofmacular degeneration. Third, the model studied chest X-rays and learned todetect cardiomegaly, a heart enlargement condition that can lead to disease.

Once the mapmaking model had been trained, the researchers compared its performanceto existing black-box AI systems — the ones without a self-interpretationsetting. The new model performed comparably to its counterparts in all threecategories, with accuracy rates of 77.8% for mammograms, 99.1% for retinal OCTimages, and 83% for chest x-rays compared to the existing 77.8%, 99.1%, and83.33.%

These high accuracy rates are a product of the deep neural network, the non-linearlayers of which mimic the nuance of human neurons.

To create such a complicated system, the researchers peeled the proverbial onionand drew inspiration from linear neural networks, which are simpler and easierto interpret.

“The question was: How can we leverage the concepts behind linear models to makenon-linear deep neural networks also interpretable like this?” said principal investigator Mark Anastasio, a Beckman Institute researcher and the Donald Biggar Willet Professor and Head of the Illinois Department ofBioengineering. “This work is a classic example of how fundamental ideas can lead to some novel solutions for state-of-the-art AImodels.”

The researchers hope that future models will be able to detect and diagnoseanomalies all over the body and even differentiate between them.

“I am excited about our tool’s direct benefit to society, not only in terms of improving disease diagnoses, but also improving trust andtransparency between doctors and patients,” Anastasio said.

Editor's notes:

The paper associated with this work is titled “A Test Statistic Estimation-based Approach for Establishing Self-interpretableCNN-based Binary Classifiers” and can be accessed online here: https://doi.org/10.1109/TMI.2023.3348699

Mark Anastasio is also a professor in the Illinois Departments of Electrical and Computer Engineering, Computer Science, and Biomedical and TranslationalScience, and an affiliate of the Coordinated Science Lab. He can be reached at [email protected].

Media contact: Jenna Kurtzweil, [email protected].