AI-boosted biopsy scrutiny

For over 100 years, pathologists have peered through microscopes and called on their medical training to decipher the meaning of a displaced cell or a malformation of tissue structure. Now, artificial intelligence is revolutionizing that practice. Pathologists at Stanford Medicine recently began collecting digital images of tissue samples, and they’ve designed an AI tool to sift through the mounds of information contained in each pixelated cell and determine its significance.

Stanford Medicine researchers expect the technology to quickly transform patient care.

“Computers can match patterns across millions of images in ways that would be difficult for humans to comprehend,” said Thomas Montine, MD, PhD, professor and chair of pathology. “Now, we can begin to unleash the power of artificial intelligence and machine learning techniques onto these data in ways we’ve only imagined.”

Tissue samples of tumors biopsied from patients can tell pathologists what type of cancer they’re dealing with, how aggressive it is and whether a treatment appears to be working. Many other diseases and disorders also reveal themselves under the pathologist’s trained eye, including infections, genetic disorders and brain diseases.

Stanford Medicine is one of only two academic medical centers in the country with a fully digitized pathology department. The scarcity of such departments reflects the steep price and broad culture change necessary to make this leap. Amassing enough data to train AI to recognize and categorize specific images is also difficult, particularly because many digital images are owned by private health care systems with little incentive to share.

“Pathology around the country still remains mostly manual,” Montine said. “A piece of tissue from a medically concerning mass or lesion comes into the pathology lab, where it is cut into thin sections and placed on glass slides and stained with chemical or biological material that will reveal structures and cells.

A pathologist looks through a microscope and captures that information in their mind and uses their medical training to interpret a biological meaning. Digitizing this process means that this image is instead captured by a computer and can be easily shared for discussion, teaching and learning.”

Although the images, which are stripped of any information that could be used to identify an individual patient, are captured digitally and categorized by the AI software, the pathologist remains firmly embedded in the process.

“We wanted to build an artificial intelligence tool with a human in the loop,” Montine said. “So the computer can screen the images, but the final judgment and diagnosis is left to the pathologist.”

Training AI to recognize patterns and images is a bit like teaching toddlers their colors by scattering a pile of colored Lego on a kitchen table and pointing out the yellow bricks in various shapes and sizes. After enough repetition — this is a yellow brick, this is a yellow brick, this is a yellow brick — children can pluck out the color themselves.

Another round of teaching can lead them to sort large yellow bricks from small yellow bricks, and so on. With pathology images, the process is similar, but far faster and more iterative — homing in not on large yellow Lego bricks, but on breast tissue with hints of ductal carcinoma in situ, for example, or lymphoma cells sneakily hobnobbing with healthy white blood cells.

“This tool has the exciting capacity to extend human capability,” said James Zou, PhD, assistant professor of biomedical data science, who partnered with Montine to create the software. “We envision having AI and humans work together as a team to save time and improve patient outcomes. Currently, we rely on clinicians evaluating images by themselves. But there can be tens of thousands of individual cells in one image, and some diseases cause only very subtle changes.”

Alone, images are just images. For a computer to learn from them it’s necessary to have expert text that describes the image (“yellow brick,” “yellow plate,” “yellow hinge,” for example, to continue with the Lego example).

Montine and Zou have launched an effort called Open Pathology to scour the internet for expertly annotated pathology images they can use to further train their tool. They’ve gathered several million so far. They envision a day when the data they’ve collected, which include both image and text, are linked to patient information and outcome from electronic medical records, or overlayed with genetic, molecular and spatial data in ways that allow AI to discern ever more complex patterns and associations beyond what even the most dedicated pathologist could perceive.

“This tool will help us improve care for our patients while doing our jobs faster, cheaper and more efficiently,” Montine said. “The sky is the limit.”