Why is a biopsy slide so hard for an AI to read?
When a piece of a tumor is removed, a specialist doctor called a pathologist examines it under a microscope on a thin, stained glass slide. This tells the care team the exact type of cancer, which guides treatment. More and more labs now scan these slides. Each digital image is huge: billions of pixels, or tens of thousands of small image tiles.
Earlier AI models often looked at only a sample of these tiles, chosen at random, and lost the big picture. Many had also learned from a public database, TCGA, which is small compared with what hospitals see every day.
What the researchers built
The team gathered 171,189 slides from 28 cancer centers in Providence, a large US health network. They came from biopsies and surgeries in more than 30,000 patients, across 31 major tissue types. In total, that is more than 1.3 billion small image tiles.
With this data, they built a foundation model: a general AI that first teaches itself to describe what it sees, without labels, and can then be adapted to many specific tasks with few examples. Its key feature is that, thanks to a method borrowed from language models, it can take every tile of a slide into account at once. They tested it on 26 tasks using data from Providence and TCGA.
The results: best on 25 of 26 tasks
Prov-GigaPath scored best on 25 of the 26 tasks, with a clear lead over the second-best model in 18 of them. For identifying cancer subtypes, such as telling apart the different forms of lung or kidney cancer, it beat every other model on all nine cancer types studied.
For tumor gene mutations, which can point to a targeted treatment, it also did better than competing models, but the scores remain modest. For five lung cancer genes, including EGFR and KRAS, its average area under the curve was 0.626, on a scale from 0.5 (chance) to 1 (perfect). When tested on 403 new colorectal cancer patients, collected after training had ended, it kept its lead.
Key figures
- 1.3 billionsmall image tiles from 171,189 slides used for training
- 30,000patients and more, across 31 major tissue types
- 25 of 26tasks where Prov-GigaPath scored best
- 0.626average area under the curve for five lung cancer mutations: still a modest score
Making sense of your pathology report with Mima
Prov-GigaPath is still a research tool: your biopsy is read by a pathologist, and any mutation is confirmed with a genetic test. What the study highlights is how much the exact cancer type and its mutations can shape treatment.
Mima explains the words in your pathology report in plain language and keeps track of your results for your care team. You can prepare a question such as: "Has my tumor been tested for mutations that could change my treatment?"
What this could mean for patients
Tools like this could one day help pathologists classify tumors faster, or flag slides where a genetic test would be useful. The authors note that tumor sequencing is still hard to access in many parts of the world.
The model is open: other teams can download and test it. But it is not a tool used in hospitals today. A pathologist still makes the diagnosis, and a mutation is still confirmed with a genetic test.
The limits of this study
The authors themselves note that performance varies a lot from one task to another: good for cancer type, much weaker for mutations. In their view, the image alone may not hold enough information for some mutations.
These are tests on existing data, mostly from a single health network. No trial has yet shown whether this tool improves diagnosis or patients' health in real life.
Frequently asked questions
What is Prov-GigaPath?
Prov-GigaPath is a research AI built by Providence and Microsoft and trained on more than a billion image tiles from pathology slides. It serves as a base for tasks such as classifying cancer.
How is AI used in pathology?
AI can analyze scanned biopsy slides to help sort tumor types or flag features worth a closer look. In this study, it was tested on existing slides, not used for patient care.
Can AI detect gene mutations from a biopsy slide?
It can pick up clues, but in this study the scores were modest. A mutation is still confirmed with a genetic test of the tumor.
Is Prov-GigaPath open source?
Yes. The authors released the model weights and code so other researchers can use and test it.
The original study
Xu H, Usuyama N, Bagga J, et al. “A whole-slide foundation model for digital pathology from real-world data” Nature, May 22, 2024. DOI : 10.1038/s41586-024-07441-w
Read the full study in NatureThis article summarizes a scientific study and is not medical advice.