Will Shapiro, vice president of data science at Flatiron Health, discusses how the practice is using large language models (LLMs) to increase efficiency, while ensuring there are frameworks set for safety and quality.
Machine learning (ML) is a very powerful artificial intelligence (AI) tool, and there needs to be guidelines to ensure patient safety and quality of care, says Will Shapiro, vice president of data science at Flatiron Health.
Transcript
How is Flatiron Health using AI in oncology care?
We're very focused on what the use case is. One of the things that that my team does a lot of is use and build machine learning algorithms to read through unstructured documents, because there's an enormous amount of really valuable information that isn't captured in a structured way routinely. Things like a patient's stage, biomarker status, even diagnosis date; these are really critical variables, especially for researchers. That traditionally has always had to be manually extracted.
Our mission as a company, part of it is to learn from the experience of every person with cancer. And that's something that really motivates me every day. Machine learning is a very powerful tool in that context.
For the task of using ML to read patient charts, we have a pretty lengthy evaluation framework that we've developed that looks at things like bias across many different strata. So, age, race, gender, ethnicity, location. But then, also says, “Okay, if we've generated something with ML, and we've generated it using humans, do they replicate in the same way when you're asking an analytic question or a research question?”
On the other hand, at the at the point of care, we've started to use machine learning more recently. So, one of the things I'm really excited about is we have a new feature that uses LLMs [large language models] to map regimens, authored by practices to NCCN [National Comprehensive Cancer Network] guidelines. And that has a totally different framework for thinking about evaluation.
One of the things this gets at is that with this really powerful new generation of LLMs and gen-AI [generative AI] tools that can do so many different things, you really need to think about what the use case is, and then think about what the appropriate evaluation framework for safety and quality within that specific context is, because it might be very different. It's very different to suggest a list of options than to say, “Here's what you should do.” Those have radically different implications. And the way that the output of an algorithm is contextualized within someone's workflow is a big part of this. And being told to do something is different than being presented with an array of options and the evidence that supported each of them.
IgE Mediation in Pediatric Atopic Dermatitis, Concurrent Immune Disorders: Amy Paller, MD
August 4th 2025Amy Paller, MD, pediatric dermatologist and clinical researcher at Northwestern Medicine's Feinberg School of Medicine, discussed the potential impact of reducing immunoglobulin E (IgE) levels in pediatric patients with atopic dermatitis.
Read More
LLMs Show Promise, But Challenges Remain in Improving Inefficient Clinical Trial Screening
July 31st 2025Large language models (LLMs) such as GPT-3.5 and GPT-4 may offer a solution to the costly and inefficient process of manual clinical trial screening, which is often hindered by the inability of structured electronic health record data to capture all necessary criteria.
Read More
Legal Navigation Services Key to Addressing Disparities in Cancer Care
July 31st 2025A significant number of patients with cancer, caregivers, and health care professionals face significant disparities in legal issues, with the most common needs revolving around health insurance, finances, employment, and disability insurance.
Read More
IgE Mediation in Pediatric Atopic Dermatitis, Concurrent Immune Disorders: Amy Paller, MD
August 4th 2025Amy Paller, MD, pediatric dermatologist and clinical researcher at Northwestern Medicine's Feinberg School of Medicine, discussed the potential impact of reducing immunoglobulin E (IgE) levels in pediatric patients with atopic dermatitis.
Read More
LLMs Show Promise, But Challenges Remain in Improving Inefficient Clinical Trial Screening
July 31st 2025Large language models (LLMs) such as GPT-3.5 and GPT-4 may offer a solution to the costly and inefficient process of manual clinical trial screening, which is often hindered by the inability of structured electronic health record data to capture all necessary criteria.
Read More
Legal Navigation Services Key to Addressing Disparities in Cancer Care
July 31st 2025A significant number of patients with cancer, caregivers, and health care professionals face significant disparities in legal issues, with the most common needs revolving around health insurance, finances, employment, and disability insurance.
Read More
2 Commerce Drive
Cranbury, NJ 08512