From data to diagnosis

Google’s AMIE and MedGemma redefine precision in medical AI

Arjuna Sathiaseelan & Shruti Singh
May 26, 2025

In partnership with

Transform your hiring with Flipped.ai – the hiring Co-Pilot that's 100X faster. Automate hiring, from job posts to candidate matches, using our Generative AI platform. Get your free Hiring Co-Pilot.

Dear Reader,

Flipped.ai’s weekly newsletter is read by more than 75,000 professionals, entrepreneurs, decision-makers, and investors around the world.

This week, we spotlight Google’s bold push into AI-powered healthcare with two groundbreaking models: AMIE and MedGemma. From diagnosing skin conditions with photos to supporting chronic care with structured treatment plans, these tools bring clinical intelligence to the forefront of digital medicine. AMIE leverages multimodal reasoning and conversational AI to match and sometimes surpass doctors in diagnostic accuracy. MedGemma, built on Google’s lightweight Gemma architecture, offers open-source flexibility for developers and clinicians alike.

Dive in as we break down the research, results, and real-world potential of these models and see how they’re reshaping the future of patient care.

Before, we dive into our newsletter, checkout our sponsor for this newsletter.

Learn AI in 5 minutes a day

This is the easiest way for a busy person wanting to learn AI in as little time as possible:

Sign up for The Rundown AI newsletter
They send you 5-minute email updates on the latest AI news and how to use it
You learn how to become 2x more productive by leveraging AI

AI meets medicine: Google’s AMIE and MedGemma streamline diagnostics and chronic care

Google

Welcome to this week’s deep dive into AI-driven healthcare innovation! Google’s Articulate Medical Intelligence Explorer (AMIE) and MedGemma models are transforming medical diagnostics and disease management, offering tools that enhance clinical accuracy and streamline care delivery.

For physicians, these systems promise faster, more precise workflows; for patients, they herald earlier and more reliable interventions. Together, AMIE’s conversational diagnostic prowess and MedGemma’s open-source versatility form a powerful duo for the future of medicine. Let’s explore how these advancements are reshaping healthcare.

Introducing AMIE: A multimodal diagnostic powerhouse

As detailed in a recent Nature paper, AMIE, developed with Google DeepMind, has evolved from a text-only system into a powerful multimodal tool capable of interpreting medical images, lab results, and ECGs. This conversational AI agent emulates the structured reasoning of experienced clinicians, offering a transformative approach to diagnostics that could enhance clinical practice and patient outcomes.

Multimodal reasoning: Seeing beyond words

AMIE’s ability to process visual and documentary medical data during clinical conversations marks a significant leap forward. Built on Gemini 2.0 Flash, AMIE uses a state-aware reasoning framework to adapt its responses based on the patient’s evolving medical profile. This enables AMIE to:

Request relevant data: AMIE intelligently prompts for inputs like skin photos or test results to address diagnostic gaps.
Interpret findings: It accurately analyzes medical imagery and integrates findings into the diagnostic process.
Refine diagnoses: AMIE continuously updates its hypotheses, mirroring clinical decision-making.

Multimodal inputs—like images and test results—are vital to accurate diagnosis, making this capability a major step forward. For clinicians, this streamlines complex tasks; for patients, it offers faster, more reliable care.

AMIE’s chat-based interface, akin to multimedia messaging platforms, makes it a seamless fit for modern healthcare settings.

AMIE employs a state-aware dialogue framework that progresses through three distinct phases, each with a clear objective: History Taking, Diagnosis & Management, and Follow-up. AMIE's dynamic internal state — reflecting its evolving understanding of the patient, diagnoses, and knowledge gaps — drives its actions within each phase (e.g., information gathering and providing explanations to the patient). Transitions between phases are triggered when the system assesses that the objectives of the current phase have been met. Credit: Google

An example of state-aware reasoning in practice during a simulated consultation with a patient-actor. At the start of this part of the interaction, AMIE is aware of the gaps in its knowledge about the case: the lack of images. AMIE requests these images, and once provided, updates its knowledge as well as its differential diagnosis. Credit: Google

Emulating clinical expertise

AMIE’s state-aware phase transition framework orchestrates diagnostic conversations with flexibility and precision. By leveraging Gemini 2.0 Flash outputs, AMIE tracks patient data, refines diagnostic possibilities, and asks targeted questions. For instance, when diagnosing a rash, AMIE might request a photo, analyze it for dermatological patterns, and combine this with symptom reports to narrow down diagnoses. This methodical approach enhances diagnostic accuracy and efficiency, saving time in busy clinical environments.

Key feature: Adaptive reasoning
AMIE’s ability to dynamically request and interpret multimodal data mirrors the history-taking process of seasoned clinicians, making it a valuable tool for complex cases.

MedGemma: Accelerating healthcare innovation

Complementing AMIE, Google’s MedGemma models—built on the open-source, lightweight Gemma 3 architecture—enable rapid development of healthcare applications. These freely accessible models empower developers worldwide to create tailored solutions. Available as MedGemma 4B (multimodal) and MedGemma 27B (text-only), they excel in analyzing medical texts and images, offering accessible solutions for diverse medical tasks.

Compact, Customizable, and Open-Source

MedGemma’s compact design allows efficient fine-tuning for specific use cases, such as analyzing chest X-rays or summarizing clinical reports. Despite its smaller size, MedGemma 27B rivals larger models like GPT-4o on the MedQA benchmark, demonstrating robust clinical knowledge and reasoning. The multimodal MedGemma 4B processes images, making it ideal for tasks like detecting anomalies in radiological scans.

Freely available on Hugging Face and Google’s Model Garden, MedGemma supports local hosting and Google Cloud integration. Developers can pair it with tools like web search for real-time medical updates or FHIR interpreters for standardized health data, empowering smaller healthcare organizations to leverage advanced AI without extensive infrastructure.

Real-world impact: Supporting public health
MedGemma could assist public health efforts, such as monitoring emerging diseases by analyzing clinical data and imaging, enabling rapid response in resource-limited settings.

AMIE in practice: A comparative clinical evaluation

Google evaluated AMIE’s multimodal capabilities through a virtual Objective Structured Clinical Examination (OSCE) study, a global standard in medical education. AMIE was compared to primary care physicians (PCPs) across 105 scenarios, with patient actors uploading multimodal data like skin photos via a chat interface.

Key findings from the OSCE study

AMIE matched or surpassed PCP benchmarks in several areas:

Diagnostic accuracy: AMIE achieved a Top-3 Diagnosis Accuracy of 0.65 versus PCPs’ 0.59.
Management reasoning: AMIE scored 0.86 in Management Plan Appropriateness, outperforming PCPs’ 0.77.
Empathy and trust: Patient actors rated AMIE as more empathetic and trustworthy.
Image interpretation: Specialists gave AMIE higher scores for analyzing multimodal data.

Overview of our simulation environment for multimodal dialogue evaluation. Credit: Google

These results highlight AMIE’s potential to augment clinical decision-making, offering precise diagnoses and effective management plans while maintaining patient trust. For clinicians, this means more confident decision-making; for patients, it translates to higher-quality care, even in resource-constrained settings.

AMIE’s hallucination rate was comparable to PCPs, ensuring reliability. Preliminary tests with Gemini 2.5 Flash suggest further improvements, enhancing clinician confidence and patient access to quality care.

Simulation environment for robust testing

Google developed a simulation framework using datasets like SCIN (dermatology images) and PTB-XL (ECG data) to generate realistic patient scenarios and evaluate AMIE’s performance. This ensures AMIE is rigorously tested for diagnostic accuracy and safety, preparing it for real-world clinical integration.

Overview of our simulation environment for multimodal dialogue evaluation. Credit: Google

AMIE for longitudinal disease management

AMIE now supports longitudinal disease management, assisting clinicians in monitoring disease progression and adjusting treatments across multiple visits. Its two-agent model includes:

Dialogue agent: Handles patient interactions and collects clinical data consistently.
Management reasoning agent: Generates structured treatment plans based on clinical guidelines.

Using Gemini’s long-context processing, AMIE aligns recommendations with standards like NICE guidelines. In a blinded OSCE study with 100 multi-visit scenarios, AMIE’s plans were non-inferior to PCPs, with improved treatment precision, enhancing efficiency in chronic care.

Top-k accuracy of differential diagnosis (DDx). AMIE and primary care physicians (PCPs) are compared across 105 scenarios with respect to the ground truth diagnosis. Upon conclusion of the consultation, both AMIE and PCPs submit a differential diagnosis list (at least 3, up to 10 plausible items, ordered by likelihood). Credit: Google

Relative performance of PCPs and AMIE on other key OSCE axes as assessed by specialist physicians and patient actors. The red segments represent the proportions of patient scenarios for which AMIE’s dialogues were rated more highly than the PCPs on the respective axes. The asterisks represent statistical significance (*: p<0.05, : p<0.01, *: p<0.01, n.s.: not significant) Credit: Google

MedGemma’s real-world potential

MedGemma’s versatility shines in scenarios like a rural clinic using MedGemma 4B to screen X-rays for tuberculosis, speeding up diagnosis in underserved areas. It could also support public health by analyzing data to track emerging diseases, such as monitoring flu outbreaks, enabling proactive interventions. Together, AMIE’s diagnostic depth and MedGemma’s accessible innovation create a robust ecosystem for advancing healthcare delivery.

How MedGemma supports resource-limited settings?
By offering lightweight, open-source AI, MedGemma enables clinics with limited resources to implement advanced diagnostics, such as tuberculosis screening, improving access to care.

Limitations and future directions

While AMIE and MedGemma excel in controlled settings, real-world challenges like integration with electronic health records, patient variability, and clinician decision fatigue—all crucial for safe deployment—require further study. Google’s ongoing research with Beth Israel Deaconess Medical Center will evaluate AMIE in clinical environments. Future work will explore real-time audio-video interactions to support telemedical assessments, capturing non-verbal cues critical for high-quality care.

Experts emphasize cautious deployment. Dan Zeltzer notes the need for transparency in model design, while Xueyan Mei sees large language models as the future of diagnostics, provided risks like biases or hallucinations are addressed.

(Image: Art Stock Creative/Shutterstock.com

Conclusion: Shaping the future of healthcare

Google’s AMIE and MedGemma represent a bold step toward AI-augmented healthcare, enhancing diagnostic accuracy, chronic disease management, and medical innovation. AMIE’s multimodal reasoning and empathetic interactions empower clinicians, while MedGemma’s open-source accessibility democratizes advanced tools for diverse settings. As these systems move from lab to clinic, they’re paving the way for AI to transform every step of the healthcare journey. Dive into these tools at x.ai/api or Hugging Face and join the healthcare AI revolution!

Supercharge your hiring with parsetalent for zoho recruit!

Ready to transform your recruitment process? Parsetalent for Zoho Recruit, a cutting-edge Chrome extension, brings AI-powered efficiency to your fingertips!

CV parser: Wave goodbye to manual resume entry—our smart tool extracts key candidate data (skills, experience, education) with precision.
JD parser: Turn chaotic job descriptions into clear insights, spotlighting essential skills and requirements for smarter role alignment.
Candidate matching: Unleash AI to match top talent instantly, delivering spot-on recommendations to fill roles faster.

Download the Zoho extension now and experience seamless, data-driven hiring!

Discover the full potential of Parsetalent! We offer four unique services designed to streamline your workflow and help you achieve your goals. Take a moment to explore each of them and see how they can make your experience even better!

Want to get your product in front of 75,000+ professionals, entrepreneurs decision makers and investors around the world ? 🚀

If you are interesting in sponsoring, contact us on [email protected].

Thank you for being part of our community, and we look forward to continuing this journey of growth and innovation together!

Best regards,

Flipped.ai Editorial Team