AI: Helpers Reducing Doctors' Workload, or New Healthcare Professionals?
In this article
Only 27% of healthcare workers' time goes to direct patient care; the rest goes to documentation and administration. AI agents promise to change that balance. Where do we stand?
🌟 Why Did We Write This?
Healthcare workloads grow annually, particularly because of electronic records and administrative processes , fueling physician burnout. Is there a trustworthy solution that could change this? 🤔
A Leading Candidate: AI agents. Beyond ordinary chatbots, they can handle record management, laboratory interpretation and workflow integration . Will these systems simply lighten workloads or create a new healthcare team role? 👩⚕️🤖
To explore this, we examine Stanford's MedAgentBench findings. NEJM AIpublished the study on 14 August 2025(DOI: 10.1056/AIdbp2500144), the first large-scale evaluationcomparing agents in realistic EHR scenarios. Below, we summarize the platform and model findings. 🚀
Why Does It Matter?
Studies show healthcare workers can devote only 27% of their time to direct patient care, with much spent on paperwork, EHR entry and administration. While this fuels burnout, AI agents promise to free clinical time by taking over routine tasks.
From AI Chatbots to Agents
- Interpret complex instructions and plan,
- Combine multiple data sources,
- Interact with electronic health records (EHRs) through standard APIs,
- Execute tasks step by step and summarize for physicians.
Example: Beyond answering questions, an agent assessing a patient with pneumonia could consider allergies, local antibiograms, interactions and risk scores, then enter a personalized treatment plan as draft orders in the system.
📊 MedAgentBench: The First Medical Platform Testing AI Agents
Developed by Stanford, MedAgentBenchis a broad benchmark evaluating agent capabilities in realistic EHR scenarios. FHIR-compatible simulation includes 300 tasks and 100 patient profiles.
📈 Model Success Rates — Interactive
Switch the view below: Overall SR, Query SR, Action SR.
Tip: Hover over bars to see rates. Click to lock a model name.
| Model | Overall SR (%) | Query SR (%) | Action SR (%) |
|---|---|---|---|
| Claude 3.5 Sonnet v2 | 69.67 | 85.33 | 54.00 |
| GPT-4o | 64.00 | 72.00 | 56.00 |
| DeepSeek-V3 | 62.67 | 70.67 | 54.67 |
| Gemini 1.5 Pro | 62.00 | 52.67 | 71.33 |
| GPT-4o mini | 56.33 | 59.33 | 53.33 |
| Qwen2.5 (72B) | 51.33 | 38.67 | 64.00 |
⚠️ Errors Encountered
- Failure to follow format instructions, such as text instead of a number.
- Invalid/incorrect API calls, including payload/parameter errors.
- Incomplete interpretation of clinical context.
👩⚕️ Could They Replace Physicians?
Current results indicate agents are more prepared to support physicians rather than replace them , taking over paperwork, orders and queries to free clinical time. Their current role is that of assistants . Over time, as reliability and standards improve, they may evolve into a new category of healthcare professional .
🚀 Looking Ahead
- Improved reproducibility and reliability,
- Richer data incorporating clinical notes and teamwork,
- Clear ethical, safety and regulatory frameworks.
🔗 Source and Citation
MedAgentBench: A Virtual EHR Environment to Benchmark Medical LLM Agents — NEJM AI (Published: 14 August 2025). DOI: 10.1056/AIdbp2500144. GitHub: stanfordmlgroup/MedAgentBench
Was this article helpful?
Your feedback helps us improve our content.
Reader comments
Share your comment
Your name and comment are published only after administrator approval. Your email address is never shown to visitors.
Be the first to comment.