AI panel outperforms generalists in complex medical simulations

A significant tension has emerged between the growing demand for medical expertise and the limited supply of human specialists, a gap that Microsoft believes artificial intelligence can now begin to bridge. The company’s latest diagnostic system, known as MAI-DxO, has demonstrated the ability to outperform generalist physicians in handling complex medical cases, raising questions about the future role of human oversight in clinical decision-making. This development marks a pivotal moment in the race between major technology firms to establish AI as a reliable partner in healthcare delivery.

According to a study published in the journal Science on Thursday, the AI system correctly diagnosed difficult cases in a simulation environment 83 percent of the time. In comparison, primary care doctors and general internists working individually with reference materials were accurate only 36 percent of the time. The study utilized complex cases derived from the New England Journal of Medicine. It is important to note that such cases are typically managed by specialists or physician teams in real-world settings, who were not included in this specific comparison group.

The system operates by coordinating several AI agents that function similarly to a panel of physicians. These agents collaborate by questioning the patient, ordering necessary tests, and debating potential diagnoses. The research indicates that much of the performance improvement stems from the use of more powerful underlying AI models, with Microsoft’s orchestration layer adding additional gains on top of the base model capabilities. Microsoft tested the system using models from several major companies, including OpenAI, Google, Anthropic, Meta, and DeepSeek, finding that the best results were achieved using OpenAI’s GPT-5 and GPT-5.5 models.

Cost efficiency was also a key metric in the evaluation. When using GPT-5 with a basic prompt, the system scored 78.9 percent on complex cases at an estimated cost of $8,400 per case. The MAI-DxO orchestration reduced this cost by approximately $3,000 while increasing accuracy by about four points. However, on emergency department cases, there was little difference in either accuracy or cost between the standalone model and the orchestrated system. With the newer GPT-5.5 model, released in April 2026, the results were more pronounced for complex cases, where accuracy rose from 72 percent to nearly 87 percent at a slightly lower cost, though emergency department results remained largely unchanged.

Mustafa Suleyman, CEO of Microsoft AI, described the results as a step toward achieving medical superintelligence. He noted that the company is currently collaborating with the Mayo Clinic to train a foundation model specifically for healthcare, with a release planned for next year. Suleyman expressed confidence that this model will enable substantial leaps in medical AI, stating that by 2027, the company expects to be on the verge of a breakthrough in processing medical records. He envisioned a future where highly accurate predictions about patient outcomes can be streamed to clinicians, providing valuable foresight.

Dominic King, vice president of Health at Microsoft AI, emphasized the substantial nature of the findings. He highlighted that the current challenge in healthcare is the mismatch between demand and supply of expertise. King stated that the technology is now ready for real-world evaluations, noting the complexity of cases seen in major urban emergency departments. Through a new partnership with Beth Israel Deaconess Medical Center in Boston, Microsoft will evaluate the clinical reasoning capabilities of the system across different hospital settings. Patients will receive standard care, while those who consent will also engage with the AI, allowing researchers to compare its recommendations against those of human doctors.

Suleyman, who previously co-founded DeepMind and joined Microsoft in 2024 to lead its AI unit, has long focused on applying AI to medical issues. He believes that healthcare models will become an order of magnitude more capable within the next 24 to 36 months. He compared this trajectory to the rapid progress seen in coding and chat interfaces, suggesting that a similar flywheel effect will occur in medical records and clinical decision support. This perspective positions Microsoft’s efforts as part of a broader infrastructure buildout aimed at realizing breakthroughs in healthcare.

The publication of these results places Microsoft alongside Google and OpenAI in claiming that its AI can match or outperform doctors. The company also released a manifesto defining medical superintelligence as the capacity to surpass contemporary human-led health systems in medical reasoning and coordination across the full arc of care. Microsoft aims to move beyond benchmarking individual tasks, such as answering exam questions, toward integrating with clinicians and healthcare systems to improve overall outcomes. The company already sells dictation and radiology tools to clinicians and views this new diagnostic system as an expansion of those existing efforts.

As the industry moves forward, the focus will shift from theoretical accuracy to practical deployment. Microsoft asserts that its commitment to accelerating the introduction of this technology could serve as a differentiator compared to other frontier labs. The goal is to establish state-of-the-art medical clinical decision support products in the coming years. This initiative reflects a broader industry shift toward leveraging AI not just as a tool for information retrieval, but as a collaborative partner in complex medical reasoning.

The stakes are high for both patients and providers, as the integration of AI into clinical workflows promises to alleviate some of the pressures on the healthcare system. However, the reliance on simulations and specific case types means that real-world validation remains critical. As Microsoft and its partners proceed with evaluations in Boston, the medical community will be watching closely to see if the simulated advantages translate into tangible benefits in diverse and high-stress clinical environments.