Microsoft's new MAI Diagnostic Orchestrator, or MAI-DxO, correctly identified 83% of complex medical cases in a study published Oct. 22 in the journal Science. The system, which utilizes multiple AI agents that challenge one another's conclusions, significantly outperformed generalist physicians who achieved a 36% accuracy rate when permitted to consult reference materials. Without such resources, the physicians' accuracy fell to 22%.
The researchers evaluated the technology using 304 complex medical cases from The New England Journal of Medicine, which often involve unusual illnesses. In a separate test involving historical emergency department cases, the AI achieved approximately 80% accuracy, while the physician comparison group reached 47%. In a cost-conscious configuration, the AI achieved 81% accuracy on complex cases with an estimated diagnostic cost of $2,241 per case, compared to an average of $4,725 for the physicians.
Microsoft's orchestration system provided measurable improvements over standard large language models. When researchers used a basic prompt without the orchestration system, OpenAI's GPT-5 correctly solved approximately 79% of the complex cases. Using the newer GPT-5.5 model, the MAI-DxO system raised accuracy from approximately 72% to nearly 87%.
Researchers noted several limitations to the study's findings. The physicians used for comparison were generalists, such as primary care doctors and general internists, rather than teams of medical specialists who typically collaborate on difficult cases in a hospital setting. Additionally, the AI was evaluated across 304 cases, while the physicians worked through a smaller set of 56 cases held out from the system's development process.
