Structured AI beats humans in disaster decision-making by 39%
Researchers have created a new AI framework that achieved 60.94% greater stability in decision accuracy compared to judgment-only AI systems, and outperformed human operators by 38.93%. The codebase is openly available and evaluated on CrisisMMD, xBD, and RescueNet datasets, with a public web app used to gather human responses.
Combined Joint Task Force 50 (CJTF-50) search, rescue and recovery elements conduct search operations of areas damaged by wildfires in Lahaina, Maui, on August 15, 2023. Credit: Hawaii National Guard
- With artificial intelligence (AI) being applied to bring autonomy to decision-making in safety-critical domains such as the ones typified in the aerospace and emergency-response services, there has been a call to address the ethical implications of structuring those decisions, so they remain reliable and justifiable when human lives are at stake.
- This paper contributes to addressing the challenge of decision-making by proposing a structured decision-making framework as a foundational step towards responsible AI.
- The proposed structured decision-making framework is implemented in autonomous decision-making, specifically within disaster management.
A structured artificial intelligence (AI) framework designed to improve decision-making in disaster management has been introduced in a study published in Scientific Reports.
Developed by researchers at Cranfield University, UK, in collaboration with Golden Gate University, USA, and the University of Oxford, the system demonstrated more consistent and accurate decisions than human operators and conventional AI.
The framework organizes disaster management into structured “Scenarios,” each containing five decision “Levels.” At every Level, specialized AI models known as Enabler agents process information from victims, volunteers, satellites, and unmanned aerial vehicles (UAVs).
These agents generate structured judgment inputs to assist the Decision Maker, which can be either a human operator or a reinforcement learning (RL) algorithm.
For disaster-phase activities, such as search, rescue, and humanitarian response, the framework used the CrisisMMD dataset, comprising 16 058 tweets paired with 18 082 images from seven disasters in 2017.
Post-disaster decisions, such as damage assessment and recovery planning, were supported by the xBD dataset of 22 068 pre- and post-event satellite image pairs and the RescueNet dataset of 4 494 UAV images from Hurricane Michael.
Evaluation results showed that the structured AI framework achieved 60.94% greater stability in consistent decision-making compared to judgment-only AI systems.
The reinforcement learning Decision Maker reached an 88% accuracy rate across scenarios, significantly outperforming human operators — victims, volunteers, and stakeholders — who achieved 61–66% accuracy on average. The system also showed a 38.93% higher accuracy than the human participants overall.
According to the researchers, this structured approach reduces uncertainty in safety-critical contexts, where unstructured human decision-making and conventional machine learning systems may lead to inconsistent or delayed responses.
By organizing decision flows into transparent and traceable processes, the framework provides a foundation for responsible AI applications in disaster management.
The full project, including datasets, methodology, and source code, is available through the open-access repository GitHub.
References:
1 Structured AI decision-making in disaster management – Dcruz, J.G., Zolotas, A., Greenwood, N.R. et al. – Science Reports 15, 32093 (2025) – September 1, 2025 – DOI: https://doi.org/10.1038/s41598-025-15317-w
I am an Assistant Editor and Severe Weather & Science Journalist at The Watchers, specializing in real-time severe weather coverage, geophysical event reporting, and research-driven scientific analysis. You can reach me at rishav(at)watchers(.)news.
Commenting rules and guidelines
We value the thoughts and opinions of our readers and welcome healthy discussions on our website. In order to maintain a respectful and positive community, we ask that all commenters follow these rules.