Chat/AI
By Clint Smith and Ryan Falkestein-Smith
Artificial intelligence (AI) offers a wealth of opportunities to improve every phase of fire service operations when used as a trusted tool. However, if AI is not developed, deployed or maintained appropriately, it has the potential to cause unintended consequences that differ significantly from its intended function. To adopt AI as a trusted tool in the fire service, consideration should be given to how best to manage any unintended consequences by applying risk management.
AI risk management is the process of assessing the likelihood of an event occurring alongside the magnitude of its consequences. Consequences can be positive, negative or, in some instances, both, such as a “near miss” incident that could improve organizational learning at the expense of individual trauma for those involved.
AI risks are contextual, meaning that awareness of the environment and how AI is used is essential to framing the benefits and risks of the system being constructed. Perspectives on an AI system’s risk may differ across its lifecycle, from design and development through validation, deployment and operational use. Additionally, the difference in perspective may stem from the various participating individuals within the AI ecosystem, which include:
There is no unified perspective on what constitutes AI system risk. Because there is no clear definition of AI system risk, it is challenging to establish a single AI risk management process or procedure that addresses all risks. This is especially true for AI use in the fire service, specifically the end-user who encounters unique scenarios daily for which the AI system was not trained. The fire service needs an AI risk management process that’s adaptable to the particular application or mission for which the AI platform is being used.
One pathway to managing risks of AI systems is to leverage the NIST AI Risk Management Framework (AI RMF), which was created at the direction of the National Artificial Intelligence Initiative Act of 2020. The Framework was designed to equip organizations and individuals with approaches that increase the trustworthiness of AI systems, and to help foster the responsible design, development, deployment and use of AI systems over time.
This general-purpose framework consists of four functions:
The core functions of the AI RMF can be adopted when using AI systems in the fire service, with these considerations in mind.
Mapping: In the fire service, defining the level of autonomy of an AI system provides insight into mapping the risk posed by that system. The level of scrutiny applied to a system’s application may be directly proportional to its autonomy.
Drawing from the Food and Drug Administration medical device classifications and the Nuclear Regulatory Commission’s assessment of autonomy levels, the fire service could categorize AI tools into distinct classes using the table below:
Measuring: In addition to the calculated metrics of an AI system’s accuracy, specificity and sensitivity, the consequences of failure and the various ways to assess them are an approach to measuring the risks of an AI system. An AI system can only work as well as it is designed and trained to work, suggesting that a poorly constructed model may fail. In the event of incorrect output or incorrect actions by the tool or by a user relying on an AI system, severe consequences may result. Based on the application’s context, the risk measurement process may vary. Risks associated with life-safety-critical tasks may warrant stricter evaluation metrics than those for low-priority functions. In either case, however, the consequences of the AI system’s failure may still apply.
Managing: Prioritizing and actively mitigating risks identified during the mapping and measurement functions creates a pathway for managing risk with AI systems. In the context of an AI system’s use in the fire service, the life safety classification may establish its risk assessment priority. A suggested ranking of the relative importance of each task or function could be categorized as follows:
High classification categories may require greater scrutiny to avoid severe consequences in the event of failure. The responsibility for managing risk isn’t necessarily on users. Still, it may require collaboration among technology developers, integrators and end-users to work as a unified ecosystem to identify strategies that actively mitigate risks at every priority level.
Governing: Governance of AI systems to manage their risks ensures that other listed core functions of the AI RMF are applied consistently throughout a system’s use. Establishing a proactive plan to rigorously test, understand and properly validate AI systems before deployment is critical to their adoption. Similar to the management of core functions, the success of ensuring proper procedures are in place when implementing AI systems may depend heavily on a unified ecosystem sustained by strong communication among all parties involved in the AI system’s life cycle.
The integration of AI into the fire service as a tool offers a transformative opportunity to enhance situational awareness and firefighter safety. The success of its adoption, however, is rooted in a system’s recognized value, which may depend on a proactive risk-management approach.
By leveraging the NIST AI Risk Management Framework’s core functions, the critical work of managing risks may be achieved by:
By committing to these core functions, the fire service has a pathway toward AI becoming a force-multiplier in a deep toolkit of resources.
ABOUT THE AUTHORS
Dr. Ryan Falkenstein-Smith is a mechanical engineer in the Firefighting Technology Group of the Fire Research Division (FRD) at the National Institute of Standards and Technology (NIST). As the leader of the Smart Fire Fighting Project, Dr. Falkenstein-Smith focuses his research on developing measurement science designed to enhance situational awareness, operational effectiveness and firefighter safety. Having extensive publications in sensor technology, Dr. Falkenstein-Smith has conducted research that enables the fusion of data from various sources with reliable predictive artificial intelligence (AI) models. Dr. Falkenstein-Smith is also an active member of the NFPA’s Electronic Safety Equipment Committee and AI Task Group and the IAFC Technology Council AI sub-committee.
Clint Smith, PE, is the CTO for NextGConnect, focusing on Artificial Intelligence (AI) and Edge Computing with resource constrained devices. He is a licensed professional engineer in New York and New Jersey. He has earned more than 175 patents and is also the author of nine wireless engineering books published by McGraw-Hill. Smith is a member of the NFPA’s Electronic Safety Equipment Committee and AI Task Group. He is an active volunteer firefighter with the Pine Island Fire Department.
Copyright © 2026 Lexipol. All rights reserved.
