Report Reviews DHS Approach To Risk Analysis
The Department of Homeland Security (DHS) is responsible for mitigating a range of threats, including terrorism, natural disasters, and pandemics -- and risk analysis is an essential part of fulfilling that responsibility. This report, undertaken by units across the National Research Council, evaluates risk assessment methods employed by the DHS, concluding that risk analysis capabilities are strong in the area of natural disaster response, but substantial improvement is needed in other areas.
Although the conceptual framework used by DHS to analyze risk appears generally appropriate, the deployment of that framework is frequently deficient. The DHS' risk analysis with regard to natural disasters is mature, employing techniques that are subject to quality assurance and control, as well as verification and validation procedures. Risk analysis capabilities with regard to areas beyond natural disasters, however, are not yet adequate for supporting DHS decision-making because their validity and reliability are untested. Recommendations for addressing these deficiencies are offered, including expanding beyond the quantitative approach that has defined DHS risk analysis to this point.
Key Findings
1. A fully integrated analysis that aggregates widely disparate risks by use of a common metric is not a practical goal and in fact is likely to be inaccurate or misleading given the current state of knowledge of methods used in quantitative risk analysis. The risks presented by terrorist attack and natural disasters cannot be combined in one meaningful indicator of risk, and so an all-hazards risk assessment is not practical. The science of risk analysis does not yet support the kind of reductions in diverse metrics that such a purely quantitative analysis would require. Qualitative comparisons can help illuminate the discussion of risks and thus aid decision makers.
2. DHS has established a conceptual framework for risk analysis (risk is a function of threat (T), vulnerability (V), and consequence (C), or R = f(T,V,C)) that, generally speaking, appears appropriate for decomposing risk and organizing information, and it has built models, data streams, and processes for executing risk analyses for some of its various missions.
3. DHS's risk analysis models for natural hazards are near the state of the art. These models -- which are applied mostly to earthquake, flood, and hurricane hazards -- are based on extensive data, have been validated empirically, and appear well suited to near-term decision needs.
4. The basic risk framework of Risk = f(T,V,C) used by DHS is sound and in accord with accepted practice in the risk analysis field. DHS'operationalization of that framework -- its assessment of individual components of risk and their integration into a measure of risk -- is in many cases seriously deficient and is in need of major revision. More attention is urgently needed at DHS to assessing and communicating the assumptions underlying and the uncertainties surrounding analyses of risk, particularly those associated with terrorism. Until these deficiencies are improved, only low confidence should be placed in most of the risk analyses conducted by DHS.
5. With the exception of risk analysis for natural disaster preparedness, the committee did not find any DHS risk analysis capabilities and methods that are yet adequate for supporting DHS decision making, because their validity and reliability are untested. Moreover, it is not yet clear that DHS is on a trajectory for development of methods and capability that is sufficient to ensure reliable risk analyses other than for natural disasters.
For more information, visit http://www.nap.edu/catalog.php?record_id=12972.