• Home
• Lessons
  • 1. Review of PRA and Introduction to Risk Assessment
  • 2. Resources Used to Conduct Pest Risk Assessment
  • 3. Conducting Pest Risk Assessment for Trade
  • 4. Practice
  • 5. Module Conclusion
• Topics
  • 1. Why is Pest Risk Assessment Important?
  • 2. Risk Assessment and PRA
  • 3. The International Framework for PRA and Pest Risk Assessment
  • 4. Common Approaches for Conducting Pest Risk Assessment
• Resources
  • Help
  • Glossary
  • Documents and Videos
  • Acknowledgements

Lesson 1: Review of PRA and Introduction to Risk Assessment

Topic 4: Common Approaches for Conducting Pest Risk Assessment

We have discussed that pest risk assessments are initiated for many different reasons. For example, the following are reasons for why PRA’s are done:

1. A request has been made by a trading partner to export a specific commodity to an importing country
2. A specific organism has been introduced into an area recently, or could be introduced, and has been identified as a potential pest
3. Changes to policies or operational approaches

Next, let us consider how pests risk assessments are done. Analysts perform pest risk assessments using one or more analytical method. These methods are, in the broadest sense, either qualitative or quantitative.

Interestingly, the international and regional standards of PRA do not prescribe how to assess risk. This means that analysts and organizations must use their best judgment in selecting the most appropriate method(s). The standards provide the specific elements that should be considered in risk assessment, and they outline the minimum requirements that should be met in preparing and documenting a pest risk assessment. The choice of method is driven by these factors:

• Nature of the issue to be analyzed
• Purpose of the PRA
• Intended audience
• Availability, type, and quality of information, evidence, and data
• Time and expertise available

In a perfect world, we would have ample information, plenty of time and resources, a clearly stated purpose, and a well-understood audience for every pest risk assessment. However, in reality, we are faced with constraints on time, information, and resources. Sometimes the audience and purpose of the assessment are not clearly defined. All of these factors will combine to determine the sort of assessment that should be done and influence the method(s) we select.

Next is a brief discussion of qualitative, quantitative, and semiquantitative assessment approaches.

Qualitative Approaches

In qualitative risk assessment, both the likelihood of introduction and consequences are measured in relative terms. This is accomplished by either ranking the hazards (i.e., quarantine pests) relative to each other or by placing them into descriptive categories such as high, medium, or low.

Qualitative risk assessment methods typically use scientific evidence, experience, and expert judgment to evaluate risk. The output of a qualitative risk assessment generally involves making subjective judgments about the likelihood of introduction and the magnitude of potential consequences in order to provide a description of the overall risk.

Qualitative methods are often effective and easy to understand. They can be done with a minimum of information and resources and can be useful for identifying subjects that might need more in-depth assessment. Qualitative methods can be applied to a variety of approaches for assessment, including:

• Narrative descriptions and reports
• Decision sheets
• Generic rating systems (e.g., Low, Medium, High)
• Keys or schemes
• Weighted evaluations

We will now discuss two common types of qualitative assessments: narrative assessments and those based on guidelines or a template with predefined criteria.

Narrative Assessment

A narrative risk assessment is a type of qualitative pest risk assessment. Often simpler than other types of assessments, the narrative assessment usually takes the form of a short report that outlines the specific issue of concern and provides a verbal description of risk. Specifically, the narrative assessment:

• Describes the likelihood that a pest will be able to enter, establish, and spread
• Explains the potential economic consequences
• Describes any uncertainty in the assessment

Narratives should be structured in such a way as to provide a logical flow, but otherwise the narrative format is extremely flexible.

In a narrative report, consequences may be described qualitatively (low, medium, high, or acceptable/unacceptable) or quantitatively if they can be reasonably estimated. For example, the analyst may say “the potential impacts from the pest would be high because it attacks many important crop plants” or the analyst may say “the estimated trade impacts from the pest could exceed a value of X dollars.”  

Decision sheets are one common type of narrative report that usually follows a specific format or template. Decision sheets are often used to provide quick, simple summaries to risk managers and decision-makers regarding a particular pest or situation. They use a highly structured format that makes it easy to find and interpret information and to formulate recommendations for decisions.

The following is a sample decision sheet that is used for analyzing pests that are newly introduced or have the potential to be introduced into a PRA area. Note that this template includes the basic elements that should be analyzed in a PRA: likelihood of entry, establishment and spread, potential economic consequences, and uncertainty.  

*Adapted from the USDA PPQ New Pest Advisory Group Report template (USDA, 2010)

 Advantages

There are several advantages to using a narrative approach to pest risk assessment.

1. The requirements for quantitative data and information are very limited for narrative reports compared to other types of assessments. The narrative is particularly well suited to situations where quantitative information is unavailable.
2. Analysts can prepare these types of reports relatively quickly. In fact, narrative reports are often used as preliminary analyses to decide whether more in-depth (and time-consuming) analyses should be conducted.
3. Narrative reports do not require special expertise or other specialized resources (e.g., computer programs). They are an accessible type of assessment that anyone can prepare.
4. Narrative reports are easy to understand and use, and they are effective at meeting at least the lowest requirements for technical justification under the SPS Agreement and IPPC.

 Disadvantages

Narrative reports have some disadvantages, too.

1. The criteria used for assigning the qualitative terms in narrative reports may be undefined. For this reason, qualitative results can be difficult to compare from one assessment to another. Analysts use their own discretion in employing terms like high, medium, low, or negligible rather than using specified ranking criteria. However, using decision sheets and templates can help to reduce problems arising from the inconsistency of narrative reports and help to increase transparency.
2. Narrative reports can have variable formats and information, so they might vary from one report to another, thus making comparisons difficult.
3. The results of a narrative assessment are not precise. They are based on general information, and therefore can only provide rough estimations and summaries of risk.
4. Narrative analyses do not clearly specify the appropriate level of protection. Additionally, they can be less transparent than other methods if the rationale for making judgments or drawing conclusions is not clearly described. They are less transparent than other methods as a basis for risk management decisions.

Qualitative Assessments Based on Guidelines

This approach to qualitative risk assessment uses a set of predetermined/preestablished criteria to evaluate the likelihood of introduction, as well as the magnitude of the consequences. Each pest is evaluated in terms of the predefined criteria and is usually then placed into descriptive categories (such as high, medium, or low) or given a rating (such as 3, 2, 1).

There are many different examples of this type of qualitative assessment, as this is the model that many countries use to conduct pest risk assessment. A few include the following:

• EPPO pest-based scheme
• CAPRA (computer-assisted PRA)
• USDA-APHIS Guidelines for Commodity Import Assessment
• USDA-APHIS Weed Risk Assessment Guidelines

An in-depth discussion of how criteria are developed is beyond the scope of this module, but it is important to note that the success of these models depends on developing easy to understand rules for rating. The rules themselves increase transparency and objectivity and provide additional analytical support to decision makers. Additionally, any criteria developed for a particular risk element or event should provide all the guidance analysts need to make consistent judgments. Those criteria also should be flexible enough that analysts can apply them to a wide range of situations. Criteria may also include examples to aid the analysts in making judgments.

 Advantages

1. Generally, a qualitative assessment is more transparent than a narrative approach if the criteria and rules are well defined.
2. It is more consistent than narrative approaches. The results of the risk assessment can be compared to other assessments that use the same criteria.
3. It is easy to learn.
4. It is faster to complete than most quantitative assessments and often faster than narrative assessments, since the criteria for making decisions is already developed.

 Disadvantages

1. The variety of pests, commodities, pathways, and other factors makes it impossible for criteria to fully cover all situations. Pests or situations will arise that were not anticipated or addressed when the criteria were developed.
2. The nature of the guidelines approach is that is has limited flexibility since criteria are predefined. This often forces information and conclusions to fit in categories that may not be appropriate or requires the analyst to select from options that are not most applicable to the situation they are analyzing.
3. It takes a lot of time up front to develop appropriate criteria.

Quantitative Assessments Based on Guidelines

In a quantitative risk assessment, the analyst uses numerical values together with formulas or models to calculate a result that quantitatively characterizes the risk, some aspect of the risk, or some effect that mitigates the risk. They are especially useful when:

• Comparing risks before and after
• Building expert consensus
• Expressing specific uncertainty
• Evaluating incremental changes
• Evaluating complex systems
• Examining disagreement or confusion on the degree of risk expressed in a qualitative assessment

There are many different models and methods the can be used for quantitative risk assessment. An in-depth discussion of these methods is beyond the scope of this module. The most common method used in phytosanitary risk assessments is probabilistic scenario analysis. When this method is used, risk is generally expressed as a probability or distribution of probabilities.

Quantitative approaches can be extremely powerful tools, but often they are not appropriate for phytosanitary risk assessments.

There is little specific guidance available to pest risk analysts about when, where, and how to apply quantitative methods. For the most part, there are no quantitative risk assessment templates available because quantitative assessments are mostly unique, having been developed to address specific and non-routine problems.

 Advantages

1. A quantitative approach can add layers of detail and understanding, particularly where there is disagreement or misunderstanding about the qualitative descriptors of risk.
2. Uncertainty can be explicitly modeled.
3. Results can be compared to other analyses.
4. It is very useful for prioritization and resource allocation.

 Disadvantages

1. It can be resource intensive, both in terms of data requirements and time.
2. There is a complicated methodology: can be difficult to explain, interpret, and review, particularly when audience is nontechnical.
3. It often requires specialized software and expertise.
4. It may not give you a better answer, and in some cases can actually cause more problems. For example, two different dispute settlement cases illustrate some of the issues regarding the use of quantitative methods (see WTO Dispute Settlement page, accessible at http://www.wto.org/english/tratop_e/dispu_e/dispu_e.htm). In one case involving Australia and salmon imports, the findings specifically noted that qualitative methods are sufficient for demonstrating risk. In a case involving Australia, apples, and New Zealand, the findings of the dispute noted that the quantitative methods used had overestimated risk.

There is currently much debate within the phytosanitary community about whether quantitative approaches are more objective than qualitative methods because they are based on numerical data rather than expert judgment. Some argue they are not necessarily more objective; they sometimes rely on estimates based on expert judgment. In addition, the models developed for quantitative approaches can also have subjective elements. They incorporate assumptions and judgments and can be easily misinterpreted if the underlying assumptions are not clearly presented and understood.

To continue, select Lesson 2 from the Lessons menu above or click here.