Workshop on Catch Forecast from Biased Assessments (WKFORBIAS; outputs from 2019 meeting)

The workshop on catch forecasts from biased assessments, WKFORBIAS, met to address and develop general guidelines for dealing with the issue of retrospective patterns in stock assessments. Sixteen working papers were presented during the workshop, addressing one or more of the terms of references. The Workshop reaffirmed previous recommendations that retrospective analysis should always be conducted as a diagnostic to examine the internal consistency of an analytical stock assessment. The Mohn’s rho statistic that compares estimates from assessments with recent years of data removed to estimates from the current assessment is the standard tool for retrospective analysis. Examination of recent category 1 and 2 ICES stock assessments indicates that a majority of the assessments does not exhibit strong retrospective patterns that require changes to standard management advice. Across the wide range of stocks examined, no obvious explanatory variables, such as model type, location, fishery type, or biological trait, separated stocks with and without strong retrospective patterns. By comparison, both the magnitude and the proportion of stocks with retrospective patterns were greater in the Northeast US than observed for ICES stocks.

For the stock assessments that do show strong retrospective patterns, the first step was to identify what constitutes a strong retrospective pattern then a decision tree was developed to help expert groups determine a course of action. A number of general recommendations from WKFORBIAS include: 1) when evaluating a retrospective pattern, the consistency of the pattern is of primary importance; 2) a large Mohn’s rho statistic driven by one outlier should not be treated in the same manner as a consistent directional retrospective pattern; 3) retrospective patterns should be viewed as one of many diagnostics to be used in determining whether to use an assessment for management advice or not; 4) a strong consistent retrospective pattern can be the basis for ad-justing catch advice or downgrading the level of an assessment. For stocks that exhibit a strong retrospective pattern where SSB is consistently overestimated and F is consistently underestimated, it is recommended to adjust catch advice. For less common retrospective patterns, where SSB consistently increases with additional years of data, then adjusting catch advice is not rec-ommended. However, in the event an assessment is being downgraded, strong consideration should be given to maintaining models with robust trends even if the scale of the assessment is uncertain (i.e. downgrading from a category 1 to a category 2 assessment); 5) Management Strategy Evaluation can potentially be a useful tool for examining the robustness of harvest control rules to different magnitudes of retrospective pattern and could be useful for situations exhibiting strong retrospective patterns over multiple assessments. However, simulating specific retro-spective patterns is challenging as demonstrated by the Rose approach. A complete set of rec-ommendations are provided in the report including a compiled action list for consideration by expert groups to evaluate possible sources of the retrospective pattern.