Technical Service - EU request on providing output on evaluating data accuracy (precision and bias) for design-based estimation at a national level in the form of a report

ICES aims to support EU Member States in evaluating the accuracy of their catch sampling data, where accuracy refers to the closeness of statistical estimates to their true values. ICES considers data accuracy in terms of two components: precision and bias. Random uncertainties inherent in sampling are described by data precision, while systematic differences between an estimate and a true value are described by bias. Since this is a complex subject and sampling programmes are usually implemented differently in different countries, the tools presented relate only to national probabilistic sampling and design-based estimation. To use the tools, Member States will need to convert their national data to the format of the Regional Database and Estimation System (RDBES), which stores commercial fisheries data.
ICES evaluates the data precision using two complementary techniques. For relatively simple sampling designs it is possible to use analytical functions to calculate the precision (or a related statistical measure such as variance) of a statistical estimate. The calculations and implementations are written in R code. For more complicated sampling designs, the use of analytical functions is usually not feasible. In these cases, it is necessary to evaluate precision using a numerical technique known as bootstrapping. The elaborated discussion concerning when bootstrapping is appropriate is given in the report underpinning this Technical Service (ICES, 2020), along with a number of worked examples describing how bootstrapping can be applied in different cases.
The evaluation of bias is a difficult subject and is hard to quantify. The approach used builds on previous work available in ICES literature to identify and enumerate common potential sources of bias in catch sampling programmes. As presented in Annex 4 of the report (ICES, 2020), these sources of bias were first collated and then evaluated to see whether data stored using the RDBES format could inform about that bias source. Reports are presented to help Member States identify deviations in their sampling programmes and sampling variability that can potentially lead to bias in catch estimates.
This service is a first step towards providing Member States with a set of tools that can be used to characterize the precision of and bias in their catch sampling data. The aim is to provide a solid foundation which, while immediately useful in itself, has greater value as a building block for future work. A summary of the further activity that is required to extend this work to other scenarios (such as regional sampling programmes) is presented in the report, along with a roadmap.