Fin condition is a widely accepted welfare indicator for farmed salmonids and a variety of other farmed finfish species. To assess fin condition, several different assessment systems have been proposed. Some of these systems are fairly simple, while others are more complex and sophisticated. What these systems all have in common, is that they assess fin condition based on ordinal scales. However, little is known about the relationship between the complexity of the assessment system, and its repeatability in terms of intra- and inter-observer reliability. To obtain insight into the relationship between complexity, repeatability, and reliability, we conducted a large-scale image-based double-blind study about the fin condition in farmed rainbow trout.
Four different questionnaires, reflecting four different assessment systems (3 to 6 scores) were distributed to 499 stakeholders of the German-speaking aquaculture sector (Germany, Switzerland, Austria). Each questionnaire consisted of 25 high-resolution color images of the tail fin of farmed rainbow trout of regular marketing size. Each picture was duplicated and randomly placed twice within the questionnaires. The questionnaires could be filled electronically and an automatic return button was implemented on the last page. We used relative agreement and Gwet’s AC1 agreement coefficient to calculate reliability within each participant (intra-observer) as well as between participants of the different groups (inter-observer).
A return rate of 15.9% was accomplished. Across all received questionnaires we identified four relevant groups. 1. fish farmers; 2. fish health services; 3. scientists; 4. others. Across all groups, we identified a negative relationship between the complexity of the assessment system and the ability of the participants to allocate duplicated pictures into the same category.
When analyzing participants combined, the data suggested a negative exponential relationship between complexity and reliability (Figure 01). Two of the groups, fish health services, and scientists, were able to allocate the duplicated pictures repeatedly with statistically proven accuracy (AC1 0.67-0.87). However, the reliability of these two groups dropped below the statistical benchmark (agreement coefficient ≥0.61), when the assessment system was based on more than 4 scores.