The effects of rater bias on hypothesis testing when using different assessment methods for estimating disease severity.

Bias (over and underestimates) in estimates of disease severity, and the impact of that inaccuracy on hypothesis testing using different disease scales was explored. Nearest percent estimates (NPE), the Horsfall-Barratt (H-B) scale and four different linear category scales (5% and 10% increments, with and without additional grades at low severity) were compared. Actual values and estimates by 4 different raters of the severity (0 to 100%) of Septoria leaf blotch on leaves of winter wheat were used to develop distributions for a simulation model. The simulations were based on i) all the 4 raters data combined, ii) only the most accurate rater estimates, and iii) only the most biased rater. Regardless of the effect of rater ability, we found that, there were lower type II error rates with NPEs as compared with the other category scales at severities of 80 to 100%. On the other hand, with lower severities (0 to 20%), the 5% and 10% scales with additional grades had type II error rates comparable to those for the NPEs. Raters who overestimated severity and used the H-B scale had the highest risk of a type II error when the mean disease severity was low. Knowledge of how rater ability and scale type can affect hypothesis testing can be used to improve disease assessment as well as to provide a logical framework for developing standard area diagrams.