Handbook of Inter-Rater Reliability (3rd Edition): The Definitive Guide to Measuring the Extent of Agreement Among Multiple Raters.

By writing the third edition of the Handbook of Inter-Rater Reliability, my primary goal was to allow researchers and students in all fields of research to be able to access in one place, detailed, well-organized, and readable materials on inter-rater reliability assessment. Chance-corrected agreement coefficients are covered in part I of the book, while part II is devoted to agreement coefficients in the family of intraclass correlations. Part III covers several rank-based measures of association, in addition to discussing agreement within the framework of item analysis.

The methods and techniques developed in this edition of the handbook of inter-rater reliability can handle missing ratings, which are common in most experiments. This is an improvement over the second edition, which describes the methods for complete data sets only. Part II and Part III contain new chapters aimed at providing researchers with a broader coverage of inter-rater reliability techniques. Even chance-corrected agreement coefficients, already covered in the second edition, are presented in the current edition with more depth and more clarity.

I wanted to ensure that the content of this book is accessible to readers with no background in statistics. Based on feedback I received about earlier editions of this book, this goal appears to have been achieved to a large extent. I expect the Handbook of Inter-Rater Reliability to be an essential reference on inter-rater reliability assessment to all researchers, students, and practitioners in all fields of research.