Elicitation The Science and Art of Structuring Judgement

This book is about elicitation: the facilitation of the quantitative expression of subjective judgement about matters of fact, interacting with subject experts, or about matters of value, interacting with decision makers or stakeholders. It offers an integrated presentation of procedures and processes that allow analysts and experts to think clearly about numbers, particularly the inputs for decision support systems and models. This presentation encompasses research originating in the communities of structured probability elicitation/calibration and multi-criteria decision analysis, often unaware of each other’s developments.

Chapters 2 through 9 focus on processes to elicit uncertainty from experts, including the Classical Method for aggregating judgements from multiple experts concerning probability distributions; the issue of validation in the Classical Method; the Sheffield elicitation framework; the IDEA protocol; approaches following the Bayesian perspective; the main elements of structured expert processes for dependence elicitation; and how mathematical methods can incorporate correlations between experts.

Chapters 10 through 14 focus on processes to elicit preferences from stakeholders or decision makers, including two chapters on problems under uncertainty (utility functions), and three chapters that address elicitation of preferences independently of, or in absence of, any uncertainty elicitation (value functions and ELECTRE). Two chapters then focus on cross-cutting issues for elicitation of uncertainties and elicitation of preferences: biases and selection of experts.

Finally, the last group of chapters illustrates how some of the presented approaches are applied in practice, including a food security case in the UK; expert elicitation in health care decision making; an expert judgement based method to elicit nuclear threat risks in US ports; risk assessment in a pulp and paper manufacturer in the Nordic countries; and elicitation of preferences for crop planning in a Greek region.

Luis C. Dias obtained a degree in Informatics Engineering from the School of Science and Technology at the University of Coimbra in 1992, a Ph.D. in Management by the University of Coimbra in 2001, and Habilitation in Decision Aiding Science in 2013 in the same university. He is currently Associate Professor and Vice-Dean for Research the Faculty of Economics, University of Coimbra (FEUC), where he has been teaching courses on decision analysis, operations research, informatics, and related areas. He held temporary invited positions at the University Paris-Dauphine and the University of Vienna. Luis is also a researcher at the CeBER and INESC Coimbra R&D centers, a member of the coordination board of U.Coimbra’s Energy for Sustainability Initiative, and currently a Vice-President of APDIO, the Portuguese Operational Research Society. He is on the Editorial Board of the EURO Journal on Decision Processes and Omega. His research interests include multicriteria decision analysis, performance assessment, group decision and negotiation support, decision support systems, and applications in the areas of energy and environment.

John Quigley has a Bachelor of Mathematics in Actuarial Science from the University of Waterloo, Canada and a PhD in Management Science from the University of Strathclyde, where he is currently Professor.  He is an Industrial Statistician with extensive experience in elicitation of expert judgment to support model development and quantification through subjective probability distributions, having worked closely over the past 25 years with various engineering organizations on problems concerned with risk and reliability.  John has been involved in consultancy and applied research projects with, for example, Aero-Engine Controls, Rolls Royce, Airborne Systems, BAE SYSTEMS and the Ministry of Defense (MOD). His collaborative work on Bayesian model development as part of the Reliability Enhancement Methods and Models (REMM) project is included in the industry standard for reliability growth analysis methods.  John is a tutor for the European Food Safety Agency (EFSA) on Expert Knowledge Elicitation (EKE) as well as being an Associate of the Society of Actuaries, a Chartered Statistician, and a member of the Safety and Reliability Society. 

Alec Morton has degrees from the University of Manchester and the University of Strathclyde. He has worked for Singapore Airlines, the National University of Singapore, and the London School of Economics, has held visiting positions at Carnegie Mellon University in Pittsburgh, Aalto University in Helsinki, and the University of Science and Technology of China (USTC) in Hefei, and has been on secondment at the National Audit Office. His main interests are in decision analysis and health economics.  Alec has been active in the INFORMS Decision Analysis Society and the OR Society. He is on the Editorial Board of Decision Analysis and is an Associate Editor forthe EURO Journal on Decision Processes, the Transactions of the Institute of Industrial Engineers, and OR Spectrum. His research has won awards from the International Society for Pharmacoeconomics and Outcomes Research and the Society for Risk Analysis and from the the INFORMS Decision Analysis Society publication award.