Experimentation in Software Engineering

This textbook, which has become the leading source for empirical software engineering research through its earlier editions, provides an in-depth introduction to experimentation in software engineering. The focus is on the steps to go through when conducting experiments. It also introduces a structure for selecting a research design and an introduction to systematic literature reviews, surveys, and case study research. Experiments are valuable tools for all software engineers who evaluate and choose between different solutions, methods, techniques, languages, and tools.

The book has three main parts. Part I introduces empirical research through a structure for selecting a research design and explains several essential areas related to empirical research. Moreover, it provides introductions to systematic literature reviews, experiments, surveys, and case studies. Part II devotes one chapter to each of the five experiment steps: scoping, planning, execution, analysis, and result presentation. Part III completes the presentation with two examples. Assignments and statistical material are provided in appendices. The book provides indispensable information regarding empirical studies, particularly for experiments, but also for systematic literature studies, surveys, and case studies. This new edition comprises substantial new material. This includes a chapter on selecting a research design, which helps put experimentation into a broader context of other research approaches. Furthermore, a new chapter on survey research has been added. Novel contributions concerning A/B testing, replications, open science, validity threats, paired comparison design and tools for statistical analysis have been added. Moreover, systematic literature studies, experiments, and case study research descriptions have been consolidated. The book has also been updated based on the evolution of software engineering since the previous edition's publication.

The book is intended for students, teachers, researchers, and practitioners who would like to compare different options, for example, feature designs, or different development methods, techniques, languages, or tools.  Teachers can use the book for undergraduate or graduate students; the book can be used as a course book on experimentation and empirical studies in general. Researchers will also benefit from the book by learning more about how to conduct empirical studies. Likewise, practitioners may use it as a “cookbook” when evaluating different solutions, alternatively for evaluating new methods or techniques before implementing them in their organisation.

CLAES WOHLIN is Professor Emeritus of software engineering at Blekinge Institute of Technology. He has been conducting empirical research, together with his PhD students, with industrial partners for 35 years. His research focuses on empirical methods in software engineering, software processes and software quality. He served as Editor-in-Chief for the Journal of Information and Software Technology for 14 years. In 2011, he was elected as a member of the Royal Swedish Academy of Engineering Sciences.

PER RUNESON is a professor of software engineering at Lund University. He has more than 25 years of experience in conducting empirical research with PhD students, and leading industry-academia collaboration programs, aiming to understand and improve practice. His research interests embrace cross-disciplinary approaches and include empirical studies on software testing and ecosystems for open data and software, in industrial software and AI engineering. He has advanced the research field through methodology guidebooks on experimentation and case studies.

MARTIN HÖST is a professor of software engineering at Malmö University, where he is dean of the Faculty of Technology and Society. He has about 30 years of experience in empirical research in software engineering, often conducted in collaboration with industry. His research interests include software quality and open source software, as well as empirical research methods in software engineering research.

MAGNUS C. OHLSSON has focused on the development and quality assurance of software for almost 25 years, and today, he is a Quality Strategist and Evangelist at System Verification, specialising in process improvement. He has experience in several different technical areas such as embedded systems, medical devices, mobile network signalling and ERP systems. He is also an appreciated lecturer at different universities and when providing presentations at various conferences.

BJÖRN REGNELL is Professor of software engineering at Lund University. He has over 30 years of experience in software engineering with focus on empirical research in software requirements engineering. He served as Vice Dean of Research at the Faculty of Engineering with focus on quality of graduate level education. He is a member of the editorial board of the Requirements Engineering journal (Springer-Nature). He is also engaged in the governing committee of the Scala open-source programming language.

ANDERS WESSLÉN is an expert at Ericsson in Lund. He has more than 20 years of experience working with system development for both mobile phones and radio base stations. His focus is system-wide architectures and future computing platforms.