Applied Affective Computing

Affective computing is a nascent field situated at the intersection of artificial intelligence with social and behavioral science. It studies how human emotions are perceived and expressed, which then informs the design of intelligent agents and systems that can either mimic this behavior to improve their intelligence or incorporate such knowledge to effectively understand and communicate with their human collaborators. Affective computing research has recently seen significant advances and is making a critical transformation from exploratory studies to real-world applications in the emerging research area known as applied affective computing.

This book offers readers an overview of the state-of-the-art and emerging themes in affective computing, including a comprehensive review of the existing approaches to affective computing systems and social signal processing. It provides in-depth case studies of applied affective computing in various domains, such as social robotics and mental well-being. It also addresses ethical concerns related to affective computing and how to prevent misuse of the technology in research and applications. Further, this book identifies future directions for the field and summarizes a set of guidelines for developing next-generation affective computing systems that are effective, safe, and human-centered.

For researchers and practitioners new to affective computing, this book will serve as an introduction to the field to help them in identifying new research topics or developing novel applications. For more experienced researchers and practitioners, the discussions in this book provide guidance for adopting a human-centered design and development approach to advance affective computing.

Leimin Tian is a postdoctoral research fellow at the Human-Centred AI Group of the Faculty of Information Technology and the Human–Robot Interaction Group of the Faculty of Engineering, Monash University. She received a M.Sc. in Artificial Intelligence in 2013 and a Ph.D. in Informatics in 2018 from the Institute for Language, Cognition and Computation at the School of Informatics, the University of Edinburgh. Her research interests include affective computing, human–robot interaction, multimodal emotion recognition, multimodal behavioral analytics, and dialogue systems. For more information on Leimin's research, please see https://tianleimin.github.io/. Leimin has contributed to formulating the outline of this book and editing it, and is the lead author of Chapters 1, 2, 8, 11, and 12.

Sharon Oviatt is a professor at the Engineering Office of the Dean, Monash University. Her main areas of research are human-centered, multimodal, mobile, and educational interfaces. Sharon is an internationally recognized computer scientist, professor, and researcher known for her work in the field of human–computer interaction on human-centered multimodal interface design and evaluation. She has published over 150 scientific publications in the HCI field and worked as a Professor of Computer Science, Psychology, and Linguistics at several different universities. She has served as an editor for several important HCI journals including Transactions on Interactive Intelligent Systems (TIIS), and she chaired the International Conference on Multimodal Interfaces in 2003. She is a former Professor and Co-Director at the Center for Human–Computer Communication (CHCC) in the Department of Computer Science at Oregon Health & Science University. She currently serves as the President and Chair of the Board of Directors of Incaa Designs, a non-profit with the aim of researching and designing new educational interfaces. She is also a Professor of HCI and Creative Technologies at Monash University in Melbourne, Australia. Much of her research is focused on examining the effectiveness of speech and pen interfaces in educational settings. For more information on Sharon's research, please see https://research.monash.edu/en/persons/sharonoviatt. Sharon has contributed to formulating the outline of this book and editing it, as well as co-authoring Chapters 1, 2, 8, 11, and 12.

Michal Muszynski is a postdoctoral research associate (SNSF fellowship holder) at the University of Geneva, Switzerland, and at Carnegie Mellon University, the United States. He carries out interdisciplinary research at the intersection of computer science, neuroscience, medicine, and psychology. He received his Ph.D. in Computer Science from the University of Geneva in 2018. His research interests are in the areas of multimodal machine learning, affective computing, multimedia analysis, healthcare, human behavior analysis, affective neuroscience, and signal processing. For more information on Michal's research, please see https://michal-muszynski.github.io/. Michal is the lead author of Chapters 3 and 5 of this book.

Brent Chamberlain is an Associate Professor at Utah State University in the department of Landscape Architecture and Environmental Planning. His research spans a wide variety of disciplines with a foundation for research in geovisualization, geospatial science, and environmental psychology. He is the co-director of the Visualization, Instrumentation and Virtual Interactive Design Laboratory. For more information on Brent's research, please see: http://laep.usu.edu/vivid/ and http://brentchamberlain.org/. Brent is the lead author of Chapters 7 and 10 of this book.

Jennifer Healey is a senior research scientist at Adobe. She is most passionate about finding ways to apply technology to solve real-world problems. Her current research focus is around understanding people's emotions in real-world situations by using machine learning algorithms to infer emotional states from sensed data. For more information on Jennifer's research, please see: https://www.linkedin.com/in/healeyjennifer. Jennifer is the lead author of Chapters 6 and 9 of this book.

Akane Sano is an Assistant Professor at Rice University, Department of Electrical Computer Engineering, Computer Science, and Bioengineering. She directs the Computational Wellbeing Group. Her research focuses on human sensing, data analysis and modeling, and intelligent system development for health, well-being, and performance. She is also a member of the Rice Scalable Health Labs. Her research spans the field of affective, ubiquitous, and wearable computing and biobehavioral sensing and analysis/modeling. Her research targets (1) the analysis and modeling of human ambulatory multimodal time series data including physiological, biological, and behavioral data and surveys for measuring, predicting, improving, and understanding human physiology and behavior and human factors such as health, well-being, and performance and (2) development of human-centered computing technologies for health, well-being, and performance. She has been working on developing tools, algorithms, and systems to measure, forecast, understand, and improve health and well-being using mobile and wearable sensors and devices in daily life settings, especially for measuring, predicting, and intervening/improving stress, mental health, sleep, and performance. She received her Ph.D. at MIT Media Lab and her M.Eng. and B.Eng. at Keio University, Japan. Before she joined Rice University, she was a Research Scientist in the Affective Computing Group at MIT Media Lab and a visiting scientist/lecturer at the People-Aware Computing Lab, Cornell University. Before she came to the United States, she was a researcher/engineer at Sony Corporation and worked on wearable computing, intelligent systems, and human–computer interaction. For more information on Akane's research, please see: http://akane.sano.web.rice.edu/. Akane is the lead author of Chapter 4 and assisted on Chapter 9 of this book.