Elements of Dimensionality Reduction and Manifold Learning

Dimensionality reduction, also known as manifold learning, is an area of machine learning used for extracting informative features from data, for better representation of data or separation between classes. This book presents a cohesive review of linear and nonlinear dimensionality reduction and manifold learning. Three main aspects of dimensionality reduction are covered -- spectral dimensionality reduction, probabilistic dimensionality reduction, and neural network-based dimensionality reduction, which have geometric, probabilistic, and information-theoretic points of view to dimensionality reduction, respectively. This book delves into basic concepts and recent developments in the field of dimensionality reduction and manifold learning, providing the reader with a comprehensive understanding. The necessary background and preliminaries, on linear algebra, optimization, and kernels, are also explained in the book to ensure a comprehensive understanding of the algorithms.

The tools this book introduces are applied to various applications involving feature extraction, image processing, computer vision, and signal processing. This book is applicable to a wide audience who would like to acquire a deep understanding of the various ways to extract, transform, and understand the structure of data. The intended audiences are academics, students, and industry professionals. Academic researchers and students can use this book as a textbook for machine learning and dimensionality reduction. Data scientists, machine learning scientists, computer vision scientists, computer scientists, statisticians, and mathematicians in the fields of applied mathematics, statistical learning, Riemannian manifolds, subspace analysis, linear algebra, and optimization can use this book as a reference book for both technical and applied concepts. Industry professionals, including applied engineers, data engineers, and engineers in various fields of science dealing with machine learning, can use this book as a guidebook for feature extraction from their data as the raw data in industry often requires pre-processing. Data feature extraction is useful in various fields of science including engineering, physics, chemistry, biometrics, biomedical signals and images, etc.

Benyamin Ghojogh received the B.Sc. degree in electrical engineering from the Amirkabir University of Technology, Tehran, Iran, in 2015, the M.Sc. degree in electrical engineering from the Sharif University of Technology, Tehran, Iran, in 2017, and the Ph.D. in electrical and computer engineering (in the area of pattern analysis and machine intelligence) from the University of Waterloo, Waterloo, ON, Canada, in 2021. He was a postdoctoral fellow, focusing on machine learning, at the University of Waterloo, in 2021. His research interests include machine learning, dimensionality reduction, manifold learning, computer vision, data science, and deep learning.

Mark Crowley:

Mark Crowley has a PhD in Computer Science from the University of British Columbia and was a postdoctoral fellow at the Oregon State University. He is now an Associate Professor in the Department of Electrical and Computer Engineering at the University of Waterloo and regularly teaches undergraduate and graduate courses on software programming, artificial intelligence, and data analysis. He is a member of the Waterloo Artificial Intelligence Institute. He carries out research to find dependable and transparent ways to augment human decision making in complex domains, especially in the presence of spatial structure, streaming data, and uncertainty. His research group focuses on developing new algorithms within the fields of reinforcement learning, deep learning, and manifold learning. This often involves collaboration with industry and policy makers in diverse fields such as sustainable forest management, ecology, autonomous driving, physical chemistry, and medical imaging.

Fakhri Karray:

Fakhreddine (Fakhri)  Karray  is the Loblaws Research Chair in Artificial Intelligence in the department of electrical and computer engineering at  the University of Waterloo, Canada. He is the founding co-director of the University of Waterloo AI Institute. He is currently serving as the Provost and Professor of Machine Learning at the Mohamed bin Zayed University of Artificial Intelligence, a first of its kind graduate level, research based artificial intelligence university. Fakhri’s research interests are in the areas of advances in machine learning, operational AI, cognitive machines, natural human-machine interaction, autonomous and intelligent systems. Applications of his research include virtual care systems, cognitive and self-aware machines/robots/vehicles, predictive analytics in supply chain management and intelligent transportation systems. Recent work of Fakhri and his research team on deep learning-based driver behavior recognition and prediction has been featured on The Washington Post, Wired Magazine, Globe and Mail, CBC radio and Canada's Discovery Channel. He was honored in 2021 by the IEEE Vehicular Technology Society (VTS) for his novel work on improving traffic flow prediction using weather Information in connected cars through deep learning and tools of AI and received the Society’s 2021 Best Land Transportation Paper Award.

Fakhri is the co-author of a textbook on applied artificial intelligence: Soft Computing and Intelligent Systems Design (Pearson Education Publishing, 2004). He has published extensively in the general field of pattern analysis and machine intelligence and is the author of 20 US registered patents. He is the Associate Editor (AE) of flagship journals in the field of AI and intelligent systems, including  the IEEE Transactions on Cybernetics, the IEEE Transactions on Neural Networks and Learning Systems and the IEEE Computational Intelligence Magazine. He served as the AE and Guest Editor for the IEEE Transactions on Mechatronics, the IEEE Computational Intelligence Magazine and IEEE Access (special issue on IoMT). He also serves on several editorial boards of AI-related journals and has served as the General Chair/Program Chair for several international conferences in the field of intelligent systems.  Fakhri is the co-founder and Chief Scientist of Yourika.ai, a provider of AI based online learning systems. He is a Fellow of the IEEE, a Fellow of the Canadian Academy of Engineering, a Fellow of the Engineering Institute of Canada and a Fellow of the Kavli Frontiers of. He received his PhD from the University of Illinois Urbana-Champaign, USA, and completed his undergraduate engineering degree at the National Engineering School of Tunis, Tunisia.

Ali Ghodsi:

Ali Ghodsi is a Professor of Statistics and Computer Science at the University of Waterloo in Ontario, Canada, and a member of the Waterloo Artificial Intelligence Institute. His current research sweeps across a broad swath of AI encompassing machine learning, deep learning, and dimensionality reduction. He regularly teaches courses on these topics. He studies theoretical frameworks and develops new machine-learning algorithms for analyzing large-scale data sets, with applications in natural language processing, bioinformatics, pattern recognition, computer vision, and sequential decision making. Dr. Ghodsi's work has been published extensively in high-quality proceedings and journals, and he is the co-author of several US patents. His popular lectures on YouTube have more than one million views.