Energy Efficiency and Robustness of Advanced Machine Learning Architectures : A Cross-Layer Approach

Machine Learning (ML) algorithms have shown a high level of accuracy, and applications are widely used in many systems and platforms. However, developing efficient ML-based systems requires addressing three problems: energy-efficiency, robustness, and techniques that typically focus on optimizing for a single objective/have a limited set of goals.

This book tackles these challenges by exploiting the unique features of advanced ML models and investigates cross-layer concepts and techniques to engage both hardware and software-level methods to build robust and energy-efficient architectures for these advanced ML networks. More specifically, this book improves the energy efficiency of complex models like CapsNets, through a specialized flow of hardware-level designs and software-level optimizations exploiting the application-driven knowledge of these systems and the error tolerance through approximations and quantization. This book also improves the robustness of ML models, in particular for SNNs executed on neuromorphic hardware, due to their inherent cost-effective features. This book integrates multiple optimization objectives into specialized frameworks for jointly optimizing the robustness and energy efficiency of these systems.

This is an important resource for students and researchers of computer and electrical engineering who are interested in developing energy efficient and robust ML.

The Open Access version of this book, available at http://www.taylorfrancis.com, has been made available under a Creative Commons Attribution-Non Commercial-No Derivatives (CC-BY-NC-ND) 4.0 license.

Alberto Marchisio received his B.Sc. and M.Sc. degrees in Electronic Engineering from Politecnico di Torino, Turin, Italy, in October 2015 and April 2018, respectively. He received his Ph.D. degree in Computer Science from the Technische Universität Wien (TU Wien) Informatics Doctoral College Resilient Embedded Systems, Vienna, Austria, in September 2023. Currently, he is a Research Group Leader with the eBrain Lab, Division of Engineering, New York University Abu Dhabi (NYUAD), United Arab Emirates. His main research interests include hardware and software optimizations for machine learning, brain-inspired computing, VLSI architecture design, emerging computing technologies, robust design, and approximate computing for energy efficiency. He (co-)authored 30+ papers in prestigious international conferences and journals. He received the honorable mention at the Italian National Finals of Maths Olympic Games in 2012, and the Richard Newton Young Fellow Award in 2019.

Muhammad Shafique (M’11 - SM’16) received his Ph.D. degree in Computer Science from the Karlsruhe Institute of Technology (KIT), Germany, in 2011. Afterwards, he established and led a highly recognized research group at KIT for several years as well as conducted impactful collaborative R&D activities across the globe. Besides co-founding a technology startup in Pakistan, he was also an initiator and team lead of an ICT R&D project. He has also established strong research ties with multiple universities in worldwide, where he has been actively co-supervising various R&D activities and student/research Theses since 2011, resulting in top-quality research outcome and scientific publications. Before KIT, he was with Streaming Networks Pvt. Ltd. where he was involved in research and development of video coding systems several years. In October 2016, he joined the Institute of Computer Engineering at the Faculty of Informatics, Technische Universität Wien (TU Wien), Vienna, Austria as a Full Professor of Computer Architecture and Robust, Energy-Efficient Technologies. Since Sep.2020, Dr. Shafique is with the New York University (NYU), where he is currently a Full Professor and the director of eBrain Lab at the NYU-Abu Dhabi in UAE, and a Global Network Professor at the Tandon School of Engineering, NYU-New York City in USA. He is also a Co-PI/Investigator in multiple NYUAD Centers, including Center of Artificial Intelligence and Robotics (CAIR), Center of Cyber Security (CCS), Center for InTeractIng urban nEtworkS (CITIES), and Center for Quantum and Topological Systems (CQTS).

Dr. Shafique has demonstrated success in obtaining prestigious grants, leading team-projects, meeting deadlines for demonstrations, motivating team members to peak performance levels, and completion of independent challenging tasks. His experience is corroborated by strong technical knowledge and an educational record (throughout Gold Medalist). He also possesses an in-depth understanding of various video coding standards and machine learning algorithms. His research interests are in AI & machine learning hardware and system-level design, brain-inspired computing, neuromorphic computing, approximate computing, quantum machine learning, cognitive autonomous systems, robotics, wearable healthcare, AI for healthcare, energy-efficient systems, robust computing, machine learning secrity and privacy, hardware security, emerging technologies, electronic design automation, FPGAs, MPSoCs, embedded systems, and quantum computing. His research has a special focus on cross-layer analysis, modeling, design, and optimization of computing and memory systems. The researched technologies and tools are deployed in application use cases from Internet-of-Things (IoT), Smart Cyber-Physical Systems (CPS), and ICT for Development (ICT4D) domains.

Dr. Shafique has given several Keynotes, Invited Talks, and Tutorials at premier venues. He has also organized many special sessions at flagship conferences (like DAC, ICCAD, DATE, IOLTS, and ESWeek). He has served as the Associate Editor and Guest Editor of prestigious journals like IEEE Transactions on Computer Aided Design (TCAD), IEEE Design and Test Magazine (D&T), ACM Transactions on Embedded Computing (TECS), IEEE Transactions on Sustainable Computing (T-SUSC), and Elsevier MICPRO. He has served as the TPC Chair of several conferences like CODES+ISSS, IGSC, ISVLSI, PARMA-DITAM, RTML, ESTIMedia and LPDC; General Chair of ISVLSI, IGSC, DDECS and ESTIMedia; Track Chair at DAC, ICCAD, DATE, IOLTS, DSD and FDL; and PhD Forum Chair of ISVLSI. He has also served on the program committees of numerous prestigious IEEE/ACM conferences including ICCAD, DAC, MICRO, ISCA, DATE, CASES, ASPDAC, and FPL. He has been recognized as a member of the ACM TODAES Distinguished Review Board in 2022. He is a senior member of the IEEE and IEEE Signal Processing Society (SPS), and a professional member of the ACM, SIGARCH, SIGDA, SIGBED, and HIPEAC. He holds one US patent and has (co-)authored 7 Books, 20+ Book Chapters, 350+ papers in premier journals and conferences, and over 100 archive articles.

Dr. Shafique received the prestigious 2015 ACM/SIGDA Outstanding New Faculty Award, the AI-2000 Chip Technology Most Influential Scholar Award in 2020, 2022 and 2023, the ATRC’s ASPIRE Award for Research Excellence in 2021, six gold medals in his educational career, and several best paper awards and nominations at prestigious conferences like CODES+ISSS, DATE, DAC and ICCAD, Best Master Thesis Award, DAC'14 Designer Track Best Poster Award, IEEE Transactions of Computer "Feature Paper of the Month" Awards, and Best Lecturer Award. His research work on aging optimization for GPUs featured as a Research Highlight in the Nature Electronics, Feb.2018 issue. Dr. Shafique was named in the NYU’s 2021 Faculty Honors List. His students have also secured many prestigious student and research awards in the research community