Fractional-Order Activation Functions for Neural Networks

This book suggests the development of single and multi-layer fractional-order neural networks that incorporate fractional-order activation functions derived using fractional-order derivatives. Activation functions are essential in neural networks as they introduce nonlinearity, enabling the models to learn complex patterns in data. However, traditional activation functions have limitations such as non-differentiability, vanishing gradient problems, and inactive neurons at negative inputs, which can affect the performance of neural networks, especially for tasks involving intricate nonlinear dynamics. To address these issues, fractional-order derivatives from fractional calculus have been proposed. These derivatives can model complex systems with non-local or non-Markovian behavior. The aim is to improve wind power prediction accuracy using datasets from the Texas wind turbine and Jeju Island wind farm under various scenarios. The book explores the advantages of fractional-order activation functions in terms of robustness, faster convergence, and greater flexibility in hyper-parameter tuning. It includes a comparative analysis of single and multi-layer fractional-order neural networks versus conventional neural networks, assessing their performance based on metrics such as mean square error and coefficient of determination. The impact of using machine learning models to impute missing data on the performance of networks is also discussed. This book demonstrates the potential of fractional-order activation functions to enhance neural network models, particularly in predicting chaotic time series. The findings suggest that fractional-order activation functions can significantly improve accuracy and performance, emphasizing the importance of advancing activation function design in neural network analysis. Additionally, the book is a valuable teaching and learning resource for undergraduate and postgraduate students conducting research in this field. 

Dr. Kishore Bingi is a Senior Lecturer in the Electrical and Electronic Engineering Department at Universiti Teknologi PETRONAS (UTP), Malaysia. He obtained his Bachelor of Technology in Electrical and Electronics Engineering from Acharya Nagarjuna University, India 2012, followed by a Master of Technology in Instrumentation and Control Systems from the National Institute of Technology Calicut, India, in 2014. He earned his PhD in Process Control and Automation from UTP in 2019. After completing his doctorate, Dr. Bingi served as a Research Scientist and Postdoctoral Researcher at UTP's Institute of Autonomous Systems from February 2019 to May 2020. He subsequently joined Vellore Institute of Technology, India, as an Assistant Professor (Senior Grade) in the School of Electrical Engineering, a role he held from June 2020 to September 2022. In November 2022, he returned to UTP as a Lecturer. Dr Bingi's research expertise spans control and automation, process modelling, optimization, fractional-order systems and controllers, fractional-order neural networks, and forecasting. His professional affiliations include membership in the Institute of Electrical and Electronics Engineers (IEEE), the Institution of Engineering and Technology (IET), and the Asian Control Association (ACA). Additionally, he holds the prestigious designation of Chartered Engineer from the Engineering Council, UK.

Bhukya Ramadevi earned her Bachelor of Technology in Electrical and Electronics Engineering from Acharya Nagarjuna University, India, in 2018. She subsequently obtained her Master of Technology in Advanced Power Systems from Jawaharlal Nehru Technological University, India, in 2020. She completed her PhD in the School of Electrical Engineering at Vellore Institute of Technology (VIT), India. Her research focuses on developing fractional-order neural networks and fractional-order long short-term memory (LSTM) networks for time series forecasting and prediction. In addition to her research, she serves as a Teaching cum Research Assistant at VIT Vellore. Her areas of expertise include artificial intelligence, fractional calculus, control and automation, and power systems.

Dr. Venkata Ramana Kasi received his B.Tech. degree in Electrical and Electronics Engineering from JNTU Hyderabad in 2007, followed by an M.Tech. degree in Power Systems from Birla Institute of Technology, Mesra, Ranchi, India, in 2010. He completed his PhD in Electronics and Electrical Engineering at the Indian Institute of Technology (IIT) Guwahati, India 2018. He briefly worked as a Technical Lead at KPIT Technologies, Pune, India. Since 2020, he has been serving as an Assistant Professor in the School of Electrical Engineering (SELECT) at the Vellore Institute of Technology (VIT), Vellore, India. He has published numerous research articles in reputed international journals and presented at both international and national conferences. His research interests include system identification, time-delayed systems, DC-DC converter modeling, and Li-ion battery state-of-charge (SOC) estimation.