Review on EMG based Hand Gesture Recognition using Deep Learning

Change may be seen as a sign of physical or emotional attitude. In order to function in daily life, humans need to be able to move. Movements are crucial for carrying out tasks and communicating. MG is used to analyse and compare the neurological effects of various muscle contraction mechanics. Noninvasively recorded sEMG signals are rapidly being used to drive myoelectric-based devices and rehabilitation equip- ment used in prosthetics, treatment, and diagnostics. Using electromyography (EMG) signals, the technology called EMG- based Hand Gesture Recognition with Deep Learning can anal- yse and comprehend human hand movements. To record the electrical activity produced during muscle contractions, EMG sensors are positioned on the skin’s surface, usually close to the forearm muscles. In order to extract crucial information about hand motions, these EMG signals are then processed and analysed using deep learning techniques, such as convolutional neural networks (CNNs) or recurrent neural networks (RNNs). The deep learning models can learn complicated patterns and correlations between the signals and gestures since they are trained on massive datasets of EMG recordings linked with related hand motions. Once trained, the models can quickly and reliably identify and recognise a wide range of hand gestures, enabling natural and effective human-computer interaction. Deep learning- based EMG-based hand gesture recognition opens up new opportunities for enriching user experiences and quality of life in a variety of industries, including prosthetics, virtual reality, and human-robot interaction. Human-computer interaction has been completely transformed thanks to EMG-based hand gesture recognition utilising deep learning, which offers a non- intrusive and extremely accurate approach for analysing hand movements. This technique uses the processing and analysis capabilities of deep learning algorithms, such as deep neural networks (DNNs) or recurrent neural networks (RNNs), to deal with the complex EMG signals produced by the forearm muscles. These signals convey important details about the purpose and execution of particular hand motions. The deep learning algorithms can learn the intricate mapping between the signals and the desired gestures by being trained on large datasets of EMG recordings and corresponding hand gestures. The trained models can then quickly and accurately categorise and recognise a variety of hand motions. This technology has a wide range of uses, including virtual reality, robotics for rehabilitation, and assistive technologies for those with motor disabilities. Through intuitive and natural hand motions, EMG- based Hand Gesture Recognition utilising Deep Learning has the potential to revolutionise how we interact with machines. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs), in particular, have shown to be highly accurate at recognising hand motions and extracting useful characteristics from raw EMG data. Gathering a labelled dataset of EMG signals captured as people make various hand motions is the initial step in the process. The training data for the deep learning model comes from this dataset. The deep learning model is made to identify intricate correlations and patterns in the EMG signals. RNNs can describe temporal relationships over time, whereas CNNs are excellent at capturing spatial dependencies in the EMG data. combining hybrid architectures. The deep learning model gains the ability to link particular EMG signal patterns with matching hand motions throughout the training phase. Through gradient descent and backpropagation, the model modifies its internal parameters to improve performance iteratively. When the model has been trained, it can be used in real-time applications where it can accept raw EMG signals as input and output a recognised hand gesture. Deep learning-based EMG-based hand gesture detection has several uses, including prosthesis control and rehabilitation systems, virtual reality interfaces, and smart home automation. It provides people with a simple, intuitive approach to use their natural hand movements to engage with technology. Future gesture recognition systems will be more seamless and reliable thanks to ongoing research that aims to significantly improve the models’ accuracy and robustness.

II. EMG-Based Hand Gesture Recognition:

The fundamentals of electromyography are described in this section, along with how it can be used to record muscle activity. Signal acquisition, noise removal, and feature extraction are discussed along with the benefits and drawbacks of EMG-based hand gesture detection. Electromyography (EMG) signals are used to recognise and categorise hand movements in a method known as EMG-based Hand Gesture Recognition. The electrical activity of muscles during hand motions pro- duces EMG signals. These signals can be analysed to identify information that can be utilised to distinguish between various hand movements. [18]

Key applications include the following:

1. Human-Computer Interaction: Without the use of physical input devices, EMG-based hand gesture recognition can enable intuitive and natural interaction with computers, cellphones, and other electronic devices. In gaming, virtual reality, and augmented reality contexts, it can improve user experience and make gesture-based control possible.

2. Prosthetics and Rehabilitation: The creation of prosthetic devices can make use of EMG-based hand gesture recognition. Amputees can regain functional control over their prosthetic limbs because the prosthesis can imitate the desired hand ges- tures by deciphering the user’s muscle signals. Additionally, EMG-based systems in rehabilitation settings can help with the restoration of motor functions by giving biofeedback during therapy sessions.

3. Assistive Technologies: It is possible to include hand gesture identification based on EMG into assistive technology for people with disabilities. These technologies let people with limited mobility operate equipment, communicate, and carry out everyday duties more independently by reading hand gestures.

III. EMG Signal Collection

The sEMG signals utilised to evaluate hand gestures were recorded in this study using the BIOPAC MP36 instrument. To record EMG signals, electrodes are placed close to the muscle groups. The electrodes, skin, and muscle all move in respect to one another while a muscle contracts, shortening its length. At

Fig. 1.

that moment, the electrodes will start to show some movement artefacts.The frequency range of the motion noise is typically 1–10 Hz, and the voltage is comparable to the amplitude of the EMG (fig. 1). Recessed electrodes can significantly lessen the movement artefact by placing a conductive gel layer between the skin’s surface and the electrode-electrolyte interface. A different type of movement artefact occurs as a result of the possibly varying skin layer thicknesses. Recessed electrodes cannot be used to remove this phenomenon. The ability to distinguish an authentic EMG signal that originates in the muscle is lost as a result of the blending of multiple noise signals or artefacts. The properties of the EMG signal are influenced by the internal structure of the person, including the particular skin formation, blood flow velocity, recorded skin temperatures, tissue structure (muscle, fat, etc.), the measurement site, and more. These properties result in a variety of noise signals, which are present in the EMG data. This could affect the results of feature extraction, which would then affect how the EMG signals are identified.

LITERATURE SURVEY

Mehmet Akif OZDEMIR, Deniz Hande KISA, and Onan GUREN, Aytug ONAN, Aydin AKAN proposed using EMG based Hand Gesture Recognition using Deep Learning [19]. In this study, a unique deep learning-based approach has been created to boost prediction accuracy for hand movements. As 30 people mimicked the seven diverse hand movements of extension, flexion, open hand, punch, radial deviation, rest, and ulnar deviation, 4-channel surface EMG (sEMG) signals were first captured (fig. 2). Each movement was detected in a different location of the recorded sEMG signals. The segmented sEMG signals were then used to create spectrogram images using the ShortTime Fourier Transform (STFT). The created coloured spectrogram images were trained using a 50- layer Convolutional Neural Network (CNN) based on Residual Networks (ResNet) architecture. The suggested method produced an F1 Score of 99.57 percent and test accuracy of 99.59 percent for seven different hand motions. Even without complex pre-processing and feature extraction, it demonstrated better accuracy when compared to the widely used conventional techniques.

Fig. 2. Seven different hand gestures used and 2D spectrogram images of segmented sEMG signals created by using STFT

Zhihua Chen, Jung-Tae Kim, Jianning Liang, Jing Zhang, and Yu-Bo Yuan proposed using Real-Time Hand Gesture Recognition Using Finger Segmentation [5]. In this study, we describe a revolutionary real-time method for hand motion detection. And to distinguish the hand region from the back- ground, the framework employs the background subtraction technique. The fingers are then located and identified by dissecting the palm and fingers. Following that, a rule classifier is utilised to predict the labels of hand movements. The tests on the dataset of 1300 photographs show that our method is effective and efficient. Furthermore, our strategy outperforms a state-of-the-art method when used on a different set of hand gesture data.The hand region is separated from the surrounding area using the background subtraction technique. The fingers and palm are then separated into parts. Segmentation is used to locate and identify the fingers in the hand image. For seven separate hand motions, the suggested strategy resulted in an F1 Score of 99.57 percent and test accuracy of 99.59 percent. When compared to the widely-used conventional ap- proaches, it displayed greater accuracy even without intricate pre-processing and feature extraction. give depth information that could improve the efficiency of hand detection. Md. Mehedi Hasan, Arifur Rahaman, Md. Faisal Shuvo, Md. Abu Saleh Ovi, Md. Mostafizur Rahman proposed using Human Hand Gesture Detection Based on EMG Signal Using ANN [12].By analysing the myoelectric signals generated by the movement of the flexor carpi radialis tendon and flexor digitorum superficial tendons in the upper part of the wrist connected to the forearm, as well as the brachioradialis muscle and antebrachial vein in the lower part of the forearm, the pa- per proposes an improved method of hand gesture recognition based on segmentation and backpropagation algorithms. Received Electromyography (EMG) signals from the forearm are segmented into their individual components, and the resulting evaluated values are fed into a feed-forward artificial neural network that functions as a classifier to produce the distinct signals or commands required for communication. To ensure low error, the neural network is trained using a range of signals from different sources and then tested with new signals. With no consideration of depth or distance, the suggested method will provide a balanced selection of classified signals for modest muscular signal changes for patient monitoring by virtual command. Cot e-Allard, Cheikh Latyr Fall, Alexandre Drouin, Alexan- dre Campeau-Lecours, Clement Gosselin, Kyrre Glette, Franc¸ois Laviolette ´ †, and Benoit Gosselin† proposed using Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning [7].The major goal of this research is to lessen the effort of recording while enhancing gesture recognition by learning broad, instructive qualities from the massive amounts of data generated by combining the signals of many users. This study recommends employing deep learning algorithms’ capacity to learn discriminative features from enormous datasets in combination with transfer learning on data that has been combined from a number of users. For this investigation, two datasets with a combined total of 19 and 17 physically fit individuals were recorded using the Myo Armband (the first dataset is utilised for pre-training). The third Myo Armband dataset was obtained from the NinaPro database and consists of ten healthy subjects. Three distinct deep learning networks are tested using three different input modalities (raw EMG, Spectrograms, and Continuous Wavelet Transform (CWT)) on the second and third datasets. On the two datasets, it is shown that the proposed transfer learning scheme significantly and systematically enhances the performance for all three networks, achieving offline accuracy for the CWT-based ConvNet of 98.31 percent for 7 gestures over 17 participants and for the raw EMG-based ConvNet of 68.98 percent for 18 gestures over 10 participants. Later efforts will modify the suggested TL algorithm and test it on amputees with upper extremities. This will bring additional challenges because to the greater muscle variability among amputees and the worse classification accuracy in comparison to able-bodied participants. Engin Kaya, Tufan Kumbasar proposed using Hand Gesture Recognition Systems with the Wearable Myo Armband [13]. In order to categorise and recognise the numbers from 0 to 9 in Turkish Sign Language, this study investigates a hand gesture recognition technique based on surface electromyography (EMG) data collected from a wearable device, specifically the Myo armband. To do this, we have employed machine learning strategies to recognise the hand motions. In this context, a sliding window method is used to extract seven different time domain properties from the raw EMG signals in order to acquire separate information. The study that is being presented covers the design, implementation, and comparison of the k- nearest neighbour, support vector machine, and artificial neural network machine learning algorithms. In this instance, each Myo armband channel yielded seven-time domain properties that together created a 56-dimensional feature matrix. The PCA method was used to reduce the dimension. Then, the generated feature matrix was classified using the kNN, SVM, and ANN algorithms. The recommended method will be tested in future research using recording signals from diverse people and for more complex hand movements. Athira Devaraj and Aswathy K Nair proposed using Hand Gesture Signal Classification using Machine Learning [8]. This study focuses on detecting a particular hand motion from an EMG signal that was obtained using sensor-based technology. For the goal of identification and classification, surface EMG and machine learning techniques [10] are applied. To prevent noise artefacts, an appropriate preprocessing procedure is first applied to the raw EMG signal that was acquired by the sensor [1]. The complete system is implemented using MATLAB 2019a. By using these strategies, KNN and SVM both attain encouraging accuracy rates of 93 perecnt and 83 percent, respectively. Many preprocessing and feature extraction approaches have been used to increase the accuracy of classification systems. A hand motion that matches an EMG signal acquired with a Myo armband [11] will be recognised by the model. It is essentially vital to remove noises like motion artefacts from the EMG data that we collect in order to obtain accurate results. Remedializing, filtering, and detrend- ing are the steps of the total preprocessing. Rectification is necessary since the signal of interest displays both positive and negative polarisations. We adjust the signal to make it positively polarised in order to avoid the average becoming zero. The classification is carried out using SVM and KNN classifiers, and the effectiveness of the methods is contrasted [16]. Ulysse Cote-Allard, Cheikh Latyr Fall, Alexandre Campeau- Lecours, Clement Gosselin, Francois Laviolette and Benoit Gosselin prposed using Transfer Learning for sEMG Hand Gestures Recognition Using Convolutional Neural Networks [6]. In this investigation, the Myo wristband (Thalmic Labs), a low-cost, low-sampling rate (200Hz), 8-channel, consumer- grade dry electrode sEMG device, was used to record two datasets of 18 and 17 healthy volunteers, respectively. Con- volutional neural networks (CNNs) are improved via transfer learning techniques using inter-user data from the original dataset, reducing the burden of data generation imposed on a single person. The results show that the proposed classifier (when combined with orientation data) is dependable and accurate enough to control a 6DoF robotic arm with the same speed and precision as a joystick. CNNs are utilised to identify gestures instead of a more traditional approach, shifting the focus from feature engineering to feature learning. The feature extraction is a natural part of a CNN, therefore there are a lot fewer calculations required before giving the classifier the input. Additionally, since the proposed technique will eventually need to run on an embedded system, major ef- forts were made to reduce the pre-processing workload. Thus, the pre-processing was condensed to simply computing the spectrograms of the raw sEMG data. [22]. Future research will concentrate on the classifier’s long-term use with unsupervised techniques to reduce the requirement for cyclical calibration. Lijing Lu, Jingna Mao, Wuqi Wang, Guangxin Ding, and Zhiwei Zhang proposed using A Study of Personal Recognition Method Based on EMG Signal [14]. This study looks at two EMG-based personal identification and verification techniques. In order to record the surface EMG signal during a hand-open gesture, the researchers first attached a Myo arm- band to the right forearm of 21 participants (more specifically, at the height of the radio humeral joint). Two different strategies were then put out for EMG-based personal identification. Discrete Wavelet Transform (DWT) and Extra Trees Classifier were used in the first strategy, whereas Continuous Wavelet Transform (CWT) and Convolutional Neural Networks (CNN) were used in the second strategy [2]. It was discovered through testing with the 21 subjects that both approaches had great identification accuracy, achieving 99.206% and 99.203%, respectively. A transfer learning approach is then used to the identification method that uses Continuous Wavelet Transform (CWT) and Convolutional Neural Networks (CNN) [17] in order to overcome the difficulty of updating the model with new data. The researchers also present an EMG-based personal verification technique that makes use of CWT and siamese networks. Experimental findings show that the accuracy of this verification approach is 99.285 %. The electromyography (EMG) signal, for example, is a biometric with living body properties that can be used to detect aliveness and thwart spoofing assaults. There are, however, not many studies on EMG-based personal recognition [15]. Personal recognition systems can function in either identification mode or verification mode depending on the application context. In the personal identification mode, the system looks up people in a database to identify them. It might be challenging to recognise a very large size dataset using the proposed identification method based on transfer learning. In order to create a large- scale dataset for additional research on the stability and robustness of the EMG-based recognition system, we will collect more EMG data in our upcoming work. Luzheng Bi a, Aberham Genetu Felekea, Cuntai Guanb are proposed using A review on EMG-based motor intention prediction of continuous human upper limb motion for human- robot collaboration [3]. This paper provides a comprehensive assessment of the motor intention prediction for continuous human upper limb motion using EMG signals. Classification and regression models have been employed most frequently in studies on the prediction of motor intention from an EMG signal. By accurately determining human motor intentions from EMG signals, their research aims to enhance human-robot interaction. By covering a number of methods, strategies, and procedures, the authors aim to provide a detailed overview of the advancements, challenges, and potential future prospects in this sector. This review paper is an essential tool for researchers, engineers, and other experts working on EMG- based motor intention prediction for human-robot collaboration. EMG signals can be a trustworthy technique for intention prediction of human movements in human-robot collaboration systems due to their low attention and motor skill requirements from users and their minimal susceptibility to external disruption. To deal with these issues, a number of strategies, theories, and presumptions have been proposed. Several challenges still need to be adequately addressed in order to improve the efficiency of EMG-based motor intention prediction, build a solid collaborative system, and widen its applicability. Md. Hafizur Rahman,Jinia Afrin are proposed using Hand Gesture Recognition using Multiclass Support Vector Machine [20].This study presents a hand gesture recognition system

Table I: Summary: Overview of Depth Estimation Methods