Figure 1: Illustration of the probabilistic framework (graphical model) used to model the dynamic integration of muscle-activity and hand-joint kinematics. The muscular activity is combined with knowledge of the velocity of each joint to predict the velocity of the joint at the next time instant.

Figure 2: Details of the monitored muscles in the anterior and posterior compartment of the forearm and sensor placement. We combine the electrical (EMG) and mechanical (MMG) effects of muscular contraction to extract more information about the subject's movement intention. Hand movements were recorder using a CyberGlove.

The ability to dextrously control our hand is key to our evolution and at the centre of almost all skilled object interaction in daily life. Currently available commercial prosthetic hands are typically constrained by limited functionality in terms of degrees of freedom, range of motion, sensory feedback and intuitive control. Despite the mechanical complexity of the human hand, the bottleneck lies in the human-machine interface required for robustly translating the user’s intention into a suitable robotic action: most commercially available devices use the electrical signal generated by the residual muscles in the wearer's limb to initiate 3-4 pre-programmed classes of grasping patterns. This greatly limits the complexity of the tasks that the subject can complete and makes prosthetics cumbersome and not very effective in daily life. 

If only the activity of the residual muscles could be used to generate a continuous and reliable movement of prosthetic hand at the level of each individual joint, the tasks that the subject would be able to complete and the perceived effectiveness of the robotic limb would greatly increase.

To this end, we combined the electrical (EMG) and mechanical (MMG) signals generated by contraction of 5  muscles in the forearm  to train two models (Autoregressive Gaussian Process and VARMAX) that continuously predict the instantaneous velocity of 11 joints of the hand from the muscles' activity. 

Publications

Michele Xiloyannis, Constantinos Gavriel, Andreas A. C. Thomik and A. Aldo Faisal, "Gaussian Process Autoregression for Simultaneous Proportional Multi-Modal Prosthetic Control with Natural Hand Kinematics", Transaction on Neural Systems & Rehabilitation Engineering (TNSRE), 2017.

Michele Xiloyannis, Andreas A. C. Thomik, Constantinos Gavriel and A. Aldo Faisal, "Dynamic forward prediction for prosthetic hand control by integration of EMG, MMG and kinematic signals",  7th International IEEE EMBS Neural Engineering Conference, Montpellier, France, 2015.

Michele Xiloyannis, Constantinos Gavriel, Andreas A. C. Thomik and A. Aldo Faisal, "Gaussian Process Regression for accurate prediction of prosthetic limb movements from the natural kinematics of intact limbs", 7th International IEEE EMBS Neural Engineering Conference, Montpellier, France, 2015.