Skip to main content Skip to page footer

Markerless Motion Tracking and Complex Object Grasping

Object conveyer belt


Jarvis Toolbox

We investigate the diversity and complexity of hand grasping movements in order to understand their underlying control processes in the brain. Our focus is on the development and optimisation of new methods to study hand movements, in particular the variety of movements resulting from interaction with different objects. To this end, we have developed a unique setup that allows the presentation and switching of a variety of objects via a precise conveyor belt system. This setup allows us to analyse a variety of hand and grasping movements caused by contacting objects of different shape and texture.

Motion tracking is of central importance for the investigation of grasping movements. We use a software toolbox developed in our laboratory, JARVIS, which uses Artificial Intelligence (AI) to capture hand movements with a series of cameras, but without markers placed on the hand. The special feature of this JARVIS toolbox allows the precise calculation of hand joint positions even when (partially) occluded (e.g. from a grasped object). This setup, together with the JARVIS toolbox, allows us to analyse hand movements and their generation in the network of neurons in the brain in a more detailed and versatile way.

With this approach, we aim to lay the foundations for improved brain-machine interfaces that enable the control of neuroprostheses through precise and versatile decoding of movement intentions. In the long term, this could contribute to the development of systems that significantly improve the hand functions of paralysed patients and increase their quality of life.