Accurate and efficient 3D hand pose regression for robot hand teleoperation using a monocular RGB camera

Please use this identifier to cite or link to this item:
Información del item - Informació de l'item - Item information
Title: Accurate and efficient 3D hand pose regression for robot hand teleoperation using a monocular RGB camera
Authors: Gomez-Donoso, Francisco | Orts-Escolano, Sergio | Cazorla, Miguel
Research Group/s: Robótica y Visión Tridimensional (RoViT)
Center, Department or Service: Universidad de Alicante. Departamento de Ciencia de la Computación e Inteligencia Artificial | Universidad de Alicante. Instituto Universitario de Investigación Informática
Keywords: Hand pose estimation | Deep learning | Robot teleoperation | Monocular
Knowledge Area: Ciencia de la Computación e Inteligencia Artificial
Issue Date: 1-Dec-2019
Publisher: Elsevier
Citation: Expert Systems with Applications. 2019, 136: 327-337. doi:10.1016/j.eswa.2019.06.055
Abstract: In this paper, we present a novel deep learning-based architecture, which is under the scope of expert and intelligent systems, to perform accurate real-time tridimensional hand pose estimation using a single RGB frame as an input, so there is no need to use multiple cameras or points of view, or RGB-D devices. The proposed pipeline is composed of two convolutional neural network architectures. The first one is in charge of detecting the hand in the image. The second one is able to accurately infer the tridimensional position of the joints retrieving, thus, the full hand pose. To do this, we captured our own large-scale dataset composed of images of hands and the corresponding 3D joints annotations. The proposal achieved a 3D hand pose mean error of below 5 mm on both the proposed dataset and Stereo Hand Pose Tracking Benchmark, which is a public dataset. Our method also outperforms the state-of-the-art methods. We also demonstrate in this paper the application of the proposal to perform a robotic hand teleoperation with high success.
Sponsor: This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243
ISSN: 0957-4174 (Print) | 1873-6793 (Online)
DOI: 10.1016/j.eswa.2019.06.055
Language: eng
Type: info:eu-repo/semantics/article
Rights: © 2019 Elsevier Ltd.
Peer Review: si
Publisher version:
Appears in Collections:INV - RoViT - Artículos de Revistas

Files in This Item:
Files in This Item:
File Description SizeFormat 
Thumbnail2019_Gomez-Donoso_etal_ExpertSystWithAppl_final.pdfVersión final (acceso restringido)3,35 MBAdobe PDFOpen    Request a copy
Thumbnail2019_Gomez-Donoso_etal_ExpertSystWithAppl_accepted.pdfAccepted Manuscript (acceso abierto)10,07 MBAdobe PDFOpen Preview

Items in RUA are protected by copyright, with all rights reserved, unless otherwise indicated.