A long short‐term memory based Schaeffer gesture recognition system

Abstract

In this work, a Schaeffer language recognition system is proposed in order to help autistic children overcome communicative disorders. Using Schaeffer language as a speech and language therapy, improves children communication skills and at the same time the understanding of language productions. Nevertheless, the teaching process of children in performing gestures properly is not straightforward. For this purpose, this system will teach children with autism disorder the correct way to communicate using gestures in combination with speech reproduction. The main purpose is to accelerate the learning process and increase children interest by using a technological approach. Several recurrent neural network‐based approaches have been tested, such as vanilla recurrent neural networks, long short‐term memory networks,and gated recurrent unit‐based models. In order to select the most suitable model, an extensive comparison has been conducted reporting a 93.13% classification success rate over a subset of 25 Schaeffer gestures by using an long short‐term memory‐based approach. Our dataset consists on pose‐based features such as angles and euclidean distances extracted from the raw skeletal data provided by a Kinect v2 sensor.