Gamified Deep Learning-Based Strategies for Supporting Speech and Motor Skill Development in Children with Autism

Abstract

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that affects day-to-day behavior. ASD presents challenges in developing speech, language, and motor skills, which require early intervention. This study proposes deep learning-based gamified techniques to support children with ASD through interactive learning experiences. The system consists of two modules, speech and language assessment and motor skills assessment, addressing both gross and fine motor development. Speech and language capabilities are assessed through analysis of lip patterns and phoneme alignment, using a hybrid model that combines 3D Convolutional Neural Networks (3D-CNN) with Bidirectional Gated Recurrent Units (Bi-GRU), achieving an accuracy of 95%. Fine motor skills are evaluated through the identification of color patterns using a You Only Look Once (YOLO) model combined with geometric methods, achieving a precision of 98%. Gross motor skills are assessed through gesture recognition using Long Short Term Memory (LSTM) networks and angle-based heuristic techniques, achieving 90% accuracy.