Localization of AI-Driven Sign Language Recognition

Abstract

Sign language recognition (SLR) is a vital research area promoting inclusivity and interaction for the deaf and hard-of-hearing communities. This thesis focuses on automatic SLR using the American Sign Language (ASL) dataset, emphasizing preprocessing, feature extraction, and Long Short-Term Memory (LSTM) networks to enhance accuracy. The process begins with data collection, where video frames from the ASL dataset are resized, normalized, and converted into grayscale to reduce computational load while retaining key features. Data augmentation techniques like rotation, flipping, and scaling are applied to improve the model’s generalization. Feature extraction captures spatial and temporal information critical for SLR. Optical flow is employed to detect hand motion and facial expressions, while Convolutional Neural Networks (CNNs) extract spatial patterns from the video frames. These features are fed into an LSTM network, designed to learn sequential dependencies in the data. LSTMs are effective for understanding dynamic gestures, as they capture both short- and long-term dependencies between frames. The model predicts sign language symbols or words, facilitating real-time recognition. The thesis further integrates semantic sentence prediction, enabling the system to recognize isolated signs and predict entire sentences. Using Natural Language Processing (NLP), input sentences are mapped to sign language sequences, which are visualized through synthesis models that generate animations. This approach captures handshapes, movements, and expressions essential in ASL. By combining preprocessing, feature extraction, and deep learning, this research improves SLR accuracy and contributes to communication accessibility. It lays a foundation for advancements in SLR systems for applications in education, healthcare, and human-computer interaction.