EasyTalk: A Translator for Sri Lankan Sign Language using Machine Learning and Artificial Intelligence

Abstract

Sign language is used by the hearing-impaired and inarticulate community to communicate with each other. But not all Sri Lankans are aware of the sign language or verbal languages and a translation is required. The Sri Lankan Sign Language is tightly bound to the hearing-impaired and inarticulate. The paper presents EasyTalk, a sign language translator which can translate Sri Lankan Sign Language into text and audio formats as well as translate verbal language into Sri Lankan Sign Language which would benefit them to express their ideas. This is handled in four separate components. The first component, Hand Gesture Detector captures hand signs using pre-trained models. Image Classifier component classifies and translates the detected hand signs. The Text and Voice Generator component produces a text or an audio formatted output for identified hand signs. Finally, Text to Sign Converter works on converting an entered English text back into the sign language based animated images. By using these techniques, EasyTalk can detect, translate and produce relevant outputs with superior accuracy. This can result in effective and efficient communication between the community with differently-abled people and the community with normal people.