Computer Vision Based Recognition of the Corresponding Sinhala Word from Continuous Finger Spelled Signs

Abstract

Sign language is the main communication method of deaf people. There are many sign languages used by the deaf people around the world to communicate. Sri Lankan Sign Language is the sign language used by the deaf community in Sri Lanka. According to the information collected through Census of Population and Housing-2012 report, the deaf population in Sri Lanka is 389,077 by 2012[26]. The ratio is 21 per 1000 people [26]. Also, according to the Sri Lanka Central Federation of the Deaf there are only 12 recognized sign language interpreters in Sri Lanka. One of the major problems facing the deaf community in Sri Lanka is the difficulty in communicating directly with the general public as very few ordinary people can understand the sign language. Therefore, deaf people should get an assistance of a trained sign language interpreter to communicate effectively with normal hearing people. It is expensive to hire trained sign language interpreters and there are few such interpreters in Sri Lanka. If there is a method that the deaf people can communicate directly with the general public without the help of a sign language interpreter then they can overcome many difficulties that they are facing in various fields, particularly in education. One of the solutions for this problem is automating the recognition of sign language. Although there are some research works available in Sri Lanka on recognition of the static signs of Sri Lankan sign language, there are no existing works to recognize the corresponding word from continuously signed fingerspelling sequence. Also, there are some research works have been published on constructing datasets for various sign languages around the world, but there is no signing dataset in video format for Sinhala Sign Alphabet. The purpose of this study is to design and implement a system to recognize the corresponding Sinhala word and numbers expressed using a continuous sequence of Sinhala sign alphabets. Words and numbers will be identified from a video clip. The other main outcome is to construct a signing dataset for Sinhala Sign alphabet in video format. Findings and the outcomes of this research will be beneficial for the deaf and hearing impaired community in Sri Lanka. Also, the end results of this research would be useful to those who wish to develop various systems to improve communication between deaf people and normal people. Furthermore, the results of this research would help to improve the interaction between deaf people and computer systems.