Empowering Communication: A Web Application for Deaf, Mute, and Sign Language Interpretation

Abstract

The inability to communicate effectively because of communication barriers severely restricts deaf and mute individuals from getting in touch with those who do not know sign language. A new web-based system has been developed to provide an effortless communication solution between users who are deaf, mute and Sign language users. The system implements Django as its backend framework collected with text- to- Sign language conversion and speech-to-text capabilities towards establish successful communication. Through the application users have the option to provide text or audio content that gets translated into American Sign language (ASL) animations through hand gesture visualization from MediaPipe. The system enables automatic speech recognition through the Speech Recognition library and functions as an ASR tool to convert spoken words into text. The Google Translator API enables native language translation of this recorded text which expands communication possibilities to different user groups. From datasets obtained through Kaggle the training of a Convolutional Neural Network model reaches 99% accuracy for sign recognition leading to accurate communication. The application creates a manageable interface which enables people with speech and hearing difficulties to get real-time Sign language animations as visual outputs. The novel system surpasses typical assistive communication solutions because it cuts out the requirement for human Sign language interpreters to provide inclusive communication channels. This proposed web application will increase accessible interactions thus promoting more social integration opportunities for both deaf and mute users. The system will be improved through language expansion of Sign language interpretation and a mobile app adaptation for maximized accessibility. The system incorporates components related to Communication accessibility, Sign language recognition, speech-to-text, ASL animation and employs CNN models in conjunction with Django framework.