Browsing by Author "Rathnaweera, C"

Filter results by typing the first few letters
Now showing 1 - 1 of 1
  • Thumbnail Image
    ItemEmbargo
    Adaptive Voice Communication in Emotion-Aware Digital Companions
    (Institute of Electrical and Electronics Engineers Inc., 2025) Rathnayake, P; Rathnaweera, C; Jithma, U; Aththanayake, I; Rathnayake, S; Gunaratne, M
    This paper presents an adaptive voice communication system for emotion-aware digital companions that dynamically responds to users' affective states through expressive speech and synchronized 3D avatar animation. The system integrates real-time voice input, emotion recognition, and context-aware dialogue generation using GPT-3.5, followed by emotional text-to-speech synthesis via neural TTS. Lip-sync data is generated using phoneme alignment and rendered in sync with the avatar's facial expressions and gestures. To enhance user trust and engagement, the avatar visually mirrors the emotional tone of the speech. A cultural adaptation layer is introduced to align voice output and speech style with Sri Lankan communication norms, including tone, pacing, and formality. Implemented using a Node.js backend and React + Three.js frontend, the system demonstrates strong potential for emotionally intelligent, culturally adaptive AI interactions. This work contributes a modular pipeline for building empathetic voice agents capable of enhancing realism and trust in human-AI communication.