Sinhala-English Multilingual AI Call Center Bot with Sentiment-Aware Dialogue and Multimodal CSAT Prediction

Abstract

This paper presents a Sri Lanka focused, voice-based call center automation framework supporting Sinhala, English, and Sinhala-English code-mixed conversations in low-resource environments. The system adopts an end-to-end architecture integrating bilingual dataset construction, privacy-preserving speech processing, retrieval-grounded response generation, multimodal sentiment intelligence, and customer satisfaction (CSAT) estimation. A Sinhala-English call-center corpus is created using noise reduction, speaker diarization, and transcript alignment, combined with transcript-aligned PII redaction. During live interaction, language identification routes calls to unified processing pipelines. Real-time sentiment analysis with explainable risk scoring supports escalation decisions, while retrieval-augmented generation ensures factually grounded responses. Emotion-adaptive text-to-speech enhances conversational naturalness. The framework enables interaction-based CSAT estimation without relying solely on post-call surveys, providing scalable and privacy-aware automation tailored to multilingual Sri Lankan call center operations