AI Interviews with Facial Emotion Recognition for Real-Time Feedback and Career Recommendations

Abstract

The hiring process is complex, requiring evaluation of candidates across multiple dimensions, including technical proficiency, behavioral traits, and credibility. Traditional interviews often suffer from biases and inefficiencies. This research presents an AI-driven Interview System integrating Machine Learning (ML), Natural Language Processing (NLP), and Computer Vision to automate and enhance recruitment. The system generates contextual interview questions, evaluates candidate responses using LLM-based scoring models, and provides real-time feedback for engagement. It includes speech-to-text transcription and offensive word detection to ensure professionalism. The behavioral analysis module leverages facial emotion recognition and computer vision to assess non-verbal cues such as confidence and attentiveness. Additionally, Curriculum Vitae (CV) parsing and LinkedIn data extraction use NLP-based entity recognition to extract educational background, work experience, and key skills, enabling personalized interviews. The technical assessment module administers real-time coding challenges, evaluating solutions for correctness, efficiency, and best practices while providing AI-generated feedback. By automating these key hiring aspects, this system enhances objectivity, efficiency, and decision-making, ensuring a data-driven, unbiased, and scalable selection process while improving the candidate's experience and employer insights