AI-Driven Code Comment Quality Assessment and Its Impact on Software Complexity

Abstract

Code comments play a critical role in software readability and maintainability. However, poorly written, redundant, or misleading comments can increase software complexity and hinder developer productivity. This research proposes a novel AI-driven framework for assessing code comment quality using a Comment Quality Index (CQI), which combines both structural and semantic features. Structural scoring was implemented through heuristic methods, while semantic scoring leveraged transformer-based models, specifically BERT, to capture the meaning and relevance of comments. To validate the approach, heuristic and AI-based scores were compared against developer-rated comments collected via a survey. Additionally, the study introduces CQI value bands to classify comments as poor, average, or good, providing actionable insights for developers. Statistical analyses, including ANOVA and correlation tests, confirm the effectiveness and reliability of the proposed scoring framework. For the AI component, the BERT model was fine-tuned on 90 developer-rated comments, showing consistent training loss reduction across epochs. The fine-tuned model was saved and applied for inference on unseen comments, demonstrating its ability to generalize and provide real-time quality assessment. The results indicate a strong alignment between AI-predicted scores and human evaluations, highlighting the potential of AI-assisted comment analysis to enhance software quality. Finally, the research explores the integration of this framework into an IDE plugin, enabling developers to receive immediate feedback on comment quality during code development. Overall, the study provides a comprehensive methodology for automated comment quality evaluation, combining empirical validation, AI-based semantic analysis, and practical implementation in modern software engineering environments.