AI for Legal Domain Identification and Guidance in Sri Lankan Civil Law: A Comparative Study of Open-Source vs Proprietary AI Models

Abstract

This study investigated the application of retrieval-augmented generation (RAG) architectures powered by large language models (LLMs) to improve access to civil-law information in Sri Lanka. It addresses a key challenge in the country’s justice system, the limited accessibility to affordable and reliable legal guidance. A RAG-based legal information assistant was designed, implemented, and evaluated using two back-end models: OpenAI’s GPT-3.5-Turbo and the open-source Mistral-7B-v0.1. Both systems were integrated into a curated Sri Lankan civil-law corpus and compared across three metrics: accuracy, latency, and cost using a set of test queries. GPT-3.5 Turbo achieved higher accuracy (92.5%) and lower average latency (4.17s) at a lower cost (USD 0.000487 per query) than Mistral-7B-v0.1 (82.5% accuracy, 15.64s average latency, USD 0.000742 average cost). Statistical tests confirmed significant differences in latency and cost. GPT-3.5-Turbo therefore exhibited superior responsiveness and efficiency for real-time, citizen-facing legal assistance, whereas Mistral-7B offers a competitive, viable, privacy-preserving alternative for institutional or offline use. The research contributes a reproducible evaluation framework for legal-domain LLMs and a localized civil-law corpus designed for retrieval-augmented systems. More broadly, it demonstrates that responsibly designed AI can enhance access to justice in low-resource contexts. The findings establish a foundation for future multilingual, ethically aligned, and jurisdiction-aware legal-AI systems in Sri Lanka.