Exploring the limitations of AI in legal reasoning: challenges, constraints, and implications for Bangladeshi law

Abstract

Artificial Intelligence specially Large Language Models, are significantly promising for automating legal tasks as they interpret texts really well. However, in the context of the Bangladeshi Legal system it is constrained by fundamental limitations in bias, contextual misalignment and accountability gap. The research is an empirical analysis on the possibilities of these LLMs in making statutory decisions, even with the presence of misinterpretations of these LLMs in Bangladeshi context. It is based on a primary dataset on Bangladeshi legal criminal cases, that compares multiple architectures for predicting legal sections and generating analogies and finding out the need for rigorous validation, oversight, and domain specific adaptation to increase AI’s applicability in Bangladeshi legal systems. Our findigs show that comparing multiple variations of sequence, transformer and state space architectures, it was concluded that metrics like F1 score, recall in section prediction and Cosine-similarity, BERTScore, Jaccard Similarity in analogy generation prevailed. This research proposes that the usage of these models in a proper section prediction and analogy generation tasks is possible only if the limitatioms of such LLMs are highly taken into consideration. Creating an equilibrium between the results of such models and the limitations is essential to ensure that AI-driven legal reasoning remains fair, accurate, and accountable.