Sadeque, Farig YousufHossain, AriyanRahman, ShehranKhairul, SameerRoy, ProtayaNasir, ShamsanAkif, Ahnaf2026-01-142026-01-1420252025-10ID 24241196ID 22101938ID 24241160ID 22101481ID 23241138http://hdl.handle.net/10361/27438Cataloged from PDF version of thesis.Includes bibliographical references (pages 81-83).This thesis is submitted in partial fulfillment of the requirements for the degree of Bachelor of Science in Computer Science and Engineering, 2025.Large Language Models (LLMs) are incredible and sophisticated models that have remarkable capabilities that towers over a wide range of language processing tasks. However, that capability comes with a substantial computational and memory cost which is often too great. This is why it is harder for the general population to access this technology. This paper explores other alternative architectures such as State- Space models in order to understand which models are lightweight but also retain strong performance. The aim is to compare the SSM based Llamba model family performances on mathematical reasoning tasks which remains unexplored. A deep investigation into the accuracy of these models is performed along with other computational metrics such as decode speed, memory usage etc. An exploration has been made on how different architectural choices impact both efficiency and effectiveness of these models. The results provide a clear picture of the existing landscape that Transformers are the obvious architectural choice for the best reasoning accuracy while SSMs can be the viable and efficient alternative in the situations when the computational resources are the main constraint.92 pagesenBRAC University theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission.Large language modelsKnowledge distillationState-space modelsMathematical reasoningReasoning accuracyTransformersNeural networksArtificial intelligenceState-space methods.Neural networks (Computer science).Computer network architectures.Thesis