Clinical note generation from doctor-patient conversations using decoder-only large language models

Abstract

Documenting clinical notes is a vital but time-consuming task in healthcare. Even in this modern era medical doctors spend considerable time documenting clinical notes from encounters with patients. While there have been significant advancements in general text summarization, research in clinical conversation summarization remains sparse due to the scarcity of open-source datasets available to the NLP community. Accurate summarization is paramount in clinical note generation, given its implications for human health. Our research demonstrates the efficacy of decoder-only models over traditional encoder-decoder models in generating more precise clinical notes from doctor-patient conversations. The study also tackles key challenges such as ensuring medical accuracy and complying with healthcare privacy standards. We utilized the MTS-DIALOG dataset [28], including 1, 700 such dialogues and corresponding clinical notes. This dataset was featured in the 2023 MEDIQAChat challenge, where the leading team, WangLab achieved a state-ofthe- art (SOTA) Rouge-1 score of 0.4466 and BERTScore of 0.7307 [27]. Our study surpasses these benchmarks by fine-tuning the ”metallama/Meta-Llama-3-8B” model enhanced with Qlora 8-bit quantization. We assessed our models using Rouge scores and BERT Scores to validate their superiority in performance. By evaluating the system on real-world clinical conversations, we show that the decoder-only LLM-generated notes closely match human-written ones in terms of completeness and clinical relevance. This research highlights the potential for decoder-only LLMs to revolutionize clinical workflows, making medical documentation more efficient while allowing doctors to focus more on patient care.