Optimizing abstractive summarization with fine-tuned PEGASUS

Abstract

Abstractive text summarization is the technique of generating a short and concise summary comprising the salient ideas of a source text without making a subset of the salient sentences from the source text. The introduction of transformer models such as BART, T5, and PEGASUS has made this sort of summarization process more efficient and accurate. The objective of this paper is to analyze the performance of different transformer models, compare them to find an efficient model and fine-tune the model on csebuetnlp/xlsum English corpus. The performance of the generated summaries from the fine-tuned PEGASUS models is evaluated using the ROUGE metric, which basically compares the auto-generated summaries with human-created summaries. The fine-tuned PEGASUS model gives a state-of-the-art performance on the XLSum English Corpus.