Text summarization based on transformer using sentences ranking and time penalty

Abstract

Text summarization systems often struggle with selecting salient content, avoiding repetition, and handling out-of-vocabulary entities. We address these issues with a two-stage approach: a supervised sentence-ranking head (SRM-head) first selects the top- $N$ sentences, and a Transformer generator then produces the summary. The generator is augmented with a time penalty in encoder–decoder attention to discourage reattending to recently focused source positions, and with a pointer mechanism that copies salient spans, thereby improving entity and number fidelity. Experiments on CNN/DailyMail and WikiHow, plus an additional evaluation on XSum, show that our model attains competitive ROUGE scores against recent pretrained systems while using lightweight, modular components.