bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization

Runs On

What it takes to serve bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization (23M parameters): the memory its weights need at each precision, and the cheapest way to rent enough data-center GPUs to hold them.

Memory is the weights at that precision plus 20% for the runtime and a short context; a long context needs more. Prices are the lowest on-demand hourly rates in the SAVRN Index, read Sep 18, 2026.

Model Card

By Manuel Romero, published under apache-2.0, revision d0efeded219c.

This model is a warm-started BERT2BERT (mini) model fine-tuned on the CNN/Dailymail summarization dataset. The model achieves a 16.51 ROUGE-2 score on CNN/Dailymail's test dataset. For more details on how the model was fine-tuned, please refer to this notebook.

from transformers import BertTokenizerFast, EncoderDecoderModel
import torch
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
tokenizer = BertTokenizerFast.from_pretrained('mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization')
model = EncoderDecoderModel.from_pretrained('mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization').to(device)

def generate_summary(text):
    # cut off at BERT max length 512
    inputs = tokenizer([text], padding="max_length", truncation=True, max_length=512, return_tensors="pt")
    input_ids = inputs.input_ids.to(device)
    attention_mask = inputs.attention_mask.to(device)

    output = model.generate(input_ids, attention_mask=attention_mask)

    return tokenizer.decode(output[0], skip_special_tokens=True)

text = "your text to be summarized here..."
generate_summary(text)

Configuration

Architecture

EncoderDecoderModel

Vocabulary size

30,522

Model type

encoder-decoder

Identity and Version

Repository

mrm8488/bert-mini2bert-mini-finetuned-cnn_daily_mail-summarization

Publisher

Manuel Romero

Task

Summarization

Modality

Text

Library

transformers

Parameters

23M parameters

Languages

en

Revision

d0efeded219c4d14a740e222de8181453da764a6

First published

2022-03-02

Last updated

2023-05-05

Files and Weights

9 files, 187.8 MB in total. The weights are 3 files totalling 187.5 MB in bin, safetensors.

License and Download

License

apache-2.0

Access

Open weights, no gate

Download size

187.5 MB

Built From

• Trained on (disclosed) cnn_dailymail

Memory Requirements

Weights only, from the published parameter count; the key-value cache and runtime add to this.

Similar Models

Model · Summarization

distilbart-cnn-12-6

Sam Shleifer

This checkpoint should be loaded into BartForConditionalGeneration.frompretrained. See the BART docs for more information.

Open weights apache-2.0 1,024 tokens transformers

View model →

Model · Summarization

pegasus-xsum

Google

Original TF 1 code here Authors: Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu on Dec 18, 2019 The following is copied from the authors' README. We train a pegasus model with sampled gap sentence ratios on both C4 and HugeNews, and stochastically sample important sentences. The updated the results are reported in this table. The "Mixed & Stochastic" model has the following changes: - trained on both C4 and HugeNews (dataset mixture is weighted by their number of examples). - trained for 1.5M instead of 500k (we observe slower convergence on pretraining perplexity). - the model uniformly sample a gap sentence ratio between 15% and 45%. - importance sentences are sampled using a…

Open weights 512 tokens transformers

View model →

Model · Summarization

bart-large-cnn-samsum

Philipp Schmid

This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.

Open weights mit 1,024 tokens transformers

View model →

Model · Summarization

distilbart-xsum-12-6

Sam Shleifer

This checkpoint should be loaded into BartForConditionalGeneration.frompretrained. See the BART docs for more information.

Open weights apache-2.0 1,024 tokens transformers

View model →

Model · Summarization

t5-small-booksum

Cosmina Nicu

Open weights mit transformers

View model →

Model · Summarization

mT5_multilingual_XLSum

BUET CSE NLP Group

This repository contains the mT5 checkpoint finetuned on the 45 languages of XL-Sum dataset. For finetuning details and scripts, see the paper and the official repository. Scores on the XL-Sum test sets are as follows: Language | ROUGE-1 / ROUGE-2 / ROUGE-L Amharic | 20.0485 / 7.4111 / 18.0753 Arabic | 34.9107 / 14.7937 / 29.1623 Azerbaijani | 21.4227 / 9.5214 / 19.3331 Bengali | 29.5653 / 12.1095 / 25.1315 Burmese | 15.9626 / 5.1477 / 14.1819 Chinese (Simplified) | 39.4071 / 17.7913 / 33.406 Chinese (Traditional) | 37.1866 / 17.1432 / 31.6184 English | 37.601 / 15.1536 / 29.8817 French | 35.3398 / 16.1739 / 28.2041 Gujarati | 21.9619 / 7.7417 / 19.86 Hausa | 39.4375 / 17.6786 / 31.6667…

Open weights transformers

View model →