This checkpoint should be loaded into BartForConditionalGeneration.frompretrained. See the BART docs for more information.
Open-weight model · Summarization
bart-large-cnn-samsum
by Philipp Schmid philschmid/bart-large-cnn-samsum
This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
Parameters—
Context1,024
Weights6.5 GB
Licensemit
AccessOpen weights
Monthly Downloads105.9k
Model Card
By Philipp Schmid, published under mit, revision e49b3d60d923.
This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
Read Philipp Schmid's full model card
If you want to use the model you should try a newer fine-tuned FLAN-T5 version philschmid/flan-t5-base-samsum out socring the BART version with
+6onROGUE1achieving47.24.
TRY philschmid/flan-t5-base-samsum
This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
For more information look at: - Transformers Documentation: Amazon SageMaker - Example Notebooks - Amazon SageMaker documentation for Hugging Face - Python SDK SageMaker documentation for Hugging Face - Deep Learning Container
Hyperparameters
{
"dataset_name": "samsum",
"do_eval": true,
"do_predict": true,
"do_train": true,
"fp16": true,
"learning_rate": 5e-05,
"model_name_or_path": "facebook/bart-large-cnn",
"num_train_epochs": 3,
"output_dir": "/opt/ml/model",
"per_device_eval_batch_size": 4,
"per_device_train_batch_size": 4,
"predict_with_generate": true,
"seed": 7
}
Usage
from transformers import pipeline
summarizer = pipeline("summarization", model="philschmid/bart-large-cnn-samsum")
conversation = '''Jeff: Can I train a Transformers model on Amazon SageMaker?
Philipp: Sure you can use the new Hugging Face Deep Learning Container.
Jeff: ok.
Jeff: and how can I get started?
Jeff: where can I find documentation?
Philipp: ok, ok you can find everything here. https://huggingface.co/blog/the-partnership-amazon-sagemaker-and-hugging-face
'''
summarizer(conversation)
Results
| key | value |
|---|---|
| eval_rouge1 | 42.621 |
| eval_rouge2 | 21.9825 |
| eval_rougeL | 33.034 |
| eval_rougeLsum | 39.6783 |
| test_rouge1 | 41.3174 |
| test_rouge2 | 20.8716 |
| test_rougeL | 32.1337 |
| test_rougeLsum | 38.4149 |
Configuration
- Architecture
- BartForConditionalGeneration
- Context length (tokens)
- 1,024
- Layers
- 12
- Vocabulary size
- 50,264
- Model type
- bart
Identity and Version
- Repository
- philschmid/bart-large-cnn-samsum
- Publisher
- Philipp Schmid
- Task
- Summarization
- Modality
- Text
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- en
- Revision
- e49b3d60d923f12db22bdd363356f1a4c68532ad
- First published
- 2022-03-02
- Last updated
- 2022-12-23
Files and Weights
25 files, 6.5 GB in total. The weights are 6 files totalling 6.5 GB in bin, pt.
Weights6 files · 6.5 GB
Configuration10 files · 7.5 KB
Tokenizer6 files · 2.5 MB
Documentation1 file · 5.7 KB
Other1 file · 184.6 KB
Repository1 file · 690 B
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| checkpoint-500/optimizer.pt | Weights | 3.3 GB | d163c581e7f6 |
| checkpoint-500/pytorch_model.bin | Weights | 1.6 GB | ccab95146195 |
| checkpoint-500/scheduler.pt | Weights | 559 B | cf91f1d26632 |
| checkpoint-500/training_args.bin | Weights | 2.4 KB | 089267ebad1d |
| pytorch_model.bin | Weights | 1.6 GB | 9f453aa6edef |
| training_args.bin | Weights | 2.4 KB | 089267ebad1d |
| all_results.json | Configuration | 1.3 KB | — |
| checkpoint-500/config.json | Configuration | 1.6 KB | — |
| checkpoint-500/special_tokens_map.json | Configuration | 239 B | — |
| checkpoint-500/trainer_state.json | Configuration | 486 B | — |
| config.json | Configuration | 1.6 KB | — |
| eval_results.json | Configuration | 454 B | — |
| special_tokens_map.json | Configuration | 239 B | — |
| test_results.json | Configuration | 438 B | — |
| train_results.json | Configuration | 457 B | — |
| trainer_state.json | Configuration | 641 B | — |
| README.md | Documentation | 5.7 KB | — |
| test_generations.txt | Other | 184.6 KB | — |
| .gitattributes | Repository | 690 B | — |
| checkpoint-500/merges.txt | Tokenizer | 456.4 KB | — |
| checkpoint-500/tokenizer_config.json | Tokenizer | 300 B | — |
| checkpoint-500/vocab.json | Tokenizer | 798.3 KB | — |
| merges.txt | Tokenizer | 456.4 KB | — |
| tokenizer_config.json | Tokenizer | 300 B | — |
| vocab.json | Tokenizer | 798.3 KB | — |
License and Download
- License
- mit
- Access
- Open weights, no gate
- Download size
- 6.5 GB
Download from Philipp Schmid
Released by Philipp Schmid through its official repository on Hugging Face. Read the license.
Built From
- Trained on (disclosed) samsum
Evaluations
Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization | Task SummarizationMetric Test ROGUE-1Comparison conditions not established | 41.3174 | philschmid Publisher reported |
Evaluated revision not stated | — |
| SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization | Task SummarizationMetric Test ROGUE-2Comparison conditions not established | 20.8716 | philschmid Publisher reported |
Evaluated revision not stated | — |
| SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization | Task SummarizationMetric Test ROGUE-LComparison conditions not established | 32.1337 | philschmid Publisher reported |
Evaluated revision not stated | — |
| SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization | Task SummarizationMetric Validation ROGUE-1Comparison conditions not established | 42.621 | philschmid Publisher reported |
Evaluated revision not stated | — |
| SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization | Task SummarizationMetric Validation ROGUE-2Comparison conditions not established | 21.9825 | philschmid Publisher reported |
Evaluated revision not stated | — |
| SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization | Task SummarizationMetric Validation ROGUE-LComparison conditions not established | 33.034 | philschmid Publisher reported |
Evaluated revision not stated | — |
| samsum | Configuration samsumTask SummarizationMetric ROUGE-1Comparison conditions not established | 41.3282 | philschmid Publisher reported |
Evaluated revision not stated | — |
| samsum | Configuration samsumTask SummarizationMetric ROUGE-2Comparison conditions not established | 20.8755 | philschmid Publisher reported |
Evaluated revision not stated | — |
| samsum | Configuration samsumTask SummarizationMetric ROUGE-LComparison conditions not established | 32.1353 | philschmid Publisher reported |
Evaluated revision not stated | — |
| samsum | Configuration samsumTask SummarizationMetric ROUGE-LSUMComparison conditions not established | 38.401 | philschmid Publisher reported |
Evaluated revision not stated | — |
| samsum | Configuration samsumTask SummarizationMetric gen_lenComparison conditions not established | 60.0757 | philschmid Publisher reported |
Evaluated revision not stated | — |
| samsum | Configuration samsumTask SummarizationMetric lossComparison conditions not established | 1.42972 | philschmid Publisher reported |
Evaluated revision not stated | — |
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 6.5 GB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About bart-large-cnn-samsum
Can I use bart-large-cnn-samsum commercially?
Yes. bart-large-cnn-samsum is released under MIT License. The MIT License is a short permissive license. It permits commercial use, modification and redistribution, provided the copyright notice and permission notice are included.
What is bart-large-cnn-samsum's context length?
1,024 tokens, from the maximum position embeddings in its published configuration.
Similar Models
Original TF 1 code here Authors: Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu on Dec 18, 2019 The following is copied from the authors' README. We train a pegasus model with sampled gap sentence ratios on both C4 and HugeNews, and stochastically sample important sentences. The updated the results are reported in this table. The "Mixed & Stochastic" model has the following changes: - trained on both C4 and HugeNews (dataset mixture is weighted by their number of examples). - trained for 1.5M instead of 500k (we observe slower convergence on pretraining perplexity). - the model uniformly sample a gap sentence ratio between 15% and 45%. - importance sentences are sampled using a…
This checkpoint should be loaded into BartForConditionalGeneration.frompretrained. See the BART docs for more information.
This repository contains the mT5 checkpoint finetuned on the 45 languages of XL-Sum dataset. For finetuning details and scripts, see the paper and the official repository. Scores on the XL-Sum test sets are as follows: Language | ROUGE-1 / ROUGE-2 / ROUGE-L Amharic | 20.0485 / 7.4111 / 18.0753 Arabic | 34.9107 / 14.7937 / 29.1623 Azerbaijani | 21.4227 / 9.5214 / 19.3331 Bengali | 29.5653 / 12.1095 / 25.1315 Burmese | 15.9626 / 5.1477 / 14.1819 Chinese (Simplified) | 39.4071 / 17.7913 / 33.406 Chinese (Traditional) | 37.1866 / 17.1432 / 31.6184 English | 37.601 / 15.1536 / 29.8817 French | 35.3398 / 16.1739 / 28.2041 Gujarati | 21.9619 / 7.7417 / 19.86 Hausa | 39.4375 / 17.6786 / 31.6667…
This checkpoint should be loaded into BartForConditionalGeneration.frompretrained. See the BART docs for more information.