This checkpoint should be loaded into BartForConditionalGeneration.frompretrained. See the BART docs for more information.
Open-weight model · Summarization
t5-small-finetuned-cnn-v2
by Luis Fernando Costa ubikpt/t5-small-finetuned-cnn-v2
This model is a fine-tuned version of t5-small on the cnndailymail dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5.6e-05 - trainbatchsize: 8 - evalbatchsize: 8 …
Parameters—
Context—
Weights242.1 MB
Licenseapache-2.0
AccessOpen weights
Monthly Downloads4.3k
Model Card
By Luis Fernando Costa, published under apache-2.0, revision a0be88a1dd4c.
This model is a fine-tuned version of t5-small on the cnndailymail dataset. It achieves the following results on the evaluation set: The following hyperparameters were used during training: - learningrate: 5.6e-05 - trainbatchsize: 8 - evalbatchsize: 8 - lrschedulertype: linear - numepochs: 8 - Transformers 4.14.0 - Pytorch 1.5.0 - Datasets 2.3.2 - Tokenizers 0.10.3
Read Luis Fernando Costa's full model card
This model is a fine-tuned version of t5-small on the cnn_dailymail dataset. It achieves the following results on the evaluation set: - Loss: 1.5474 - Rouge1: 35.154 - Rouge2: 18.683 - Rougel: 30.8481 - Rougelsum: 32.9638
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training: - learning_rate: 5.6e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 8
Training results
| Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum |
|---|---|---|---|---|---|---|---|
| 1.8823 | 1.0 | 35890 | 1.5878 | 34.9676 | 18.4927 | 30.6753 | 32.7702 |
| 1.7871 | 2.0 | 71780 | 1.5709 | 34.9205 | 18.5556 | 30.6514 | 32.745 |
| 1.7507 | 3.0 | 107670 | 1.5586 | 34.9825 | 18.4964 | 30.6724 | 32.7644 |
| 1.7253 | 4.0 | 143560 | 1.5584 | 35.074 | 18.6171 | 30.8007 | 32.9132 |
| 1.705 | 5.0 | 179450 | 1.5528 | 35.023 | 18.5787 | 30.7014 | 32.8396 |
| 1.6894 | 6.0 | 215340 | 1.5518 | 35.0583 | 18.6754 | 30.791 | 32.8814 |
| 1.6776 | 7.0 | 251230 | 1.5468 | 35.2236 | 18.6812 | 30.8944 | 33.0362 |
| 1.6687 | 8.0 | 287120 | 1.5474 | 35.154 | 18.683 | 30.8481 | 32.9638 |
Framework versions
- Transformers 4.14.0
- Pytorch 1.5.0
- Datasets 2.3.2
- Tokenizers 0.10.3
Configuration
- Architecture
- T5ForConditionalGeneration
- Vocabulary size
- 32,128
- Stored precision
- float32
- Model type
- t5
Identity and Version
- Repository
- ubikpt/t5-small-finetuned-cnn-v2
- Publisher
- Luis Fernando Costa
- Task
- Summarization
- Modality
- Text
- Library
- transformers
- Parameters
- Not stated by the source
- Languages
- Not stated by the source
- Revision
- a0be88a1dd4cb4e76ae49337445f0c08e8a51493
- First published
- 2022-06-30
- Last updated
- 2022-07-01
Files and Weights
13 files, 244.3 MB in total. The weights are 2 files totalling 242.1 MB in bin.
Weights2 files · 242.1 MB
Configuration2 files · 3.2 KB
Tokenizer3 files · 2.2 MB
Documentation1 file · 2.4 KB
Other3 files · 14.9 KB
Repository2 files · 1.2 KB
Every file
| File | Type | Size | SHA-256 |
|---|---|---|---|
| pytorch_model.bin | Weights | 242.1 MB | 29602b925f2d |
| training_args.bin | Weights | 2.8 KB | ee2bb0ab609e |
| config.json | Configuration | 1.4 KB | — |
| special_tokens_map.json | Configuration | 1.8 KB | — |
| README.md | Documentation | 2.4 KB | — |
| runs/Jun30_19-12-05_tukuyomi.tstcl.local/1656583936.3207455/events.out.tfevents.1656583936.tukuyomi.tstcl.local.3869584.1 | Other | 4.8 KB | 58890ed59d4d |
| runs/Jun30_19-12-05_tukuyomi.tstcl.local/events.out.tfevents.1656583936.tukuyomi.tstcl.local.3869584.0 | Other | 9.6 KB | 8f700f4c8e29 |
| runs/Jun30_19-12-05_tukuyomi.tstcl.local/events.out.tfevents.1656645284.tukuyomi.tstcl.local.3869584.2 | Other | 523 B | af66e74cb733 |
| .gitattributes | Repository | 1.2 KB | — |
| .gitignore | Repository | 13 B | — |
| spiece.model | Tokenizer | 791.7 KB | d60acb128cf7 |
| tokenizer.json | Tokenizer | 1.4 MB | — |
| tokenizer_config.json | Tokenizer | 1.9 KB | — |
License and Download
- License
- apache-2.0
- Access
- Open weights, no gate
- Download size
- 242.1 MB
Download from Luis Fernando Costa
Released by Luis Fernando Costa through its official repository on Hugging Face. Read the license.
Built From
- Trained on (disclosed) cnn_dailymail
Evaluations
Each result is shown as reported, with the conditions its reporter stated. None is a SAVRN measurement. A comparison lines two results up only when their configuration, unit and setup are all stated and identical.
| Benchmark | Conditions | Result | Reported by | Revision | Date |
|---|---|---|---|---|---|
| cnn_dailymail | Task Sequence-to-sequence Language ModelingMetric Rouge1Comparison conditions not established | 35.154 | ubikpt Publisher reported |
Evaluated revision not stated | — |
Memory Requirements
| Precision | Weights in memory |
|---|---|
| As published | 242.1 MB |
Weights only, from the published parameter count; the key-value cache and runtime add to this.
Questions About t5-small-finetuned-cnn-v2
Can I use t5-small-finetuned-cnn-v2 commercially?
Yes. t5-small-finetuned-cnn-v2 is released under Apache License 2.0. The Apache License 2.0 is a permissive open-source license. It permits commercial use, modification and redistribution. It requires keeping the license and copyright notices and any NOTICE file, stating significant changes, and it includes an express patent grant from contributors.
Similar Models
Original TF 1 code here Authors: Jingqing Zhang, Yao Zhao, Mohammad Saleh and Peter J. Liu on Dec 18, 2019 The following is copied from the authors' README. We train a pegasus model with sampled gap sentence ratios on both C4 and HugeNews, and stochastically sample important sentences. The updated the results are reported in this table. The "Mixed & Stochastic" model has the following changes: - trained on both C4 and HugeNews (dataset mixture is weighted by their number of examples). - trained for 1.5M instead of 500k (we observe slower convergence on pretraining perplexity). - the model uniformly sample a gap sentence ratio between 15% and 45%. - importance sentences are sampled using a…
This model was trained using Amazon SageMaker and the new Hugging Face Deep Learning container.
This checkpoint should be loaded into BartForConditionalGeneration.frompretrained. See the BART docs for more information.
This repository contains the mT5 checkpoint finetuned on the 45 languages of XL-Sum dataset. For finetuning details and scripts, see the paper and the official repository. Scores on the XL-Sum test sets are as follows: Language | ROUGE-1 / ROUGE-2 / ROUGE-L Amharic | 20.0485 / 7.4111 / 18.0753 Arabic | 34.9107 / 14.7937 / 29.1623 Azerbaijani | 21.4227 / 9.5214 / 19.3331 Bengali | 29.5653 / 12.1095 / 25.1315 Burmese | 15.9626 / 5.1477 / 14.1819 Chinese (Simplified) | 39.4071 / 17.7913 / 33.406 Chinese (Traditional) | 37.1866 / 17.1432 / 31.6184 English | 37.601 / 15.1536 / 29.8817 French | 35.3398 / 16.1739 / 28.2041 Gujarati | 21.9619 / 7.7417 / 19.86 Hausa | 39.4375 / 17.6786 / 31.6667…