This model is a fine-tuned version of distilroberta-base on the financialphrasebank dataset. It achieves the following results on the evaluation set: This model is a distilled version of the RoBERTa-base model. It follows the same training procedure as DistilBERT. The code for the distillation process can be found here. The model has 6 layers, 768 dimension and 12 heads, totalizing 82M parameters (compared to 125M parameters for RoBERTa-base). On average DistilRoBERTa is twice as fast as Roberta-base. Polar sentiment dataset of sentences from financial news. The dataset consists of 4840 sentences from English language financial news categorised by sentiment. The dataset is divided by…
Open weights
apache-2.0
82M parameters
514 tokens
transformers
This model is a warm-started BERT2BERT (small) model fine-tuned on the CNN/Dailymail summarization dataset. The model achieves a 17.37 ROUGE-2 score on CNN/Dailymail's test dataset. For more details on how the model was fine-tuned, please refer to this notebook.
Open weights
apache-2.0
transformers
This model is a warm-started BERT2BERT (mini) model fine-tuned on the CNN/Dailymail summarization dataset. The model achieves a 16.51 ROUGE-2 score on CNN/Dailymail's test dataset. For more details on how the model was fine-tuned, please refer to this notebook.
Open weights
apache-2.0
23M parameters
transformers
This model is provided by BETO team and fine-tuned on SQuAD-es-v2.0 for Q&A downstream task. Language model ('dccuchile/bert-base-spanish-wwm-cased'): BETO is a BERT model trained on a big Spanish corpus. BETO is of size similar to a BERT-Base and was trained with the Whole Word Masking technique. Below you find Tensorflow and Pytorch checkpoints for the uncased and cased versions, as well as some results for Spanish benchmarks comparing BETO with Multilingual BERT as well as other (not BERT-based) models. The model was trained on a Tesla P100 GPU and 25GB of RAM with the following command: 1. Set the context and ask some questions: 2. Run predictions
Open weights
512 tokens
transformers
This model was created by Google and fine-tuned on XQuAD like data for multilingual (11 different languages) Q&A downstream task. Deepmind XQuAD As the dataset is based on SQuAD v1.1, there are no unanswerable questions in the data. We chose this setting so that models can focus on cross-lingual transfer. We show the average number of tokens per paragraph, question, and answer for each language in the table below. The statistics were obtained using Jieba for Chinese and the Moses tokenizer for the other languages. As XQuAD is just an evaluation dataset, I used Data augmentation techniques (scraping, neural machine translation, etc) to obtain more samples and split the dataset in order to…
Open weights
178M parameters
512 tokens
transformers
