Tatsu Lab

Alpaca is a dataset of 52,000 instructions and demonstrations generated by OpenAI's text-davinci-003 engine. This instruction data can be used to conduct instruction-tuning for language models and make the language model follow instruction better. The authors built on the data generation pipeline from Self-Instruct framework and made the following modifications: - The text-davinci-003 engine to generate the instruction data instead of davinci. - A new prompt was written that explicitly gave the requirement of instruction generation to text-davinci-003. - Much more aggressive batch decoding was used, i.e., generating 20 instructions at once, which significantly reduced the cost of data…