The Jack of All Trades (JAT) dataset combines a wide range of individual datasets. It includes expert demonstrations by expert RL agents, image and caption pairs, textual data and more. The JAT dataset is part of the JAT project, which aims to build a multimodal generalist agent. The following table presents a comparative analysis of scores across various domains and tasks. The scores highlight the performance difference between a random agent and the episodes recorded in our dataset. - text: a string feature - images: a image feature - imageobservations: a Sequence(image) feature - textobservations: a Sequence(string) feature - discreteobservations: a Sequence(Sequence(int64)) feature…