DeepMind has released "A Generalist Agent", a paper that introduces their new multi-modal model Gato. But is Gato truly a generalist agent? It is a transformer based model with the goal of generalizing over new tasks. It is trained fully autoregressively with supervised learning (no reinforcement learning) on a total of 603 different tasks. The tasks include robotics, Atari, DM Lab, Procgen, and a lot more. It also includes text and image tasks. This video is a paper review / explanation where I also give my thoughts on the paper.

Outline:
0:00 - Intro & Gato Overview
1:20 - Paper Motivation
2:56 - Model Explanation
6:13 - Data & Tasks
7:45 - Initial Results
9:38 - Zero Shot Transfer
12:58 - Generalization Ablation
17:05 - Fine-tuning
19:38 - My Thoughts

Gato blog post: https://www.deepmind.com/publi....cations/a-generalist
Gato paper: https://arxiv.org/pdf/2205.06175.pdf