Week 9, Lecture 2 - Percy’s Talk

On in-context learning.

Outline:

1. What transformers can learn in-context (functions!)
2. What exactly is in-context learning (via Bayesian inference)?

What is “learning” in in-context learning?

• Standard learning: gradient steps
• in-context: probabilistic inference (computing a posterior) conditioned on in-context samples

to-do: learn more about icl and vision-icl.