How Agents Decide: Goodfire's Eric Bigelow on Critical Tokens, Phase Shifts, & In-Context Learning

Hello, and welcome back to The Cognitive Revolution. After recently speaking with Bronson Shane of Apollo Research, who explained that even with access to models' internal chain of thought, it is still often extremely difficult to determine how a model will decide to act, I asked Claude to survey the literature to see what the field as a whole understands about how these critical decision tokens are chosen.

It goes beyond the title with direct discussion of like, think, really, including: Today, my guest is Eric Bigelow, member of technical staff at Unicorn Mechanistic Interpretability Startup, Goodfire.

1. 01After recently speaking with Bronson Shane of Apollo Research, who explained that even with access to models' internal chain of thought, it is still often extremely difficult to de
2. 02Eric recently completed a PhD in the Harvard Psychology Department with a dissertation titled Toward a Cognitive Science of Large Language Models
3. 03We start today with a survey of Eric's work over the last couple of years, beginning with his 2024 paper on Forking Paths in Neural Text Generation, in which he conducted a massive