Who Grades the AI Models? | Ben Horowitz & Rayan Krishnan

Every time a new trillion-dollar industry emerges, there's a need for this independent testing group. When Meta released Lama 4 on our held-out private benchmarks, the model was actually underperforming, but on all of the major public benchmarks, it was showing incredible capabilities.

It goes beyond the title with direct discussion of think, like, actually, including: When Meta released Lama 4 on our held-out private benchmarks, the model was actually underperforming, but on all of the major public benchmarks, it was showing incredible capabilit.

1. 01Every time a new trillion-dollar industry emerges, there's a need for this independent testing group
2. 02When Meta released Lama 4 on our held-out private benchmarks, the model was actually underperforming, but on all of the major public benchmarks, it was showing incredible capabilit
3. 03In an ideal world, take a frontier model and have it train the next version of itself, but obviously that's very expensive and slow, and so what we're doing is forming a set of pro