Excellent piece. The Othello example is especially powerful because it shows emergence in a controlled setting where we can verify what the model learned. The fact that the internal board representation is linearly encoded makes it harder to dismiss as statistical noise. What's fascianting is how this parallels debates in other fields where people confuse descriptions with limits. The brittleness findings you mention don't undermine the world model argument, they just show that these models are fragiel at the boundaries of what they've generalized, which is consistent with systems that build real structure but haven't fully explored it.
This lands-"fragile at the boundaries of what they've generalized" is precisely the reframe. Not refutation, characterization. Genuine structure with edges it hasn't fully explored yet.
And yeah, Othello is special because it's checkable. The linear encoding means you can literally see what the model built. Harder to wave away.
Your last question is the one that keeps nagging at me too: why does this move keep getting made, even in fields that should recognize it?
Excellent piece. The Othello example is especially powerful because it shows emergence in a controlled setting where we can verify what the model learned. The fact that the internal board representation is linearly encoded makes it harder to dismiss as statistical noise. What's fascianting is how this parallels debates in other fields where people confuse descriptions with limits. The brittleness findings you mention don't undermine the world model argument, they just show that these models are fragiel at the boundaries of what they've generalized, which is consistent with systems that build real structure but haven't fully explored it.
This lands-"fragile at the boundaries of what they've generalized" is precisely the reframe. Not refutation, characterization. Genuine structure with edges it hasn't fully explored yet.
And yeah, Othello is special because it's checkable. The linear encoding means you can literally see what the model built. Harder to wave away.
Your last question is the one that keeps nagging at me too: why does this move keep getting made, even in fields that should recognize it?