How Many Tests Is Enough for a Brain Model?

2026-10-18 · 4 min read · consciousness, ai

So, new paper. Paper 13. And honestly this one’s been living in my head for a while.

Here’s the question. Say somebody builds a model of a brain. Like a real one, wired up from an actual connectome. How do you even know it works like the brain?

One of the best ways we have is breaking it on purpose. You turn off one type of cell in the model, you turn off the same cells in a real fly, and you see if they change the same way. Scientists call it silencing.

The problem is the combinations. The fly visual system model I’ve been working with has 65 cell types you can turn off. So that’s 65 tests for singles. Pairs is 2,145. Triples is 45,825. Every set of eight is like 5 billion. Nobody’s ever running all of that. Not in a model, and definitely not in a real fly.

So the actual question is, if you test the singles and the pairs, how much does that tell you about the bigger combos you never tested?

That’s Paper 13.

If turning off two cell types always did exactly what you’d guess from turning off each one alone, you’d be done. Test the singles, add them up, whatever. But that’s not how it works. Sometimes two things together do way more than the sum, or they cancel each other out. That leftover part is the interaction. And basically the whole thing comes down to, do those interactions get smaller as you stack more cells, and how fast?

If they shrink fast, a small pile of tests covers a lot. If they don’t, you’re kind of never done testing.

Back in Paper 10 I argued a finite set of tests can’t vouch for a brain model on combos it’s never seen, unless something keeps those interactions in check. So this time I went and tried to measure that something.

For a simple linear network I could actually prove it. When you turn off a set of cells, the interaction is basically a sum over every path through the network that goes through all of them. Bigger sets need longer paths, and the signal fades a little with every step. So the interactions have to shrink at least as fast as that fade. And if they shrink like that, testing up to some size covers everything bigger, with an error you can actually calculate. “How many tests” turns into a budget. I had that part checked by a proof assistant, Lean 4, so it’s not just me saying it.

But there’s a catch, and the paper proves that part too. Real model neurons aren’t linear. They can’t go below zero. And for neurons like that, nothing forces the interactions to shrink. You can build a tiny network where they stay the same size forever. So for real models, you just have to go measure it.

So I measured it. I used flyvis, which is 50 published models of the fly visual system built from the connectome, and turned off every set up to five cell types. 63,400 new simulations. Before I ran fours and fives, I wrote down my predictions for them based only on singles, pairs and triples, and locked them in. Then I ran it once.

The interactions did shrink. Like 70 times smaller from singles to sets of five. And the share that was big enough to matter roughly halved every step.

But it slowed down. Every step shrank it less than the one before, and at five it came in about twice as big as I predicted. So my main prediction failed. I’m keeping that in the paper as a fail, because it was one.

Then I went looking for where the leftover was hiding. Almost all of it sits in sets that include one of a few cell types that basically max out the response on their own. Take those out and everything else pretty much just adds up.

And this one kind of got me. The wiring diagram can tell you where the interactions are, but not which way they go. Synapse counts ranked where they were pretty well at every size. But whether two cells push together or against each other? At or below a coin flip.

So how many tests is enough? For these models, testing every triple covers sets of five on average. Sets of eight would need tests of five or six at a time, and that’s millions of experiments.

I also went through what the published fly emulation projects have actually tested. Most of them test one cell type at a time, or nothing at all. So their tests don’t really tell you much about combos, unless you assume the interactions behave like they do in flyvis, and nobody’s checked that. Eon, one of the fly upload companies, wrote that a model should be validated “especially for interventions it was not tuned on.” I agree. This is a way to actually measure that.

To be clear, this isn’t about consciousness, or whether uploading a fly is possible. It’s one set of models, one way of measuring, and the math only covers linear networks. It’s just about what a stack of tests buys you and what it doesn’t.

I don’t know. I’m kind of proud of this one. Part of it’s proved, part of it’s measured, and the part that failed is in there too.

The paper and everything you need to rerun it is on Zenodo.

Licensed CC-BY-4.0.