A null result worth reading twice

In 2023, de Barros, Ganimian and Muralidharan published a pre-registered randomised trial of personalised instruction with 3,331 students. The headline finding is the kind that usually ends a conversation: no significant average effect on maths achievement.

If you sell personalised instruction, that is an awkward number. We think it is the most useful number we have.

The average was hiding two different results

The trial reported effects by where students started. For students who started with the lowest scores, the effect was +0.22 standard deviations. That is not a rounding error. It is a large effect in education research, and it sat inside an average of approximately zero because it was being cancelled out by students for whom personalisation added little.

Both halves are true at once. Personalised instruction is not a general improvement to teaching. It is a specific intervention that does most of its work in one place.

What that means for who we build for

There is a phrase in Chinese education for this distinction: 補底 rather than 拔尖 — shoring up the foundation rather than sharpening the top. They are different jobs and they need different tools.

We build for the first. Not because the second is unworthy, but because the evidence says that is where the same effort produces several times the result, and because it is where the current system is least useful. A student who is already well served by ordinary classroom teaching does not need what we make.

This is also why the product is designed for neurodivergent students first rather than adapted for them later. That decision is ours and the evidence for it is weaker — the trial above says something about where personalisation pays, not about neurodivergence specifically. We hold the two claims at different strengths on purpose.

The uncomfortable part

A null average result is the sort of finding a company quietly does not cite. We cite it because the alternative is to claim personalisation works for everyone, which this trial says it does not, and because a tutor who tries our software with a student it was never going to help will correctly conclude we were overselling.

The claim we are willing to make is narrower and testable: for students the current system serves least well, structured personalisation produces a real effect. Everything else we say about our own product is still a hypothesis, and we label it as one.

Source: de Barros, A., Ganimian, A. J., & Muralidharan, K. (2023), pre-registered randomised trial, n = 3,331. The targeting claim is measured, with a comparison group. The neurodivergent-first design decision is our own reasoning, not a measurement.