A different test for every student — and a reason to trust it

A standardized test gives the same questions to a future physicist and to a student who is still shaky on fractions. The first is bored. The second is overwhelmed. Both walk away with a single number and almost no idea what they actually know.

There has always been a better way. A test that starts in the middle, then adjusts: get one right, the next is a little harder; miss one, it eases off — homing in on what you can actually do. It measures more precisely, in far fewer questions, without wasting anyone's time. It even feels less stressful.

This isn't speculative. It's called adaptive testing, the theory dates to the 1970s, and you've likely already taken one: the GRE, the GMAT, and — as of 2024 — the SAT itself all work this way.

The science was never the hard part

So why doesn't every student get this?

Because an adaptive test is hungry. To always have the right next question ready — at every ability level, without repeating itself enough to leak — it needs a deep, carefully calibrated bank of questions. Writing and maintaining that bank is a tens-of-millions-a-year operation, run by a handful of organizations. That cost, not the science, is the wall between adaptive assessment and the billion-plus students who never get near it.

Which makes the obvious move tempting: just ask an AI to write the questions. And for reading or history, that mostly works today.

For mathematics, "mostly works" is a trap

A language model is a brilliant, confident guesser. On a novel math problem it will sometimes produce an answer key that is simply wrong — and state it with total fluency.

On a normal test, a human reviewer catches that before anyone sees it. Inside an adaptive system, there's no such safety net. The same machine-made question can be served to thousands of students before anyone notices the key was wrong — and every score it touched is now quietly corrupted. A sharper prompt doesn't fix this. You can't prompt your way to certainty.

The unlock isn't a smarter model. It's a referee the model has to get past.

So the design flips. Instead of trusting the AI's answer, you make the AI prove it. Every question runs through a Computer Algebra System — the same kind of exact symbolic engine mathematicians use — which solves it independently. If the math checks out, the question ships. If it doesn't, it's thrown away and never reaches a student. The answer isn't correct because a model is confident. It's correct by construction.

This is not a thought experiment. I built it. satlike_problem_creator is an open-source engine that generates exam-grade mathematics this way — symbolic verification at its core, plus a few moves borrowed straight from psychometrics: it varies what each question asks so students can't pattern-match, it dials difficulty up or down on purpose, and it studies seventeen years of past competitions to stay true to their style. It already runs in production, generating the problems for the Premio Di Nicola mathematics competition, now in its eighteenth edition.

Where this goes

When the cost of a good question falls, school changes

Here's the part that's bigger than one competition. Verified generation collapses the cost of building and maintaining a calibrated question bank by one to two orders of magnitude. The thing that only ETS and Pearson could afford starts to fit inside a single school's means.

Follow that thread and the classroom looks different. Assessment stops being a once-a-year verdict and becomes a quiet instrument running in the background — one that meets each student exactly where they are, tells a teacher precisely which idea hasn't landed yet, and refreshes itself with new, verified questions on demand. Personalized, trustworthy measurement stops being a privilege of well-funded institutions and starts looking like something every student could simply have.

And the principle reaches past education. "Don't trust the model — verify its output against something that can't be talked into agreeing" is one of the most promising directions in AI right now. A language model proposes; an exact, symbolic system disposes. Education is just an unusually clean place to prove the pattern works, because in mathematics there is no arguing with whether the answer is right.