Teaching Mathematics to a Machine
I spend my working days doing something that did not exist as a profession when I finished my mathematics degree: I evaluate how well a frontier AI model reasons about pure mathematics, and I build the material that teaches it to reason better.
The work has three shapes. I take contemporary research papers and turn them into original problems the model has never seen, hard enough that solving them requires understanding rather than recall. I perform error analysis, dissecting the model's attempts to find the exact step where its reasoning left the road. And I do reasoning-gap completion, presenting arguments with a missing span and asking the model to build the bridge. Every attempt is graded against a rubric of more than fifty points, and for every problem I author a gold-standard solution: the reference the model ultimately learns from.
Here is what surprised me most. The failures that matter are not the obvious ones. When a model makes an arithmetic slip, anyone can catch it. The dangerous failures are fluent. The model writes a paragraph that has the cadence of a proof, the vocabulary of a proof, the confidence of a proof, and somewhere in its third sentence it asserts a claim that is almost true. Almost. The gap between almost true and true is where mathematics lives, and finding that gap in a wall of confident prose is the actual skill of this job. It has changed how I read human mathematics too. Fluency and correctness are different properties, and we conflate them far more often than we admit.
People sometimes ask whether this work is "real mathematics." I understand the question, because I asked it myself. My answer now is that it is mathematics held to an unusual standard. When I write a solution for a human reader, I can lean on shared intuition and say "clearly" at the steps we both find obvious. When I write a gold-standard solution for a machine, nothing is clearly anything. Every inference must be load-bearing. It is the most honest mathematical writing I have ever had to produce, because the reader extends no charity at all.
The larger reason I find this work worth doing is where it points. Models are beginning to function as research collaborators: suggesting bounds, proposing constructions, noticing patterns across literature that no single human has read in full. The suggestions are raw. They arrive unverified, ungeneralised, and sometimes subtly wrong in the fluent way I described. Turning them into actual mathematics, verified, generalised, and taken to their logical conclusion, is human work, and it is the work I want to spend the next years doing. The mathematicians who learn to do it well will not be replaced by these systems. They will be the ones who decide what the systems' output is worth.
A decade ago, the profession I have now did not exist. I suspect the profession I will have in another decade does not exist yet either. That no longer worries me. It is the most interesting property of the frontier: it moves.