OpenAI's Mystery Math Bot Drops 722 Proofs, But Only 22% Pass The Receipt Test 🧾

OpenAI published 722 math manuscripts on GitHub on Tuesday, all generated by an unnamed internal model the company has not released, prompting mathematicians to demand transparency before accepting the results as verified. An OpenAI spokesperson said nearly every paper was produced from a single prompt delivered to a single AI agent, though some may have required multiple attempts. The company said it posed roughly 4,000 problems to the model and kept the outputs it judged significant enough to publish, with the average result consuming the equivalent of about three hours of ChatGPT Pro thinking compute. The release follows a separate Navier-Stokes claim last month, which OpenAI said used 10,000 coordinating agents working for 88 hours.

The manuscripts are organized into 372 "families" of related results, a structure that can bundle a main theorem with companion arguments, consequences, or alternative proofs, meaning 722 represents the number of papers rather than distinct solved problems. OpenAI released abridged reasoning summaries for 10 of the results. Only 162 of the 722 papers include a computer-checked main result, according to a formalization catalog in the repository, equivalent to roughly 22% of the collection and translated into Lean, a program that mechanically verifies every logical step. OpenAI acknowledged that not all manuscripts carry Lean formalizations and that "some of the unformalized results could have issues." A passing Lean check confirms only that the proof follows the statement as written in Lean and does not verify that the statement matches the original problem or that the result is novel or significant.

Reaction from the mathematics community was skeptical. "Until and unless they release the model and people can replicate their results, I think you should treat any claims about one-shotting problems with a single agent as unverified," Andrew Sutherland, a mathematician at MIT, told Scientific American. "We should ask for receipts," he said. Researchers flagged additional concerns, including that the openai/math repository has Issues disabled and has never accepted a pull request, limiting channels for outside Lean formalizations. The Institute for Advanced Study in Princeton, New Jersey, said in a statement, "It is now the case that AI can output mathematical arguments in situations without the human who prompted it being able to understand the arguments, verify them, or take responsibility for them." OpenAI has not announced a release date for the model used in the experiment.