Put your math hats on for a minute, and let’s check out what this beef from mid-October was about. It’s an ideal instance of what’s unsuitable with AI proper now.
Bubeck was excited that GPT-5 appeared to have in some way solved numerous puzzles referred to as Erdős issues.
Paul Erdős, probably the most prolific mathematicians of the twentieth century, left behind tons of of puzzles when he died. To assist maintain monitor of which of them have been solved, Thomas Bloom, a mathematician on the College of Manchester, UK, arrange erdosproblems.com, which lists greater than 1,100 issues and notes that round 430 of them include options.
When Bubeck celebrated GPT-5’s breakthrough, Bloom was fast to call him out. “It is a dramatic misrepresentation,” he wrote on X. Bloom defined that an issue isn’t essentially unsolved if this web site doesn’t listing an answer. That merely means Bloom wasn’t conscious of 1. There are tens of millions of arithmetic papers on the market, and no person has learn all of them. However GPT-5 in all probability has.
It turned out that as an alternative of arising with new options to 10 unsolved issues, GPT-5 had scoured the web for 10 current options that Bloom hadn’t seen earlier than. Oops!
There are two takeaways right here. One is that breathless claims about huge breakthroughs shouldn’t be made through social media: Much less knee jerk and extra intestine examine.
The second is that GPT-5’s means to seek out references to earlier work that Bloom wasn’t conscious of can be wonderful. The hype overshadowed one thing that ought to have been fairly cool in itself.
Mathematicians are very enthusiastic about utilizing LLMs to trawl via huge numbers of current outcomes, François Charton, a analysis scientist who research the appliance of LLMs to arithmetic on the AI startup Axiom Math, informed me after I talked to him about this Erdős gotcha.
