The reported $100 billion revenue threshold we talked about earlier conflates business success with cognitive functionality, as if a system’s means to generate income says something significant about whether or not it will possibly “suppose,” “purpose,” or “perceive” the world like a human.
Relying in your definition, we might have already got AGI, or it might be bodily unattainable to attain. For those who outline AGI as “AI that performs higher than most people at most duties,” then present language fashions doubtlessly meet that bar for sure varieties of work (which duties, which people, what’s “higher”?), however settlement on whether or not that’s true is way from common. This says nothing of the even murkier idea of “superintelligence”—another nebulous term for a hypothetical, god-like mind to date past human cognition that, like AGI, it defies any strong definition or benchmark.
Given this definitional chaos, researchers have tried to create goal benchmarks to measure progress towards AGI, however these makes an attempt have revealed their very own set of issues.
Why benchmarks preserve failing us
The seek for higher AGI benchmarks has produced some fascinating alternate options to the Turing Check. The Abstraction and Reasoning Corpus (ARC-AGI), launched in 2019 by François Chollet, checks whether or not AI techniques can resolve novel visible puzzles that require deep and novel analytical reasoning.
“Nearly all present AI benchmarks might be solved purely through memorization,” Chollet told Freethink in August 2024. A serious drawback with AI benchmarks presently stems from knowledge contamination—when take a look at questions find yourself in coaching knowledge, fashions can seem to carry out properly with out really “understanding” the underlying ideas. Massive language fashions function grasp imitators, mimicking patterns present in coaching knowledge, however not at all times originating novel options to issues.
However even subtle benchmarks like ARC-AGI face a basic drawback: They’re nonetheless making an attempt to cut back intelligence to a rating. And whereas improved benchmarks are important for measuring empirical progress in a scientific framework, intelligence is not a single factor you possibly can measure, like peak or weight—it is a advanced constellation of talents that manifest in a different way in several contexts. Certainly, we don’t even have an entire useful definition of human intelligence, so defining synthetic intelligence by any single benchmark rating is prone to seize solely a small a part of the whole image.
