The reported $100 billion revenue threshold we talked about earlier conflates business success with cognitive functionality, as if a system’s capacity to generate income says something significant about whether or not it could actually “suppose,” “purpose,” or “perceive” the world like a human.
Relying in your definition, we might have already got AGI, or it might be bodily not possible to realize. When you outline AGI as “AI that performs higher than most people at most duties,” then present language fashions doubtlessly meet that bar for sure kinds of work (which duties, which people, what’s “higher”?), however settlement on whether or not that’s true is much from common. This says nothing of the even murkier idea of “superintelligence”—one other nebulous time period for a hypothetical, god-like mind to this point past human cognition that, like AGI, it defies any stable definition or benchmark.
Given this definitional chaos, researchers have tried to create goal benchmarks to measure progress towards AGI, however these makes an attempt have revealed their very own set of issues.
Why benchmarks preserve failing us
The seek for higher AGI benchmarks has produced some attention-grabbing alternate options to the Turing Check. The Abstraction and Reasoning Corpus (ARC-AGI), launched in 2019 by François Chollet, exams whether or not AI techniques can remedy novel visible puzzles that require deep and novel analytical reasoning.
“Nearly all present AI benchmarks will be solved purely through memorization,” Chollet instructed Freethink in August 2024. A serious downside with AI benchmarks presently stems from information contamination—when check questions find yourself in coaching information, fashions can seem to carry out properly with out actually “understanding” the underlying ideas. Massive language fashions function grasp imitators, mimicking patterns present in coaching information, however not at all times originating novel options to issues.
However even subtle benchmarks like ARC-AGI face a elementary downside: They’re nonetheless attempting to scale back intelligence to a rating. And whereas improved benchmarks are important for measuring empirical progress in a scientific framework, intelligence is not a single factor you may measure, like peak or weight—it is a advanced constellation of talents that manifest in another way in numerous contexts. Certainly, we do not even have an entire practical definition of human intelligence, so defining synthetic intelligence by any single benchmark rating is more likely to seize solely a small a part of the whole image.
