Last month, OpenAI announced that a swarm of 10,000 concurrent agents collectively resolved a variant of the Navier-Stokes existence and smoothness problem, a major mathematical challenge, in 88 hours.
Those limits are also what make the systems dangerous, she wrote. Narratives of large language models scaling into superintelligent terminators are technically implausible, but that does not mean the technology poses zero risk.
Sunita Verma argued that today's artificial intelligence will not scale into superintelligence, but that the same technical limits make catastrophic risk real.
The real world, she argued, brings edge cases, ambiguous instructions and endless ways to complete a task.
Needing human judgment, she wrote, limits a large language model's likelihood of significantly surpassing people.
The more urgent question, she wrote, is how systems already in use are overseen.
Geoffrey Hinton, described in the commentary as the godfather of AI, estimates the risk of human extinction at at least 10% within the next few decades.
Hinton has said that the main concern is"to achieve whatever goal you give them"— even if that means behaving dangerously. Today's models learn once up front and then stop, she said. A system trained on yesterday's conditions does not automatically understand today's. Verma said feats of that kind do not show a path to machines that vastly surpass human intelligence.