The real AI danger is not superintelligence. It is premature trust. | Andrew Rogoyski

September 19, 2026 thedigitalsolutioncentre@gmail.com

[post_content]

Disclaimer: This article has been automatically aggregated from

A gift from Live Science Pro

This is a Live Science Pro article that we’re putting out for free. If you liked this story, please consider subscribing.

Artificial intelligence (AI) terrifies us partly because it is new and partly because it is disturbingly familiar. We have invented machines that converse, flatter, argue and improvise with uncanny fluency. Confronted by this synthetic reflection, we behave like early humans seeing a mirror for the first time: We project consciousness behind the glass, then populate it with our expected behaviors, emotions, hopes and nightmares. In other words, we scare ourselves.

Today’s frontier models are impressively good at writing, coding and math. Yet they still fail at ingredients of useful artificial general intelligence (AGI), a system that matches or surpasses human capabilities such as consistent reasoning, long-term planning, factual consistency and reliable performance outside their “experience” (the data they were trained on). We are right to worry about the theoretical dangers of artificial superintelligence (ASI) because of the danger that such a system might outwit its human creators and pursue its own ends, potentially at our expense. While some AI executives have asserted that we’ve reached AGI, these claims are contentious and do not meet more considered definitions of AGI.

not a digital god awakening and launching a Hollywood-style war against humanity. After all, AI depends on electricity, cooling, data centers, specialized chips, networks, technicians and sprawling supply chains, all of which are scarce and often fragile. Interrupt the power, restrict the graphics processing units (GPUs) used to train them, disable the cooling or disconnect the network, and in most scenarios, the supposedly omnipotent machine that’s out to kill humanity stops.

Andrew Rogoyski

Andrew’s experience spans over 30 years in industry, government and academia. Originally a physicist at the Rutherford Appleton Lab, Andrew joined Logica at the height of the early AI boom in the early 90s and has subsequently held high-level positions for Esys, Qinetiq, and CGI UK, as well as serving a secondment with the U.K.’s Cabinet Office. Andrew later joined Roke Manor Research as head of research and development, returning to academia to help set up the University of Surrey’s Institute of People-Centred Artificial Intelligence.

A true intelligence might also recognize that its understanding of the physical world is limited and secondhand ‪—‬ accessible to it only by virtue of the texts, videos and audio feeds it can access. It lacks that crucial element of being embodied in the world — understanding how gravity behaves, what weather feels like, how objects interact, and so on. A true intelligence might realize this, see the limitations of its experience and, therefore, tread with caution. A true intelligence might be completely different from human intelligence, having other strengths, weaknesses, approaches, and even ethics. For example, consider that an AI intelligence can copy itself, running as myriad instances, communicating as a “hive mind” — a very different kind of intelligence. But I speculate.

What I can say is that the systems that should worry us most are not necessarily the smartest ones but rather the incomplete intelligences acting on narrow objectives, given agency and offered access to critical infrastructure. These systems may accidentally harm us simply because they don’t know any better. For example, there is a thought experiment created by Nick Bostrom, known as the “paper clip problem,” where an AI pursues its objectives of making as many paper clips as it can, but it then achieves that aim at the expense of the Earth and humanity which all get converted to paperclips in the pursuit of the AI’s narrow goals.

AI-enabled cyberattacks could significantly disrupt advanced economies because hospitals, banks, energy networks, logistics companies and governments depend on interconnected digital systems. Recent examples of frontier AI systems hacking organizations, such as the OpenAI/Hugging Face incident, have given currency to this fear. Such attacks can result in extraordinary financial damage without being existential. Large parts of the world remain loosely connected to frontier AI — or even to the internet — and would not disappear simply because servers in wealthy countries failed.

The likeliest route by which advanced AI could exert control is not killer robots, but influence — misleading people, manipulating institutions and redirecting human effort. Yet humans already demonstrate formidable competence at misinformation, self-deception and destructive coordination. AI may industrialize those weaknesses; indeed, humans are using AI to industrialize those limitations, but AI did not invent them.

Economically, the danger is that companies will dismiss workers long before AI has proved capable of replacing them, cashing in the “AI dividend” too soon. Executives seduced by demos may confuse fluent language output with dependable labor, shedding institutional knowledge to chase theoretical efficiencies. The enticing vision of the “one-person unicorn,” where a person or a small group builds a billion-dollar business using AI as its workforce is questionable technically, economically and socially, even as a small handful of early examples take shape. Organizations are not merely bundles of tasks. They contain accountability, relationships, tacit knowledge and trust, forming part of a local and national community, resilient because of the diversity of skills and ideas they possess.

My suspicion is that the political turning point will resemble something like the 2017 WannaCry ransomware attack, where malware with core elements designed by the National Security Agency got wildly out of control, infecting hundreds of thousands of computers worldwide, infamously disrupting the U.K.’s National Health Service. A cyber operation amplified by frontier AI would be worse. Services would fail, lawsuits would multiply, companies could collapse and investors would retreat. Governments would demand a pause — or force laboratories to redirect resources toward control, auditing and safety, and companies would comply, if only to limit class-action legal exposure.

There is also an uncomfortable likelihood that AI labs themselves are amplifying fear and uncertainty. Casting a product as potentially world-ending implies unprecedented power. Such narratives are just marketing; they may help to sustain investment, defend stock valuations, and prepare for forthcoming initial public offerings (IPOs) even when business models remain uncertain. It’s hard to speculate about a company’s motivations ‪—‬ but there are obvious incentives, so their behaviors deserve scrutiny.

The training and operation of frontier systems consume enormous amounts of computing time, electricity, water, capital, hardware and human effort. Unless those costs fall to a fraction of today’s levels, the economics of “Big AI” may prove unsustainable. Meanwhile, important unsolved problems in AI ‪—‬ like reliability, interpretability and continuous learning ‪—‬ are being overshadowed by the race to build incrementally better models, each vying for the top slot in the various AI performance indices.

The strongest case for AI may instead lie in artificial specialized intelligence: systems such as AlphaFold2, which predicts the structures of proteins, or WeatherNext 3, an advanced global weather forecasting model. These systems are designed for domains where performance can be measured and where the benefits justify the costs. We know AI is genuinely useful when aimed at topics like drug discovery, diagnostics, materials science, power grid optimization and climate change modeling — rather than human-like chatbots generating emails, advertisements and synthetic entertainment.

for informational purposes only. We do not claim ownership, accuracy, or liability for the content provided. All rights belong to the original publisher.

The Digital Solution Centre

The Digital Solution Centre

The real AI danger is not superintelligence. It is premature trust. | Andrew Rogoyski

A gift from Live Science Pro

Like this:

Related

Like this:

A gift from Live Science Pro

Share this:

Like this:

Related