The Quiet Surrender to AI

The danger isn’t that AI takes over, it’s that humans will stop bothering

The take-over by AI, when it comes, may be remarkably dull. No wild-eyed robots storming Parliament or screens suddenly going blank in City trading rooms. No superior computer will announce that mankind has been relieved of its duties.

Artificial intelligence doesn’t need to take control for humans to lose it. It’s much more likely that a chain of small, sensible choices will get us there. Arguably, it’s already happening. How many of us check AI results in any detail? Not even experts the designers know exactly how the decisions are made. AI is already making decisions and we’re not bothered about it.

There’s been many reports recently about AI in swarms going it alone, sometimes for weeks before any human knew. We’re obviously starting to know what we don’t know. AI is already ahead of us and we have to chase it to catch what it’s doing because the controls are not adequate.

The situation is not unlike with human criminals. The police are almost never one step ahead. Neither are state intelligence agencies. AI does whatever it does and we have to stay vigilant. At least we know how this goes, we just haven’t set up the AI Police yet.

We know that the big AI developers are on to it but, so far, they look a bit like PC Plod. In some ways they are the people to do it, and in some ways they’re not.

The problem is getting bigger all the time and harder to grip, simply because AI is doing more and more, and is becoming much more sophisticated and faster. A model does a task well, and it’s given another. It works quickly and the whole thing speeds up. Its operations become too numerous and hard to scrutinise, and the time that takes makes that scrutiny seem wasteful. People spend less time doing the work and more time watching the machine do it. In time they become poor judges of what they watch.

A Ukrainian AI battlegroup commander was interviewed by the Economist last month. He said that the driving force behind everything is the need to do damage to the enemy. AI started to offer targets a long time ago, and a human would look, inspect, decide, and either push the button to fire or not. That was humans in the loop. As the weeks went by, less and less time was spent examining potential targets, and the button was pressed more easily. This has become human on the loop.

This is happening everywhere, not just on the battlefield. As the range of competent action grows, delegation moves up the chain. The machine no longer merely types the letter. It drafts it, chooses the supporting evidence, decides which objections matter and sends it for approval.

Each transfer looks modest. Together they alter the job. A doctor may still sign the diagnosis, a banker the loan and a civil servant the benefits decision. But the person at the end of the process increasingly encounters a conclusion rather than a case analysis. The facts have already been selected, ranked and compressed. The official remains responsible for a decision whose construction has largely disappeared from view.

Speed makes the problem worse. A machine can screen thousands of applications, revise millions of prices or conduct a long sequence of digital actions before a person has finished reading the first explanation. Scale creates its own authority. Once an organisation is built around such throughput, careful human review becomes not merely expensive but obstructive.

The system was set up to speed things up, and the human in the loop becomes counter productive. Scrutiny and responsibility becomes an awkward brief, because assurance eats into the benefit that was the whole point in the first place.

Complexity adds another turn. A record of every automated step may exist, yet still be useless to the person with the responsibility. A hundred pages of logs are not an explanation. Nor does the availability of piles of data amount to control. Human-in-the-loop systems suffer from cognitive overload, trust drift and scalability as insideous obstacles to effective oversight. Compliance is counter productive.

Convenience makes coercion unnecessary. If a system is right most of the time, checking every answer begins to look pedantic. If checking the answer takes as long as producing it, the point of automation appears lost. The sensible manager reduces the sample, raises the review threshold, shortens the review or reserves human attention for cases the system itself marks as unusual. That last step is particularly corrupting: the machine now helps decide when the human should doubt the machine.

People drift from doing to monitoring. The trouble with monitoring is that humans are ill suited to watching reliable systems for rare mistakes. I learned in the security industry that a human can scan a bank of screens with keen attention for about thirty minutes, then eyes start to skid across them. I the swimming safety world, a life-guard can remain attentive for a while and then concentration on vigilance declines.

An irony follows. Automation leaves people responsible for the cases least suited to automation, the novel, ambiguous and dangerous ones. Yet good automation deprives them of familiarity with the exceptions. The pilot must take over in the storm, after the autopilot has handled the clear skies. Human judgment is treated as an emergency service but given little opportunity to practice.

Capability leads to progressive delegation, which creates distance and unfamiliarity. Routine begins to look safe, and scrutiny declines. Scrutiny adds friction and the rest of the organisation sees it working against them. The conditions set up the slow process of bothering a little less each day.

So, preserving human control will require deliberate inefficiency. Independent audits will be important but cannot be part of a day-to-day routine in a fast moving business. Greater friction must be accepted in that routine: time to inspect and analyse the evidence, routes of appeal and escalation, and people authorised and empowered to make choices. Firms will need skills on the payroll, and processes that automation makes uneconomic, and the scrutiny will cost a loss of performance.

These things will be unpopular. That is what spare capacity always looks like before it is needed. No one hopes to use a fire extinguisher or claim on insurance, but almost everyone understands their purpose and accepts the cost.

The systems most dangerous to humans, then, may well be those that obey so helpfully, quickly and comprehensively, that giving orders becomes the only part humans remember how to do. By then even that authority may be thin. The goals presented to the machine will have been shaped by the machine and what it can measure; the system designed around what can process, and the pace of work by what it can execute.

There will still be meetings, signatures and people described as decision-makers. Those forms of command and structure will continue but the underlying reality may not.

Artificial intelligence need never develop a taste for power. Convenience, speed and institutional economy can do the work instead. Machines will not have taken control.

Humans will have built a world in which control is too slow, too expensive and too difficult to exercise, and then congratulate themselves for remaining in the loop.