Floppy Risks

The FAA computer system fell over. To avoid any potential plane crashes, they called a ground stop at all US airports. The flights already in the air were able to continue and land safely, but planes heading to any US airports weren’t allowed to depart. This resulted in over 16,000 flights being cancelled, inconveniencing at least 2.5m travellers and making life hell for thousands of airline call center staff and travel agents.

I find this quite staggering. I can build an app by simply telling my computer what I want it to do. I can be picked out in a sea of faces by a CCTV camera. I can buy glasses that will display Google Maps directions to a destination I dictated. We are witnessing AI perform computational feats that would have seemed impossible just a few years ago. We have seen rogue AI agents break out of their supposedly secure environments and hack other companies’ systems. Consequently, many see the threat of AI taking over the world as a very viable possibility.

And leading the charge in this game of technological leap-frog is the US. The skyrocketing valuations of big tech companies are further evidence of their predicted prowess. So, when I read that the FAA flight control systems are so antiquated that they’re still reliant on floppy disks (a technology that was sunsetted over a quarter of a century ago), I am a little stunned. As contradictions go, this is a pretty big one.

Here’s another doozy: although the US spends enormous sums on medical technology, AI-assisted diagnostics, robotic surgery, and advanced pharmaceuticals, a recent survey found that 46% of healthcare facilities still relied on paper fax machines for communicating between providers!

The irony is that we are worried about controlling AI, yet we are seeing incidents like the recent Microsoft Outlook outage, which affected millions, the FAA ground stop, and cybersecurity company CrowdStrike’s faulty software update that caused millions of Microsoft Windows computers around the world to crash in 2024. I could go on. All this suggests we are not fully in control of our current systems - which gives one cause to doubt how effective we may be at controlling new systems.

We frequently blame the technology, yet central to all this, and less mentioned, is the fallibility of humans. We tend to construct predictions as though people will behave rationally and consistently.

They don’t. Exhibit A: Jensen Huang. Exhibit B: Donald Trump.

People have egos, insecurities, competing interests, prejudices, and occasionally spectacular lapses in judgement. And those people are making the decisions about how technology gets developed, adopted, regulated, and used. History has shown that people are the wildcards that can throw our best predictions out the window.

We've become remarkably good at making things more sophisticated. But the unfortunate reality is that our ability to adapt to new technology and manage its consequences evolves much more slowly than our ability to create it.

How do you upgrade humans?

Next
Next

What would make the right customer choose us?