A single update pushed out from an anti-virus company in the US has been said to be to be the cause of the global havoc yesterday.
It’s being described as the biggest outage ever, and while there have been a few lately, it’s hard to recall something that has taken out as many services and companies across the world as this one has. You may never have heard of the anti-virus firm Crowdstrike before but something it did to its virus scanner Falcon had a very adverse effect on computers running Windows software – in their millions.
Microsoft was quick to say it was a “third-party issue” – in other words, not its fault. Apple and Linux users, unaffected, rejoiced. Crowdstrike was quick to say it had issued a fix but several IT contacts said every single machine in their organisations required a manual reboot in safe mode, with some devices more physically accessible than others.
There was no suggestion that it was malicious, or that anybody’s data was compromised, accessed or stolen. The cyber-security world still advised that it was a good idea to keep on top of software updates – although perhaps yesterday was not the day to bang that particular drum.
Crowdstrike’s statement, when it arrived, stopped short of an apology, which infuriated people online. However, shortly after the statement was released, CEO George Kurtz told US broadcaster NBC News: "We were deeply sorry for the impact that we caused to customers, to travellers, to anyone affected by this, including our companies."
It was a poignant reminder of how reliant the world has become on devices managed remotely by huge companies, and how powerless it leaves us when they fail. These enormous platforms are bombarded all the time with attempted cyber-attacks, and ill-thought-out software updates, and most of them are caught by the tech giants’ robust systems. There will no doubt be a post-mortem at Microsoft as to why this one was not.
Timing is also everything. “Never push an update on a Friday,” sighed one computer scientist. That's because if something goes wrong and it takes time to fix, firms typically have fewer people working at weekends so it will inevitably take even longer to resolve.
For that reason, many big firms do tend to prefer updates during the middle of the week. For Crowdstrike customer, there are details on its support website explaining the fix. For people working for a company with an IT team, they may well be co-ordinating a company-wide response.
Often by the time an outage is noted, it has fixed itself. This is certainly not the case here.
It is likely to be a few days at least before the world returns to normal.
