The biggest news we had. Humans make mistakes!
We will not quicky forget July 19th, 2024. Over 80 million Microsoft servers and PC's were experiencing a Blue Screen of Death (BSOD) and immediately halted. Banks, airlines, supermarkets, logistical companies and public offices. There was no sector in the world that was not affected. Because total companies went 'offline', this lead to chaotic situations.
The issue was not being caused by Microsoft, but by a third party Crowdstrike.
A faulty update from this third party is knocking affected PCs and servers offline, forcing them into a recovery boot loop so machines can't start properly, 'A human mistake'.
Now obviously, you can never rule out human mistakes. That is simply impossible. And for that reason and long time ago, we introduced change management to ensure, whatever happens, a save return to a state that did work. This is called a rollback mechanism. If there was no rollback possibility, there would be no update.
Somehow, we seem to have forgotten this. And this example gives a painful view of its consequence! What strikes met even more, is that Microsoft does not feel responsible at all...Well, i disagree. Of course Microsoft is responsible and 'caused' this. Not because they made this 'human error', but in allowing updates that are capable of creating a non reversible situation. What else should you expect from a IT company so many sectors have to rely upon for their professionalism?
So where are we now? In my view we are in big trouble. Looking at many newsfeeds, this just seems to be the new standard how updates are pushed throughout the internet.
If you are responsible for an IT operation that effectively has no control over updates that are flying in from everywhere, I would seriously reconsider my responsibilities. You can not be held responsible for the irresponsible behaviour of companies that ignore or step away from any form of change management, safety measures and risk management. Either adjust your responsibilities, or take your leadership role and re-enforce the well thought thru change management we were so used to.