CrowdStrike’s Thursday Goof Creates Chaos with Systems and Social Media

Additional information from CrowdStrike shows that the behavior pattern that was being updated relates to how named pipes are being abused by attackers. A self-remediation process was given out on how to fix the problem (which required direct/physical access to the system) annnnnd that is about all the “factual” information we have on the issue from CrowdStrike. We are now waiting for a real RCA (Root Cause Analysis) from CrowdStrike to explain what actually went wrong.
 
All anyone has available to consume is a “technical” details post which is not very technical and seriously short on detail. Of course, the lack of a fully transparent RCA has given rise to several theories ranging from the technically plausible to the insane. It has also led to some seriously bad marketing decisions on the part of a few vendors. We had vendors attempting to make fun of the incident (like Kaspersky did on Twitter), Cybereason who posted what they claimed was a help line that turned out to be little more than a sales attempt (they have since taken it down), and other companies claiming they would never suffer from this (which is simply not a true statement).

The Vendors trying to take advantage of the situation were bad enough, but in other areas there were some people claiming to be experts who were publishing incredibly wrong takes on the situation and when I say wrong, I mean knowingly wrong, but with enough technical jargon to get people not deeply familiar with the subject of kernel mode drivers, C++ programing, and how to properly perform static analysis (and read the outcomes) to buy into the terrible theory. One of these even tried to spin DEI into it in incredibly bad faith. The same account then doubled down blaming Microsoft in a follow-on post which was even more filled with garbage. The thread had more than 25 million views and inspired more than a few quote posts which then added crazy conspiracy theories that this was intentionally done to delete data on government systems and disrupt other systems at a critical time.
 
These posts are continuing to pop up on different social media platforms from Twitter to LinkedIn to Facebook, and so on. In most cases they end up pointing back to a post from someone claiming expertise on the matter who actually posted a large pile of crap (perhaps knowingly). Causing the flood of weapons grade FUD to stream out to all parts of the internet. I had multiple people reach out to me on Friday and throughout the weekend with questions about the “cyberattack.” I got to spend time explaining that it was not a cyberattack, then explain what it was, then explain that this can happen and has happened to other companies. It was a fun endeavor. Then there were the uninformed takes from people in IT operations who wanted to point fingers at the CrowdStrike customers without understanding that this was an automated push that clients do not control. Then there were the assumptions that no QA was done, without any proof of this claim.  To call the situation sad is an understatement of epic proportions.
 
Now, I want to be clear: I am not a fan of censorship and historically most efforts at combatting mis/disinformation are little more than a censorship body with people getting to choose what is and is not mis/disinformation based on personal leanings. This makes combating bad information around events like this not only difficult, but borderline impossible. Even worse were the media outlets looking to the first to report on the event, often with no information and lots of supposition. These outlets are trusted by others leading to worse information on the event and increasing the chaos. Fortunately, I did see several well-known and trusted people commenting on the event. For the most part these people stuck to the known facts of the event and made sure their opinion and speculation was clearly differentiated from the available facts. These commentators do have a bit of a calming effect on the general market, but not too much in the overall chaos and uncertainty of the event. By now, many will have made up their mind to “understand” the event and make sense of the chaos. Sadly, some of these people may be able to influence decisions on cybersecurity trends (like some on LinkedIn) which will have a negative impact via hesitance in adopting proper protective tools (or staff).
 
So, what have we learned so far about the CrowdStrike event? Well, not a who lot. We know some of the basics of the What happened (a Logic Error in a configuration update file). We do not know the How or the Why yet. We have some communication from CrowdStrike along with promises of an RCA and that they will “do better”. This incomplete communication has been seen as some as exceptional and, sadly, it is when compared to other organizations who have had similar incidents. However, it is not good, and it is barely what we should be able to expect from any company that has had this level event.  I know some will say that the event is not over so a true RCA should not be expected. I would disagree with this though as the issue was identified on Friday and a Fix was pushed out. The teams that are working directly with affected clients are not (or at least should not be) the same ones who will investigate the how and why meaning there is still a lack of transparency that no apology will make up for.
 
CrowdStrike is a good product and is, in general, a good company. However, their lack of providing any real technical detail has helped fuel the general speculation and created an environment where bad and even malicious takes on this can fester (There are a few threat actor campaigns taking advantage of this already). This also erodes general trust in the product (via the lack of transparency). Events suck, there is almost no good in them (unless you include finding ways not to fail). The people that have to deal with the fallout while getting yelled at for answers that they do not have are the ones that take the brunt of this. Meanwhile, I am sure CrowdStrike’s legal team has weight in on how much the company should release on what happened. While this legal interpretation might serve to protect the company, the lawsuits which are sure to follow are going to bring it to light anyway and if CrowdStrike is sitting on any critical details, it is going to have some serious impacts on their reputation and that has a dollar amount attached to it.
 
So, while we might not have really learned what caused the CrowdStrike initiate issue, we have learned that organizations have still not learned what real transparency is. We have also learned that there are some less than ethical companies (or their marketing teams) who are willing to exploit a bad situation for their own gain. This along with the lessons we already know that there are some people on the internet who appear to actually enjoy making fermenting conspiracy theories and making a bad situation just that much worse.
 
If and when CrowdStrike does release a full RCA, I will be reviewing it. I will probably not be the first person to cover it, but I will do my best to make sure what I do report is as complete and as accurate as it can be.