Microsoft IT outage: Here's what happened

A CrowdStrike update on Friday caused computers running the Microsoft Windows software around the world to crash, displaying a blue screen on the devices.

This sudden outage caused companies worldwide, including several news channels, to be unable to reboot their systems.

Immediately after the outage hit, concerned users reported the problem on forums such as Reddit. One user mentioned being stuck in a boot loop and their entire organisation being affected.

So if you encountered this issue when you reached work Friday morning, you weren’t the only one.

What happened to IT systems?

On Friday at 04:09 UTC, as part of ongoing operations, CrowdStrike released a sensor configuration update — an ongoing part of the protection mechanisms of the Falcon platform — to Windows systems.

This configuration update triggered a logic error resulting in a system crash and blue screen (BSOD) on impacted systems.

The sensor configuration update that caused the system crash was remediated at 05:27 UTC.

However, this issue was not the result of or related to a cyberattack.

What was its impact?

Customers running Falcon sensor for Windows version 7.11 and above, that were online between 04:09 UTC and 05:27 UTC on Friday may have been impacted.

Systems running Falcon sensor for Windows 7.11 and above that downloaded the updated configuration from 04:09 UTC to 05:27 UTC – were susceptible to a system crash.

Configuration file primer

The configuration files mentioned above are referred to as “Channel Files” and are part of the behavioural protection mechanisms used by the Falcon sensor.

Updates to Channel Files are a normal part of the sensor’s operation and occur several times a day in response to novel tactics, techniques, and procedures discovered by CrowdStrike. This is not a new process as the architecture has been in place since Falcon’s inception.

Technical details

On Windows systems, Channel Files reside in the following directory:

C:\Windows\System32\drivers\CrowdStrike\

They have a file name that starts with “C-”. Each channel file is assigned a number as a unique identifier. The impacted Channel File in this event is 291 and will have a filename that starts with “C-00000291-” and ends with a .sys extension. Although Channel Files end with the SYS extension, they are not kernel drivers.

Channel File 291 controls how Falcon evaluates named pipe1 execution on Windows systems. Named pipes are used for normal, interprocess or intersystem communication in Windows.

The update that occurred at 04:09 UTC was designed to target newly observed, malicious named pipes being used by common C2 frameworks in cyberattacks. The configuration update triggered a logic error that resulted in an operating system crash.

Channel File 291

CrowdStrike corrected the logic error by updating the content in Channel File 291. No additional changes to Channel File 291 beyond the updated logic will be deployed. Falcon is still evaluating and protecting against the abuse of named pipes.

This is not related to null bytes contained within Channel File 291 or any other Channel File.

Remediation

The most up-to-date remediation recommendations and information can be found on CrowdStrike’s blog or in its Support Portal.

“We understand that some customers may have specific support needs and we ask them to contact us directly,” CrowdStrike stated on its website.

Systems that are not impacted will continue to operate as expected, continue to provide protection, and have no risk of experiencing this event in the future.

Systems running Linux or macOS do not use Channel File 291 and were not impacted.