It’s time to modernize logging requirements for agencies

When the White House released memorandum M-21-31 in August 2021, it marked a turning point for federal agencies by establishing much-needed baseline logging and data preservation requirements. The memorandum created a clear framework for capturing, retaining, and managing data.

While M-21-31’s laudable intention is to improve visibility and empower federal agencies to detect and respond to security incidents more effectively, its methods are increasingly outdated. 

To keep pace with modern cyber threats, M-21-31 must evolve to mandate continuous logging. A revised memorandum should encourage the adoption of open standards collection and open data-storage protocols to ensure the government retains ownership of the logs throughout M-21-31’s long retention period. 

In effect, these changes will empower agencies to proactively and continuously hunt threats, break down data silos and cut costs, regardless of the changing technology that underpins agency enterprises. 

M-21-31 needs an upgrade

M-21-31’s foundational principles were developed during a time when technology evolved far more slowly than it does today. These standards were established on the premise that a system could be thoroughly audited over several weeks and then certified as stable and compliant for an extended period. 

This notion is fundamentally incompatible with current federal IT infrastructure, which operates under continuous integration and continuous delivery (CI/CD) principles — meaning applications are updated, modified and redeployed hundreds of times a day. Trying to apply M-21-31 — a compliance model designed for static systems — to dynamic, cloud-native environments is like trying to police a bustling city with rules designed for a quaint village. 

M-21-31 is not only incompatible with modern IT systems, but ineffective at combating today’s threat actors. Adversaries no longer engage in simple smash-and-grab attacks. Instead, modern threats are defined by low-and-slow operations, multi-stage attacks and destructive payloads. 

The logging model outlined in M-21-31 doesn’t mandate the continuous collection of data, leaving the door open to logging architectures that capture data intermittently. Without continuous evaluation and holistic, long-term held logging of our ever-changing apps, defenders are left with an incomplete understanding of the incident. 

Improving M-21-31 with continuous logging 

First and foremost, to defend against modern cyber threats, the Office of Management and Budget and Cybersecurity and Infrastructure Security Agency should update M-21-31 to mandate continuous data collection.

Continuous logging is the practice of continuously monitoring what data needs to be captured for the system’s current and ever-changing state, capturing and streaming all relevant data from all relevant sources in real time, and holding this data in a way that it can be queried for long durations. This practice is to treat log data as a constant stream of intelligence rather than a static file. With continuous logging, agencies can proactively hunt for threats, rapidly respond to security incidents, and reconstruct an attacker’s entire path, from initial entry to data exfiltration. 

Consider the Log4j vulnerability: the real challenge was not applying a patch but locating the countless undocumented and widely distributed instances of the code embedded across systems. Organizations with continuous monitoring resolved the issue in minutes, while others spent months and vast resources on the search, regardless of their compliance status. Intermittent, or event-tiered logging, is prone to dangerous blind spots that persistent adversaries will exploit. 

By revising the memo’s language to require agencies to continuously collect and stream all log types in real time to a centralized location, agencies can close the gaps that are inherent in intermittent collection methods and create an unbroken chain of evidence for security events. 

Government agencies should also embrace continuous compliance methodologies that leverage real-time data, machine learning and automation to maintain an ongoing security and compliance posture, rather than relying on periodic, point-in-time audits. A robust, continuous compliance methodology enables agencies to maintain a real-time understanding of their compliance posture, identify and address deviations promptly, and demonstrate ongoing adherence without the need for disruptive, lagging audits. 

Open standards offer a dynamic solution

In addition to continuous logging, a revised M-21-31 should encourage open standards for telemetry data collection and transport. In government IT, data silos are a pervasive and serious issue — data collection systems are often disconnected, which increases complexity, hinders interoperability and locks agencies into specific vendors.

Open standards, such as OpenTelemetry (OTel), Auditd, and eBPF, offer a powerful solution and allow for changing enterprise technology ecosystems — evolving with agencies as they modernize their enterprise infrastructure. A unified telemetry strategy, which requires agencies to use standardized telemetry data for both cybersecurity and operational purposes, would enable agencies to maximize the value of collected data, enhance collaboration between security and operations teams and increase overall efficiency. 

By embracing open standards, agencies can collect data once and use it across multiple platforms, resulting in enhanced security, improved performance and deeper operational insights. This approach breaks down traditional barriers between security (SecOps) and operations (DevOps) teams, providing a truly unified view of the entire IT enterprise. 

Moving from reactivity to resilience 

M-21-31 was a crucial first step toward strengthening federal cybersecurity, but federal agencies cannot defend against today’s threats with yesterday’s policies. M-21-31, a framework that assumes systems are static and logs are intermittent, is incompatible with modern security needs. 

To remain effective, M-21-31 must evolve toward a continuous, unified collection of specific, endpoint, network and cloud datasets. By updating the memo to mandate continuous logging and adopt open standards, agencies can transition from a reactive defense to a proactive and resilient security posture. 

This evolution will create a more efficient and secure digital government by preparing our agencies to defend against emerging cyber threats. There’s no time to waste — through a modernized M-21-31, federal leaders can save money, enhance security and future-proof the nation’s digital infrastructure.

Bill Wright is the global head of government affairs at Elastic.