One year after the disclosure of a critical vulnerability in the Apache Log4j logging utility, the nation’s software supply chain remains under considerable threat as federal authorities and the information security community struggle to transform how it develops, maintains and consumes applications in a more secure fashion. 

The vulnerability, dubbed Log4Shell, allowed unauthenticated and untrained threat actors to gain control over applications using a single line of code. 

Thus far, many of the initial fears of catastrophic cyberattacks have failed to materialize, but federal authorities warn this constitutes a long-term threat that must be carefully monitored and fully remediated to prevent a major security crisis. 

Criminal actors as well as actors linked to some of the nation’s top adversaries — including Russia, Iran, China and North Korea — have used the vulnerability to target U.S. government agencies, critical infrastructure sites and other organizations over the past year. 

The logging utility is found in millions of devices around the world and despite heroic efforts to limit the risk of attack, experts warn the security industry is still in the early stages of a years-long effort to contain the fallout. 

“The popularity of the Log4j logging framework meant that it was, and continues to be, deeply embedded in software composition, and that most organizations across the globe were impacted,” Erik Nost, senior analyst at Forrester said via email.

Ongoing threat

Coupled with the ease and potential impact of an attacker executing remote code against targeted organizations, Log4j marked a significant event for the information security community. 

A report from Arctic Wolf shows a significant percentage of organizations were targeted by threat actors looking to leverage of the vulnerability. The company said 25% of its customer base were targeted by Log4Shell exploitation attempts and 11% of the company’s incident response cases involved Log4Shell exploitation cases as the root point of compromise. 

Threat actors tend to engage in malicious activity based on opportunity, and not necessarily in a way that others would expect, according to Adrian Korn, manager, threat intelligence research at Arctic Wolf Labs. 

“However tooling may impact how threat actors choose potential victims,” Korn said via email. “For example, threat actors use internet search engines to identify potentially vulnerable devices, but each threat actor is likely going to get different results due to varying search parameters.”

The average incident response cost of a Log4Shell compromise was more than $90,000, according to company data. Nearly two-thirds of Log4Shell incident response cases were attributed to three ransomware groups: LockBit comprised 27%, Conti comprised 19% and Alphv/BlackCat 12%.

Mark Cox, VP of security at the Apache Software Foundation, said in the year since the vulnerability was originally disclosed, it has taken a number of steps to learn from the episode: 

• Holding what it calls productive meetings with the White House, members of Congress and other stakeholders to gain a better understanding of the role of open source software. 
• Hiring an engineer dedicated to handling internal security issues in order to provide consistency and speed in how it responds. 
• Engaging in ongoing dialogue with the open source community about how to collectively improve open source security. 
• Assisting the Cyber Safety Review Board with its inaugural report on the Log4j crisis. 

Continued vulnerabilities

Despite extensive efforts to find software vulnerabilities, many organizations remain exposed to risk from Log4Shell. More than 10% of assets were vulnerable to Log4Shell as of December 2021 when the flaw was discovered, including a range of servers, web applications, containers and IoT devices, according to telemetry data from Tenable.

Fast forward to October 2022, Tenable data indicated considerable improvements, with 2.5% of assets remaining vulnerable. However 29% of assets showed recurrences of Log4Shell despite previously achieving full remediation. 

“Remediation is rarely a one-and-done process, especially for a flaw as pervasive as Log4Shell,” Bob Huber, chief security officer at Tenable said via email. “Anytime an organization adds new systems or assets to their environments, they could be inadvertently reintroducing the vulnerability, even after full remediation.”

The Log4j crisis underscores the deep connections between open source and proprietary code, and how deep the stack of dependencies and infrastructure has become, according to Brian Behlendorf, general manager of the Open Source Security Foundation. 

Citing data from Sonatype’s State of the Software Supply Chain report, Behlendorf said the average technology product includes anywhere from 70% to 90% of open source software. 

“Log4j has been around 20 years,” Behlendorf said. “It’s become embedded into nearly every meaningful Java application, and the Log4Shell incident led to compromises in everything from the iTunes store to physical security systems.”

The cost and complexity of the crisis has highlighted the importance of greater transparency and automation in the software community, according to Eric Goldstein, executive assistant director for cybersecurity at the Cybersecurity and Infrastructure Security Agency. 

“A year after Log4j, there are more organizations with greater awareness of their software supply chains, which not only improves security, but can lead to better quality and greater cost savings,” Goldstein said. “A host of new tools, companies and products have emerged over the past year to help better understand software dependencies and Log4j is often used as a primary motivation for innovation and adoption.”