The attempted supply chain attack against XZ Utils is raising troubling questions about the motivations of the suspected threat actor behind the incident as well as the overall security of the larger open source ecosystem.
A Microsoft engineer accidentally found obfuscated malicious code installed in the xz library, which could lead to a major supply chain compromise.
Security researchers and other industry experts are pointing to the suspicion that a longtime contributor is behind what is now considered a multiyear effort to establish themselves as an insider, leading up to the attempted supply chain attack.
XZ Utils, a data compression software utility found in most Linux distributions, has long been considered a widely trusted project, according to researchers.
“The most unique and unsettling aspect of this attack is the significant effort and investment made by the attacker in gradually establishing themselves over several years as a credible open-source contributor and carefully advancing their position until they gained trust and the opportunity to maintain and add malicious code into a widely used package,” Jonathan Sar Shalom, director of threat research at JFrog, said via email.
Researchers point to a Github account @JiaT75, which has since been suspended, as the suspected original source of the backdoor.
GitHub confirmed that it “suspended user accounts and removed the content” in keeping with its acceptable use policies, however after an investigation the account belonging to @Larhzu was reinstated.
The @Larhzu account is linked to Lassie Collin, the original and legitimate maintainer of the site.
What followed was a multiyear effort to gain trust within the community, while at the same time allegedly testing the waters by making subtle changes that failed to raise any immediate alarm bells.
“Now when we look back at the tale of the tape, what we see is Jia kind of surreptitiously inserted all these little changes over time,” Omkhar Arasaratnam, general manager at the Open Source Security Foundation, said in an interview. “None of them catastrophic, none of them very flashy. But you know, just to see if people were watching.”
Maintainers in focus
The open source community has seen previous cases of maintainers throwing tantrums or using the community as a platform to protest larger issues. But the patience and sophistication of this attack is raising questions for an increasing pool of experts about whether nation-state support is a factor.
“Our analysis suggests that the sophistication and operational security observed in this incident, including the strategic use of email addresses and IP addresses, point to a highly trained and sophisticated adversary,” said Brian Fox, co-founder and CTO of Sonatype, a supply chain management platform. “The lack of tangible evidence of the threat actor’s existence beyond their precise and limited engagements further distinguishes this from the actions of a rogue open source contributor.”
Red Hat on Friday warned that malicious code was present in the latest versions of xz tools and libraries. The vulnerability was assigned CVE-2024-3094 with a CVSS score of 10.
Users were urged to immediately stop using Fedora Rawhide instances for work or personal use and the Cybersecurity and Infrastructure Security Agency warned developers and users to downgrade to an uncompromised version.
Andres Freund, a principal software engineer at Microsoft, stumbled upon some anomalous activity last week and publicly disclosed the incident. Freund observed sshd processes using an unusual amount of CPU, however noted that the wrong usernames had been applied.
“Recalled that I had seen an odd valgrind complaint in automated testing of postgres, a few weeks earlier after package updates,” Freund said in a post on Mastodon.
Microsoft confirmed his role in discovering the attack and released guidance on how to respond, with a list of impacted Linux distributions.
Jake Williams, a faculty member at IANS Research, said the incident highlights the need for defense in depth, including the need to have properly staffed vulnerability intelligence teams and proper investments in tooling.
“Organizations with strict firewall rules preventing access to their SSH servers limited exploitation opportunities, even for vulnerable deployments,” Williams said via email. “Some [cloud security posture management systems] had scans for vulnerable instances released the same day this was detected.”
