The fallout when OT companies lack confidence in IT security barriers

There is a reason industries running operational technology lack confidence in the security protocols put in place to protect their systems. Those protocols are untested — they are too new. The recent Colonial Pipeline attack highlights the ongoing security dilemma between IT and OT.

Colonial Pipeline chose to shut down its OT environments following a ransomware attack on its IT systems. The company said the controlled shutdown was a proactive measure to prevent the malware leaping from IT to OT.

The company reportedly shut its pipeline down because its billing system was impacted by the down IT system, according to CNN and cybersecurity publication Zero Day on Substack. With the pipeline running, the company would be unable to automatically determine and process invoices for distributors.

In response to the reporting, the company told Cybersecurity Dive "we proactively took certain systems offline to contain the threat," in reaction to the cyberattack, according to a spokesperson.

While a controlled pipeline closure is preferred over disruption carried out by nefarious groups, the decision is up for debate in the security community. In security, most practitioners err on the side of caution and skepticism as air gaps evaporate between IT and OT.

"It didn’t seem like there was a clear and non-disruptive way of understanding the possible severity of the malware infection without putting all systems into a static state."

Chris Strand

Chief compliance officer of IntSights

"Safely shutting down a pipeline involves a proven set of structured, methodically executed activities," involving people and technology, said Curtis Simpson, CISO of Armis, who has more than 10 years of experience with OT. While relaunching the pipeline was stalled by some IT-related complications, "I fully agree with the decision that was made as it revolved around reducing the significance and duration of impact."

Prior to the ransomware attack, the Biden administration launched its 100-day plan for addressing security issues within OT and industry control systems (ICS) in the energy sector in April. And because of pipeline closure, the government could call for mandatory cybersecurity reporting "into the scope of the [Transportation Security Administration] authority on pipelines, but that would be a considerable lift," said Chris Strand, chief compliance officer of IntSights and energy and gas cybersecurity regulation advisor.

Cyberthreats in oil and gas are expanding beyond facilities, and into the ICS layer of OT. "Individual verticals cannot ignore threats to other ICS entities if they are not specifically targeted because an adversary’s interests and targeting can be highly variable," according to a Dragos report. Companies just don't know what attackers could or will target in an OT environment.

Strand agrees with Colonial's decision to close its pipeline, "to a point," he said. "It didn’t seem like there was a clear and non-disruptive way of understanding the possible severity of the malware infection without putting all systems into a static state," said Strand. "The absence or adequacy of security measures will probably come to the surface during the investigation."

OT industries lack confidence in the security protocols in place to protect OT systems from IT intrusions because their interdependencies are relatively new, no older than the last decade or so.

The only reason Colonial would decide to shut down its pipeline — its livelihood — "is if the attacker gained control of the network and they had no other option," said Eric Cole, founder of cybersecurity consultancy firm Secure Anchor and former member of the Obama administration's Commission on Cybersecurity. An organization would refrain from shutting down their operational network, and instead close up the connection it has to IT.

"In my experience, the last thing Colonial would do is shutdown their pipeline," said Cole. "If what [Colonial is] saying is true, and that the attacker was not in the operation network that runs the pipeline, they could have been up within 24 to 48 hours."

On Thursday, Bloomberg reported Colonial paid hackers $5 million within hours of discovering the attack. While the attackers gave the decryption key to Colonial in exchange for the payment, sources told the news service "the tool was so slow that the company continued using its own backups to help restore the system."

Cole noted the company's slow IT reboot, which made Colonial's decision to pay "the only reasonable action," he said. "While they tried to deny it, to minimize exposure, setting a precedence and potential embarrassment, the truth did come out."

Colonial has not commented on the reporting — companies could face regulatory backlash for paying ransoms — but the company relaunched its entire fuel pipe Thursday, ahead of schedule.

Into the OT

Ransomware is currently an IT issue, focused on computer systems instead of embedded systems like programmable logistic controllers (PLCs), according to Jeff Shearer, member of the SANS Institute ICS team, during the SANS webcast. "There have been embedded system malware, right now ransomware is just getting its day in the sun … eventually there will be ransomware on embedded system devices."

But ransomware operators know targeting critical infrastructure organizations is an almost guaranteed payout because they cannot afford the downtime.

Colonial's downtime cost the company and U.S. economy money — gas prices are up to a national average of $3.03, a first in six years. Higher gas prices can suppress consumer spending, though experts don't expect Colonial's shut down to have long-term impact. Colonial's slow IT reboot was partly to blame in the meantime.

"Often based on past experiences, many OT leaders have been hesitant to partner with IT in support of managing any aspect of their environments due to concerns that IT will severely impact availability or safety," Simpson said.

While investigations are still underway and the ransomware's point of entry is debated, the rest of the industry is anxious to learn more about how IT systems interact with its OT, and how ransomware could weasel its way into more damaging areas.

Curtis Simpson

CISO of Armis

"If you kind of very simplistically consider a bookend, you know, a bookshelf with it on one far end of IT, and OT on the other end, there's a whole bunch of stuff that lives in that in between," said Tim Conway, technical director for the ICS and SCADA programs at SANS Institute, during a SANS webcast Thursday.

Organizations within OT-enabled critical infrastructure have "in-between zones" consisting of business, OT, assets and business intelligence. And then at the bottom of the stack, the organizations have physical technology in the field, including pumping stations or valve controls.

Companies bridge enterprise resource planning (ERP) systems with a manufacturing execution system (MES), which communicates with OT systems to deliver quality assurance information, performance management, material tracking and so forth.

Conway suggested using 2017's NotPetya ransomware attack as an example: Maersk was a collateral victim of the malware, but the shipping company didn't have any issues on the "far end" of the OT side, such as crane control or maritime shipping.

"But if the issue was, they didn't know what was inside the containers, that impacts all of its operations," he said. It would then complicate labeling the NotPetya attack as one on IT or OT.

Maersk reinstalled more than 4,000 servers, 45,000 PCs and 2,500 applications within 10 days of the NotPetya attack. OT and ICS are not as easily ripped out and replaced.

"Operational technology is built to last for decades," said Simpson, resulting in legacy systems riddled throughout critical infrastructure. And because of OT's primary function of availability, systems will always remain slightly behind, according to Cole.

The electricity, oil and gas industries know how to react to storm-related delays; where and when it occurred and how to recover. They have a sense of routine to prepare and anticipate for restoration, said Conway. Cybersecurity incidents disrupt those decades-old, learned expectations, and companies will only improve with more cyber-related exercises and incident response planning.