In December 2015, grid control center operators in Ukraine watched helplessly as their cursors moved across their computer displays, clicking substations offline. They frantically struggled to retake control until being involuntarily logged out.

Before the cyber attack ended, three stations were seized and over 225,000 Ukrainian electricity customers lost power. The hackers also shut down the three centers’ uninterruptible power supplies (UPS), leaving the operators themselves in the dark, according to a detailed study of the incident by the Electricity Information Sharing and Analysis Center (E-ISAC).

“The remote cyber attacks directed against Ukraine’s electricity infrastructure were bold and successful,” the E-ISAC, co-founded by the federal government and electric utility industry to analyze cyber threats, concluded. It was “the first time the world has seen this type of attack against OT systems in a nation’s critical infrastructure.”

That hack, along with subsequent (and quickly debunked) headlines of a cyber attack on a Vermont utility, appears to have increased utility concern about the security of their power systems.

In Utility Dive’s fourth annual State of the Electric Utility Survey, more than 600 utility professionals named cyber and physical security the most pressing concern for their companies, with 72% saying it is either “important” or “very important” today.

Whereas security issues ranked sixth among utility concerns in the 2015 and 2016 surveys, they registered above a number of high-profile sector issues this year, including distributed energy policy and rate design.

Doug Westlund, senior vice president at electric utility consultant AESI believes increased attention to the cyber threat from the media and industry groups set the security issue apart from other concerns.

“The Ukraine attack was well documented and it was well understood the same attack could happen in North America,” he told Utility Dive.  “And utilities are starting to share information on cyber-attacks and threats.”

Westlund and others say the attack continues to animate utility sector planning for enhanced security. But as new distributed resources and grid technologies add more complexity to the system, keeping security practices up to date is likely to be a continuous job.

The security threat

The Utility Dive survey found the five most important issues facing utilities in 2017 are physical and cyber security, DER policy, rate design reform, aging grid infrastructure, and the threat to reliability from integrating variable renewables and DERs.

It is likely the Russia-based hacker Fancy Bear was involved in both the attack on the Ukrainian grid and the hacking of Hillary Clinton and other Democratic Party officials during the presidential election, according to the MIT Technology Review. The proximity and threat of these incidents has likely “increased focus on grid security,” the Utility Dive survey observes.

The physical and cyber security concern is a national trend, the survey reports, with a majority of respondents from every region in the United States indicating it is “important” or “very important” today. It is a top concern at all types of utilities, with 73% of those at IOUs, 72% of those at munis, and 64% of those at co-ops saying it is “important” or “very important.”

The survey results indicate utility professionals across the sector have taken notice of a variety of security reports and warnings over the past year.

According to the E-ISAC report, the Ukraine attack was “specific to Ukrainian infrastructure” and the strategies observed are “employable in infrastructures around the world.”

Along with E-ISAC, the Department of Energy’s Idaho National Laboratory (INL) called for new awareness about grid security in its August 2016 report, “Cyber Threat and Vulnerability Analysis of the U.S. Electric Sector.”

“The potential for malicious actors to access and adversely affect physical electricity assets of U.S. electricity generation, transmission, or distribution systems via cyber means is a primary concern for utilities,” the researchers wrote.

At the report’s writing, there had been no successful cyber attacks against U.S. utilities that caused permanent or long term damage to power system operations. But, researchers cautioned, there has been “a steady rise in cyber and physical security related events.”

The good news is that grid operators recognize the threat and their reliability practices have so far kept the power system “secure and up to date.”

The new need, researchers wrote, is to meet “a lack of knowledge or strategy to mitigate new risks that emerge as a result of an exponential rise in complexity of modern control systems.”

MIT’s Cybersecurity White Paper focused on similar concerns coming from changes in the grid’s power mix. Drivers are the growth of variable renewables, the broader fight against climate change, and “the increasing interconnectedness of electricity grids and other critical infrastructure.”

In particular, higher penetrations of DER “will increase digital complexity and attack surfaces, and therefore require more intensive cybersecurity protection,” the paper argued. A new kind of cybersecurity is now needed that will allow system operators “to operate, maintain, and recover a system that will never be fully protected.”
 

Sector security lessons

Many of the protocols now being considered to bolster utility cyber defenses were learned from assessments of the Ukraine attack, Scott I. Aaronson, security director for the Edison Electric Institute, recently testified to a U.S. House of Representatives subcommittee.

The E-ISAC assessment of the Ukrainian incident described five general lessons learned. The first is that the cyber attack was preceded by “intrusions” months earlier.

“In a prolonged attack campaign, there are likely numerous opportunities to detect and defend the targeted system,” the assessment reported.

Second, the highly coordinated cyber attacks came minutes apart. “Important opportunities for defenders to disrupt the adversary’s sequence of events were identified.”

Third, future protections should not be focused on the “BlackEnergy 3” and “KillDisk” malware that was used in the Ukraine. “This attack could have been enabled by a variety of approaches to gain access and utilize existing assets within a target environment.”

Fourth, the attack worked across multiple programs, Windows‐based operating system workstations, and servers. Those capabilities “provide specific lessons learned for defenders to take action on.”

Finally, “information sharing is key in the identification of a coordinated attack and directing appropriate response actions.”

AESI’s Westlund said these are precisely the efforts now being put in place by U.S. government-utility industry partnerships. Increased attention is being brought to the issue by industry groups and trade associations like EEI, the American Public Power Association and the National Rural Electric Cooperative Association.

“They have been raising the visibility of this issue for many years,” Westlund said. As a result, “utilities are conducting risk assessments, vulnerability assessments, and penetration tests to better understand their risks.”

Some utilities are working in-house and some are working with consultants. Many are beginning to develop formal cyber security programs. A National Institute of Standards and Technology (NIST)-developed Framework for Improving Critical Infrastructure Cybersecurity “is gaining great momentum in the industry for distribution utilities,” Westlund said.

NERC’s Critical Infrastructure Protection (CIP) Framework is preferable for the bulk electric system, he added.

The next crucial steps should begin with utilities revising their perspective and beginning to see cyber security “as a risk management issue and not an IT issue,” he said. Using that perspective, the utility should train and engage its executive team and Board of Directors and put the NIST or NERC framework in place.

With those things in place, the utility should begin developing a three year or longer roadmap to its cyber security program. “A cyber security program is not a ‘one and done’ type of initiative. It is a long term process,” Westlund said.

Perhaps even more importantly, he added, utilities are now moving past their longstanding reluctance to discuss this subject and realizing there is greater protection through information sharing and collaboration.

Information sharing, cooperation vital

In their testimonies to Congress, both Aaronson and Arkansas Electric Cooperative Corporation (AECC) President/CEO Duane D. Highley stressed the importance of information sharing through public-private partnership.

The Federal Power Act assigns oversight to the Federal Energy Regulatory Commission (FERC). The NERC sets CIP reliability standards that include security requirements. System operators that fall short of the NERC standards face penalties that can be over $1 million per violation per day.

FERC recently approved seven updated CIP reliability standards derived from NERC rules. It also imposed new requests on NERC to advance cyber protections further.

Both Highley and Aaronson also stressed the role of the Electricity Subsector Coordinating Council (ESCC) in connecting the federal government and the electric power sector on preparations for and responses to national-level incidents or threats.

The ESCC works with E-ISAC which, in addition to its reporting duties, manages the Cyber security Risk Information Sharing Program (CRISP), a public-private partnership co-funded by DOE and the electric industry. CRISP handles sharing of actionable real-time threat information. Utilities that use it serve over 75% of all U.S. electricity customers.

Both described the work of the Cyber security Capability Maturity Model (C2M2), a public-private partnership pushing for adoption of the NIST Framework.

“Regulations and standards provide a solid foundation,” EEI's Aaronson said. But “they alone are insufficient. As the threat environment evolves, so must the industry’s security efforts.”

Man-made events and natural phenomena “require coordination between government and industry, as well as across the critical infrastructure sectors,” he added.

An example was a briefing, following the Ukraine incident, by DOE and Department of Homeland Security (DHS) officials to the ESCC and others in the electric sector. It provided new information about Russian cyber incidents against U.S. private-sector critical infrastructure.

Because of this intelligence sharing, the power industry realized the urgency of the threat and took “immediate steps to review and secure their systems,” Aaronson said.

The most important step was the formation of ESCC’s cyber mutual assistance program, Aaronson told Utility Dive.

Driven by their sense of urgency, U.S. utilities put a program in place that was able to respond to the Mirai botnet cyber attack on internet service provider Dyn less than a year later.

“It shows what can happen and how fast it can happen when there is a sense of urgency,” Aaronson said.

Finally, both Highley and Aaronson highlighted the importance of past legislation that established DOE jurisdiction on security issues and pending legislation that reduce utility liability in the event of a cyber attack.

“We didn’t originally design the electric grid to defend against intentional physical or cyber attacks nor acts of war,” Highley told the House committee. But to protect against extreme weather events, vandalism and major equipment failure, “a high level of redundancy” is built into a system that is continuously monitored. These redudancy systems, he said, “will continue to be our first and best defense.”