As the U.S. electricity grid gets cleaner, it’s also getting more difficult to manage.

The U.S. had 24 GW of solar and 75 GW of wind at the end of 2015. Both are growing faster than ever. The EIA’s just-released Annual Energy Outlook 2016 forecasts 246 GW of new solar and 149 GW of new wind by 2040, and that’s from an agency that chronically underestimates renewables growth.

The Department of Energy (DOE) SunShot Initiative is pushing the growth along, aiming to cut the cost of solar electricity to $0.06/kWh by 2020, excluding incentives. That would make solar cost-competitive with conventional generation and grow it from today’s 1% of the power mix to “about 14% by 2030 and 27% by 2050,” according to department estimates.

That’s an unprecedented level of variable renewables, and growing portion is expected to come from distributed energy resources (DERs) — assets that sit on the utility’s distribution system.

That can cause big problems for utilities and grid operators alike. While the output of central-station renewables is relatively easy to control, utilities typically cannot control the output of the growing number of rooftop solar systems and other distributed resources on the system, leading to voltage issues and curtailment of other resources, among other issues.

To find innovative solutions to the unique challenges grid operators will face with the proliferation of distributed resources, DOE just opened a competitive funding opportunity for projects that enable utilities to handle 100% penetration of rooftop solar at peak hours.

Through the Enabling Extreme Real-Time Grid Integration of Solar Energy (ENERGISE) program, DOE expects to make ten to fifteen awards totaling $25 million.

The first will go to field-demonstrated solutions that make it possible to manage distributed solar penetrations of 50% of distribution peak load or higher. Those will receive awards between $500,000 and $4 million.

A second round of awards from $500,000 to $2 million will go to large-scale simulated solutions that demonstrate how to manage 100% peak solar penetration on the distribution grid by 2030.

Already, the initiative has attracted praise from some utilities dealing with growing rooftop generation.

“ENERGISE starts at today’s real-time system and asks for innovations for a coming much more complicated real-time system,” said San Diego Gas and Electric (SDG&E) Chief Engineer Tom Bialek. “The systems to be developed will allow managing that more complicated system in a much more streamlined way.”

What DOE wants

DOE is asking for software and hardware solutions “to enable dynamic, automated, and cost-effective management” of the distribution system. The solutions must be “highly scalable, data-driven, and capable of real-time system operation and planning.”

They must also incorporate sensor, communications, and data analytics technologies that will allow grid operator to see, forecast, and optimize DERs performance.

DOE wants ENERGISE proposals to address four specific technical areas of grid modernization.

First, solutions must demonstrate “performance and reliability” of the transmission and distribution grids both safely and cost-effectively, despite the unprecedented penetrations of solar and wind. This is a significant challenge for the distribution system because it is largely not designed for two-way power flows, DOE points out.

Second, the innovations must include “dispatchability” for solar. It must be “available on-demand, when and where it is needed, in the desired quantities, and in a manner that is comparable to or better than conventional power plants.”

Third, the capabilities of “power electronics” such as advanced smart inverters and other intelligent devices should be used in the proposals to maximize solar output and interface with the grid’s systems. It must be done cost-effectively without compromising “performance, safety, reliability, and controllability.” What DOE is seeking is management of net load as the transmission system and the distribution system are integrated, department materials explain.

Finally, ENERGISE places a big emphasis on scalable “communications, sensing, and data analytics” technologies and infrastructure to monitor and control the “millions of nodes” on a system with a high DER penetration.

The utility perspective

All projects targeting the 2020 objectives must have utility partners for the field demonstrations, said Lidija Sekaric, director of the SunShot Initiative.

“Without utility participation, these solutions will not work. They need to be field-tested in a live environment,” she said.

SDG&E is considering applying for a grant, Bialek said. Any utility that accepts the ENERGISE future scenarios has likely been thinking about solutions, their cost, about private sector vendors, and about starting work soon.

“In that context the ENERGISE funding would be an appealing opportunity,” he said.

High penetrations of DERs introduce challenges and complexity, and utility capabilities are advancing to accommodate the challenges and understand the system’s behavior, Bialek said.

“You don’t know until you measure,” he said. “When you measure, you have data. With data, you can understand the system performance. Data is a critical enabler of resolving some of the challenges.”

Data can show how non-traditional tools like smart inverters, energy storage, and dynamic reactive power sources match up against traditional tools like larger conductors, modified circuit impedances, and new transformers.

“The next question is how to integrate that information and develop planning tools that show what the impact on the system will be and what the available solutions are and how they can be deployed and controlled,” he said.

With the kind of data ENERGISE is asking applicants to deliver, the utility can address the question of which technologies and solutions can integrate and control the coming high DER penetrations, he believes.

“SDG&E is at over 89,000 rooftop solar systems representing about 582 MW, which is comparable to a power plant,” Bialek said. “ENERGISE asks what the impacts will be on the transmission system, the distribution system, the interaction between them, and how that evolves if the next 582 MW comes even faster.”

The crucial importance of data

The capability to collect data from both the bulk transmission system and the distribution system is growing rapidly and it is supplying a range of new data streams from SCADA to smart meters, said Sila Kiliccote, staff scientist at the Stanford-SLAC National Accelerator Laboratory (SLAC).

But even so, “the data is not being used holistically,” Kiliccote said.

A real world example is the Southern California Edison region that lost supply when the San Onofre Nuclear Generation Station was shuttered in 2012.

“There are transmission-constrained areas and resource constrained areas,” Kiliccote said. “We need to use new sensor technologies and existing and additional data streams to understand how to meet those needs and how to value the solutions. ENERGISE is a right fit to do that.”

Kilicote’s research is on the use of data in planning. “ENERGISE is about adopting new technologies and methods for both planning and operations,” she said.

“We are so used to utilities having siloed approaches. For example, the people who run demand response programs do not necessarily interact with those who do planning and operations,” Kiliccote said. “We don’t know the cost of our actions. If utilities had more sensors on their networks and combined these data streams and looked at them more holistically instead of in their silos, they would better understand the consequences of their planning and operations decisions.”

The same opportunities exist in markets, she said. There are new products to manage peak demand, new DER products, and new ramping and flexibility products. They are aimed at managing the system, she added.

“But we don’t know how to value these services for the distribution network,” Kiliccote said. “Technologies that introduce variability and two-way power flows may cause problems if uncontrolled. But if they are controllable, they could solve locational challenges or broader problems for the distribution system.”

The ENERGISE field demonstrations will be an important first step to increase visibility, integrate systems, and operate them at a higher efficiency and reliability, she said. That data will reveal some technical and locational value. But it will only begin to answer the market questions.

“We can measure and quantify some of the impacts but the market value also has to be explored,” Kiliccote said. “A great solution for a $2 problem could cost $1 million and that is not a good solution. That is what understanding value is about. Then we can develop solutions that meet the value of the problem.”

The transparent data that emerges from ENERGISE projects could also provide solutions to two current data-sharing challenges, Kiliccote thinks. First, it could offer the possibility of some coordination between federally-regulated systems and state-regulated systems. That could lead to better-informed regulatory decisions.

Second, it could help resolve barriers to system interoperability, she said. “We need to think about data-sharing and interoperability among the systems using it.”

The power system has already begun learning what is possible with big data from other fields, Kiliccote said. “ENERGISE can be a great opportunity to expand on that but both utility and vendor engagement is critical.”

The grid as ‘one machine’

ENERGISE moves toward analyzing and using the grid “as one machine, which it is now because of distributed energy resources,” said Carol Stimmel, founder and chief analyst at the advisory firm Manifest Mind. “The utility challenge is to understand the grid as a single unit.”

The traditional utility was a very successful project management company that generated electricity and pushed it out, she explained. “Now it must deal with variability and two-way electricity flows. And technology.”

A simple example is in evaluating the cost of a utility-scale solar project. It can appear to be a lower-priced option than distributed generation sources. But it could add stress to the existing system, Stimmel said. “The cost of the utility-scale solar installation must also include control room analytics, situational awareness, the cost of inverters, and many other things. The need is to think bigger.”

Another potential benefit of ENERGISE is that it offers a way past the tension between the need for pilot projects and the way pilots can obstruct progress, Stimmel added.

“Stakeholders don’t always seem to fully comprehend the utilities’ daily challenge to keep the grid running 24/7,” she said. “Utilities can’t just let new technologies go out on the grid and start running. An incremental approach through trials and pilots is necessary.”

But treating new technologies as perpetual science projects is a way to kill them, Stimmel added. “ENERGISE is the kind of opportunity we need to help utilities move from perpetual science projects to scale deployments in a timely manner.”