Stakeholders in utility policy proceedings across the country are learning that sometimes you don’t know what works until you try it.

That’s why the trial, pilot, and demonstration projects increasingly being ordered by electric utility regulators may be a way to resolve stakeholder debates.

They offer real world experience, former Xcel executive Mike Bull, policy director for Minnesota’s Center for Energy and Environment, told me. Leaders in the electric utility industry who try to manage the “significant change” within the power system by “‘doing what they have always done’” will fail customers, fail utilities’ financial interests and fail the public interest. 

“The only way to adapt to significant change is through innovation,” Bull said. “Trials, pilots, and demo projects are ways to test new things and keep the cost and risk of innovation low.”

A recently-released Rocky Mountain Institute (RMI) paper on best practices defines pilots as tests of “technical feasibility” and demonstrations as tests of “business models, customer adoption, and other elements.”

Some utilities have slightly different definitions but I will use the term “pilot” in a broad sense to stand for “pilots, demonstrations, and trials” that test innovations of any kind. Using “pilot” is easier than inventing a portmanteau word like “pilodemos.”

An emerging trend

RMI’s paper predicted that today's pilots“will test utilities’ ability to meaningfully advance a new set of solutions."

Using pilots to resolve regulatory proceeding differences is becoming an emerging trend, according to Autumn Proudlove, manager of policy research for the North Carolina Clean Energy Technology Center. At least nine new pilot program actions were brought to regulators in the first half of this year within solar policy proceedings. 

They are "being used mainly to test time-varying rates, residential demand charges, and community solar,” Proudlove said. “Many are related to high-profile proceedings, so the impact is significant."

These were reflected in grid modernization policy actions; at least 181 took place in the second quarter of this year. Many of these actions focused on to "information- and data-gathering on pilot deployments intended to detail the impacts of new technologies,” Proudlove pointed out. They represent a shift away from the historical dependence on utilities to provide data, she added.

By the way, I recommend the Clean Energy Technology Center’s policy quarterlies.

The trend was also verified in responses to Utility Dive’s 2017 State of the Electric Utility survey. Utilities are more likely to turn to pilots for emerging distributed energy resources (DER) than integrate them before testing into their core operations, the survey found.

A good idea?

It seemed self-evident that the trend toward pilots was because contentions in regulatory proceedings are better resolved with data from those experiments. But reporters know that even the self-evident should be questioned. So I asked some people who have had experience with pilots.

The first thing I found out is that, though a popular country song promises “life’s a dance, you learn as you go,” it’s not that easy for utilities.

Private sector technology providers may have deep enough pockets to learn by trial and error. But regulators and utilities oversee life-sustaining services and answer to irate customers and their spotlight-stealing elected representatives for every shortcoming. They therefore have to learn carefully.

As Arizona Public Service (APS) State Policy Director Greg Bernosky said, the grid is not “a plug and play tool that anything can be tested on. It relies on so many interdependent parts that any compromise of it has real impacts.”

But in pilots, emerging technologies can be tested while reliability and operations are protected, he said. “We can walk before we run.”

Former Hawaii Public Utilities Commission Chair Mina Morita said, “You don't want to make a full commitment to new technologies or rate designs without fully understanding the impacts to the whole ecosystem.”

The commission during Morita's tenure blocked projects “that were not justified,” but also encouraged some. The regulatory process does not have to be an impediment to innovation if the intent of the utility and its stakeholder partners is clear to regulators, Morita said.  Making sure of that is “incumbent on the utility.”

Darren Ishimura, Hawaiian Electric Company (HECO) manager of grid technologies said piloting allows the utility “to evaluate and understand the performance and economics of new technologies and applications before making larger investments.”

It also gives HECO the opportunity to collaborate with the private sector, which allows it to leverage outside technical expertise and funding, he added. And the utility and its regulators have agreed that pilots are a way minimize risk by pacing technology deployment.

Last year at this conference, I heralded Colorado’s settlement between Xcel Energy and solar advocates as exemplary. Its success was based, in part, on agreements about rate design pilot and trial projects.

I checked with Western Resource Associates attorney Erin Overturf, who participated in the settlement, to see how implementation is going. She said the stakeholder groups are working together and conversations are expanding.

“We are getting really good at talking to each other,” she told me. “We have developed an increased level of respect about where everybody is coming from.”

More recently, a general rate case settlement quieted the long-standing belligerence between Arizona solar stakeholders and APS. It includes rate design options for rooftop solar and non-rooftop solar customers, such as time of use (TOU) rates and demand charges. 

The utility is now preparing the pilot, APS’s Bernosky told me. But tests of technology present a different challenge, he said.

“The regulatory compact requires utilities to put technology into service, prove its use and usefulness, and request cost recovery,” he said. “But not every technology will prove itself. That can be a barrier to innovation because utilities have to protect shareholders and ratepayers by being risk averse.”

Pilots are a way past that barrier, Bernosky said. Several veterans of regulatory compact debates urged me to remember the pilot's design is also crucial.

Ned Ross, Director of Corporate and Regulatory Affairs at Direct Energy, said he has seen “pilots that took to the air and never came down.” A pilot can be a “thoughtful and measurable way to apply different products and rates,” he said. But too often they impose costs on customers “that continue in perpetuity.”

Former Texas Public Utilities commissioner Karl Rabago, now executive director of the Pace Law School Energy and Climate Center, is also a skeptic. Pilots can be “tools of innovation” but are too often used “to forestall the inevitable.”

It is not clear that pilots lead to “full-fledged products and services" in the most efficient time frame, he said.  But he told me that he “advocated for them just to make some movement happen."

A pilot is only as good as the people designing and implementing it, he added. Too many are designed poorly by utilities without meaningful stakeholder input and then watered down by “risk-averse, inexperienced regulators.”

For example, there were “many green pricing pilots where the biggest lesson was that the scale of a pilot was too small to overcome administrative costs," he said.  Pilots can play a role in settling debates between parties in regulatory proceedings, he acknowledged. “But that's not innovation, that's kicking the can down the road.”

Or, as Mark Twain put it, “noise proves nothing. Often a hen who has merely laid an egg cackles as if she laid an asteroid.”

Are pilots a bad idea, then?

Does that mean pilots are a bad idea?

Former Nevada Commissioner Rose McKinney-James said the structure of the pilot is crucial, and well-structured pilots provide the “ability to collect data and determine the extent to which external factors may impact outcomes.”

She recalled a green tariff pilot that revealed, to the benefit of both regulators and the utility, that there were fewer takers than the initial customer survey suggested because of the price premium. It confirmed  “that cost was still the most significant barrier," she said. 

Mike Bull described Xcel Energy’s 2016 Belle Plaine project, which the utility designed to get experience using solar-plus-storage to defer distribution system investments. Private sector vendors opposed it because they were not included in Xcel’s planning process and Minnesota regulators did not approve it.

A 2016 grid modernization study from SolarCity, before it merged with Tesla, argued on behalf of Belle Plaine types of investment and Policy and Energy Markets VP Becky Stanfield applauded Xcel. “It absolutely is important that Xcel is looking at this idea because step one is not investing in traditional wires-based solutions and looking at alternatives,” she told me.

Surprisingly, though, Stanfield’s proceeding filing recommended the Commission reject Xcel’s project. Because step two, Stanfield said, “is can they get even more savings by opening the market to private parties to bid in to solve the problem even more cost-effectively?”

The coalition of private providers represented by Stanfield advocated for an open bidding process in which third parties can compete for ownership and operation rights of projects to defer Xcel infrastructure investments. 

Xcel stood staunchly with the classic utility point of view, best expressed by John Glenn when he became the first U.S. astronaut to orbit the earth. “Through space, one thought kept crossing my mind,” Glenn said. “Every part of this rocket was supplied by the lowest bidder.”

Stanfield’s private sector coalition argued “innovation can be more rapid and diverse in a competitive market.” A competitive bidding process, their filing added, “could result in a broader set of options, including technologies and ownership models, beyond the limited scenarios Xcel considered.”

HECO, PSEG Long Island and other utilities have successfully used that kind of competitive bidding open to the private sector, the filing argued.

If private sector vendors "cannot put forward cost-effective options, Xcel’s utility ownership model would be an acceptable approach,” Stanfield told me. But Xcel’s proposal, by not giving private providers an opportunity, limits learning, she added.

The takeaway from the Belle Plaine debate, Bull told me, is “utilities need to work with their stakeholders” because it makes for better proposals and for stakeholder support.

Xcel learned from its defeat, Bull said. The utility has engaged a Minnesota think tank to lead a collaborative stakeholder process to develop a TOU rate design pilot. The final design will be Xcel’s, he stressed. But the utility will know what its stakeholders want, making innovation easier and ensure regulatory approval. 

Former California Commissioner Mike Florio expanded on an observation made by several people now involved in shaping pilots. “The regulatory process has become too litigious for finding common ground,” he said. “Different ways of looking at the world can increase distrust and lead to unproductive arguing in a hearing room.”

But small groups of stakeholders, working together to design a pilot project, can learn to appreciate others’ points of view and address those concerns, Florio told me.

Data

Several veterans of regulatory proceedings stressed the potential value of data in both small discussions and formal hearings.

APS’s Bernosky said he had seen data from utility pilots shared with diverse working groups help settle debates. “Demonstrative analysis from pilots narrows in on the pros and cons and takes some of the emotion out of the debate by focusing on tangible, measurable facts,” he told me.

Hawaii’s Morita agreed. Data from reliable sources are a regulator’s “most unbiased resource” and can be key to “informed decisions,” she said.

Rabago, whose take on pilots is double-edged, simply said “Get lots of data and make it public.” 

McKinney-James agreed. It is important to have “independent and credible parties” participate in gathering and analysis of the data to ensure objectivity, she said.

Former Nevada Commissioner Rebecca Wagner expanded on the point. “Reasonable minds can disagree on what data means,” she said. The value of a pilot depends on how transparently the data is gathered and how effectively it is structured to control for regulators’ biases. Unfortunately, that is hard to achieve.”

She also described pilots as a way for regulators and lawmakers to “’try on’ new policies without fully committing.”

Nevada’s solar incentive program started as a demonstration, underwent “a lot of attention and tinkering” and was “bogged down” by the regulatory process before it went into effect, she said. But it identified the volume and types of customers likely to be interested.

Best practices

For the past few decades, we have been running the biggest demonstration project ever on the earth’s climate. With Harvey and Irma, everybody can see the data is in. That makes it all the more important to use best practices when running pilots on technologies that will help address climate change.

RMI partnered with Consolidated Edison (ConEd), Avista Utilities, and APS on the best practices paper I mentioned earlier. ConEd Director Jamie Brennan is confident well-designed pilot projects can help resolve policy disputes.

“It is always easier to have a debate about real data,” he said. “The debate is less ideological, more analytical, and enables stakeholder engagement that might not otherwise be likely.”

RMI argued that cost-effective, collaborative pilots can “benefit customers, utilities, and the environment.” But poorly designed pilots, the think tank paper cautioned, will be “bogged down by contentious disputes between utilities and technology providers.”

RMI’s paper offered five best-practice recommendations.

The first is to do planning with specified strategic goals. “Embrace a strategy for energy system transformation and craft a complementary roadmap for innovation,” the paper recommended.

Utility pilots should be designed to achieve goals on the roadmap that regulators can support, RMI added. Private DER providers can then structure proposals that meet these strategic goals. This approach leaves the utility open to any technology from any private sector provider that can achieve the goals.

The second recommendation is to define which pilots are about technical feasibility and which can be scaled. If the design does not include how to evaluate a pilot’s performance and scale its success, “it can lead to an ambiguous state where the utility does not act on the results,” RMI paper co-author Mike Henchen told me. The result is a lost opportunity to innovate.

A longstanding obstacle to innovation is silos within utilities. To take a pilot to scale, RMI recommended clearly designated leadership that is accountable, but also can call on the utility’s wide-ranging resources.

The failure of those spearheading a successful pilot to hand it off for wider implementation “may be the most common problem with pilots,” Henchen said. “Few organizations have any formalized process to advance from one step to the next, leaving even successful pilots neglected.”

Another recommendation was to identify early on common ground for utilities, technology providers, regulators, customers and advocates. It can, RMI said, avoid “contentious and unproductive disputes."

Regulators should press for this and can impose requirements for competitive solicitations that open collaboration between utilities and technology providers, RMI recommended. But utility requests for proposals should not be so prescriptive they “limit the solution space.”  

The Xcel Energy Belle Plaine solar-plus-storage pilot I talked about earlier failed, in part, because it was designed “inside the walls of the utility,” Henchen said.

Another example was the electric vehicle (EV) charging station installation pilot that California regulators twice forced Pacific Gas & Electric to revise. Private sector charger manufacturers convinced the commission that pilots should give them the opportunity to show they can do charger installations more cost-effectively.

Avista Utilities won approval from its regulators for a $3 million install-own-operate EV charger pilot. It worked with charging station software provider Greenlots and a range of charger vendors to integrate their technologies into its system.

Greenlots CEO Brett Hauser said utilities “can be proactive or reactive. With this pilot, Avista and its commission were proactive.”

Mike Bull said proactive stakeholder engagement “helps the utility and its stakeholders understand each other.” That “critical first step” can narrow areas of disagreement and create a shared platform on which to settle debates during evaluation of the pilot, he added.

RMI paper co-author Leia Guccione pointed out there are ways private sector vendors also need to step up. One is facing at least some of a pilot’s financial risk. “It frustrates utilities when vendors offer only a new widget and not the business model innovation or capital to make it work,” she told me.

Finally, utilities can accelerate innovation by asking regulators for opportunities to collaborate with other utilities, RMI recommended. “Utilities could build on each other’s learning to skip steps and advance more rapidly toward the roll out of scale programs," Henchen said.

California’s Florio pointed out one other potentially disruptive – almost subversive – advantage in pilots. “A pilot is an internal demonstration project that allows the designer of it to be an innovator,” he said. Utilities have a general reputation for not being innovative but “there are people within the companies who are.”

Inside any utility are competing voices trying to get recognition for their ideas from the higher-ups, he said. “If they do, it elevates their profile. We want to encourage that and help advance the careers of those people who are willing to think differently because it benefits everyone.”

Or, as Mike Bull put it, innovation breeds innovation.

Anybody who saw the pictures of Texas and Florida knows innovation is long overdue. Some even say it is too late. But, as Abraham Lincoln said, “the probability that we may fail in the struggle ought not to deter us from the support of a cause we believe to be just.”

For that reason alone, innovation is worth a try. We might like it.