These are uncertain times for competitive electricity markets. Cheap natural gas, zero marginal cost renewables, and stalled demand growth have deflated energy prices. Merchant generators are getting subsidies to survive and states that were once pioneers of deregulation are getting cold feet.

At this important inflection point for our energy future, regulators need to take a step back and look at the full picture. With electricity markets struggling, time is running out to give demand participation an equal role in competition. Competitive retailing was supposed to accomplish this, however the results have been disappointing.

Will the genie of deregulation go back in the bottle? How can we get retail competition right? What is the role of regulators?

The trend towards greater regulation is becoming clear with rollbacks of competitive retail. Britain, where retail competition started strong in the late 1990s, has imposed restrictions on customer offerings and limitations on door-step selling. In the midst of regulatory reform, New York has ordered competitive energy providers to cap prices at the incumbent utility rates. Australia, where all retail price controls were removed in 2009 with good results, is scapegoating retail competition for increases in regulated network tariffs and wholesale energy prices.

What went wrong?

Competitive retailing has been an important feature of deregulated markets since they were introduced decades ago. The 1994 California decision to initiate electricity restructuring stated that "in the absence of well understood and easily exercised consumer options, the genius of competition is thwarted." Opening energy services to free market investment by empowering consumers to choose among different innovative offerings maintains an important balance with merchant generation, where well-established incumbent companies have dominated stakeholder-driven decisions in order to lock in profitability.

Retail competition in electricity markets has challenges and benefits that are useful to understand for decisionmakers.

Challenges of retail competition:

• Increased transaction costs have been observed in competitive retail, at least in the early stages. Compared with a single incumbent provider, marketing costs increase when multiple retailers compete for the same customers.
• Limited customer appetite is particularly true with smaller customers, including residential and small commercial, for whom the time and effort required to shop often exceed the expected benefit of finding a better provider.
• Abusive practices have been a problem. Some retailers cherry pick their customers to maximize their profits and the sophistication of providers relative to consumers enables bad actors to engage in predatory or fraudulent practices.
• Market power is a concern despite generally low market concentration. Risk of market power abuse is pronounced by the presence of “gentailers” that control a significant share (i.e. 10%+) of both generation capacity and customer load.

Benefits of retail competition:

• Downward rate pressure results from increased rivalry and new entrants. Although overhead costs such as marketing may increase in the early stages of competition, over time competitive pressure pushes rates down.
• Better options for consumers result from the need to grow and maintain market share to survive. Winning in the market requires retailers to provide products and services that consumers really want while remaining competitive.
• Responsive demand-side from retailer offerings improves system reliability, mitigates market power, and promotes efficient market outcomes. Retailers and consumer advocates balance the strength of merchant generators.
• Active risk management is taken on by market participants rather than risks being regulated across customers. Sophisticated risk hedging strategies enable retailers to manage volatile and uncertain market signals.

Unfortunately, the results of deregulation and competitive retailing have been underwhelming relative to their enormous potential. If the purpose of retail competition is to unshackle the way that energy services are rendered and consumed, then the current state of restructuring has failed to deliver radical innovation commensurate with the expanding array of enabling technologies.

For example, the availability of value-added services has been limited and demand-side participation continues to rely on supply-side mechanisms. As the power sector pivots from a century of nationwide asset deployment to an era defined by coordination of nested systems, vested interests and outdated priorities impede progress.

---

In order for competitive retailing to function as intended, there needs to be a complete transition away from the regulated monopolies that preceded them.

Erik Desrosiers

Power sector consultant

---

In order for competitive retailing to function as intended, there needs to be a complete transition away from the regulated monopolies that preceded them. Unlike regulated monopolies, where choices and outcomes are planned centrally, markets are such that the forces of supply and demand determine what is built and how the system operates.

In other words, electricity restructuring implies a free market unimpeded by any government intervention, price-setting monopoly, or other authority. This ideal has proven difficult to implement due to engineering complexities, gaming behavior, and the intrinsic need to define what and how exchanges happen. Nonetheless, regulators engaged in restructuring must be consistently committed to fixing the market itself rather than promoting any specific outcome.

There are several areas where regulators have compromised the free market model and undermined their own restructuring efforts. Trusting the "invisible hand" is not for the faint of heart, and leaders need the courage to stay the course in the face of uncertainty. Competitive retailing has been handicapped by compromises to deregulation.

Below are five steps that regulators can take to improve retail competition and get restructuring back on track.

1. Unbundle distribution and retailing. Restructuring has already forced the separation of power generation from transmission, however only Texas has extended restructuring to distribution utilities. Unbundling distribution and retailing enables the network to be managed independently, ensures fairness and provides new service providers the opportunity to innovate. This is the essential first step in establishing independent distribution system operators.

2. Phase out default service. Default service undermines retail competition because it regulates the services the market was designed to provide. This maintains the role of regulators as price-setters, tempts further regulation and distorts incentives. With the availability of default service, many residential consumers are less active in monitoring the market and making choices.
   
   Default service also focuses competition on commodity rates rather than value-added services, stifling innovation. Evidence shows that the number of choices increases after price regulation is phased out. Default service should be transitional, relatively high-priced, auctioned to competitive retailers and basic in nature. Texas has phased out default service in favor of providers of last-resort which are intended to be temporary and used only under rare circumstances when a retailer is unable to provide service.

3. Implement efficient signals. Efficient price signals factor in all available information to align prices with marginal costs. In practice, price signals include energy, ancillary services, network capacity and power factor. Technology improvements in modeling, sensing, state estimation, communications and data analysis are increasing the availability of information and making more detailed price signals possible.
   
   Competitive retailers have the sophistication and financial wherewithal to be exposed to fully efficient price signals — which may be complex, volatile and uncertain — on behalf of customers who are incentivized according to their own risk-reward preferences. Regulators engaged in market restructuring can improve signal efficiency by ensuring adequate scarcity pricing and introducing missing signals such as distribution system use-of-network charges.

4. Promote access to information. Markets are animated by information that enables participants to make decisions. Consumers need information about competitive offerings to choose a provider and manage their consumption. Retailers need information about customers and system operations to formulate marketing, investment, and hedging strategies. Regulators have an important role to play in educating consumers, enabling information exchanges and establishing standards for access to data.

5. Enforce consumer protection. Although abusive practices have been an undeniable problem in the implementation of retail competition, regulators must be careful not to undermine their restructuring efforts. There is a delicate balance between protecting vulnerable customers and stifling innovation. Rather than rolling back restructuring with restrictions and price caps, regulators need to establish clear rules and effective enforcement mechanisms. For example, during 2013 and 2014 the Texas PUC initiated 266 investigations that levied $4 million in penalties and resulted in 17 retail licenses being revoked or relinquished.

Fortunately, there are some encouraging trends for competitive retailing. After 15 years of experience with competitive retail, Texas has seen competitive residential rates converge towards wholesale prices to beat those of municipal monopolies. The energy-only market in Texas has also delivered growth in wind power and demand response. California, bursting at the seams with emerging technologies, is mulling a return to retail competition. This is a meaningful step forward, as the decision to maintain regulated retail rates in the early stages of deregulation exacerbated the 2001 energy crisis and left restructuring in limbo.

Technology will continue to change what is possible, raising the pressure on regulatory frameworks and exposing their flaws. We need to constantly remind ourselves that past policy and market design decisions were made with the realities and assumptions of the time. Unfortunately, path-dependent compromises are often made in the name of prudence or vested interests, veering us further away from fundamentally correct economic regulation. Courage is needed to get us back on track and deliver on the promise of market competition.