The fundamental business principles and very fabric of American utility infrastructure have remained basically unchanged for more than a century.

The regulatory compact set forth for public utilities was driven by economic principles. Public utilities, beginning with railroads then the distribution of electricity and gas, were (and are) considered to be natural monopolies because the economies of scale are so great that it is more efficient for one firm to provide the service than for competitors to do so.

The absence of competition enables public utilities to exact monopoly profits by producing less and charging more — an economically inefficient outcome.

Regulation is a surrogate for competition and market forces. The role of public utilities regulators, therefore, is to ensure public utilities act in a prudent manner in lieu of competition and market forces. Rates are determined by regulators through what is known as the rate-of-return regulation with utilities earning a theoretically competitive return on their reasonable capital investments.

The requirement that prudency exists is a safeguard against the tendency of a monopoly to make excessive and unnecessary investments. Historically, the burden of proof for prudency has lied with the utility and is applied conservatively by public utilities commissions.

To earn a rate of return on capital investment, utilities must prove that benefits of an investment outweigh its costs. Although sensible in theory, this burden often results in an impassable test for utility investment in innovation — which is often inherently unproven.

In comparing data across industries, utilities rank the lowest in their spending in research and development (R&D) by orders of magnitude.

High technology companies often spend up to 25% of their revenue on R&D, whereas utilities typically spend less than 1% (usually much less). The average across all industries is roughly 5%. Until recently, utilities were not incentivized to change or innovate because steadily growing demand (load) justified large investments in traditional infrastructure that earned reliable annual returns.

Innovation challenges

But concern over climate change and the improving economics of renewables, electric vehicles, battery storage and the revolution of digital transformation are driving a step-change in utility business models. The confluence of these forces will require utilities and regulators to reevaluate traditional assumptions by recognizing and rewarding innovation.

We are at the precipice of multiple technological revolutions that will fundamentally change how essential services are provided and paid for.

If the driving principle behind public utility regulation is to ensure the prudency of capital investments, and operations and maintenance, in lieu of market forces, it stands to reason that in fundamental ways utilities and their unregulated corporate peers should behave similarly. In the case of investment in innovation, this is clearly not the case.

Compared to their peers, utilities invest a very small fraction of their revenue in R&D. It is becoming imprudent for utilities not to invest in innovation. The regulatory paradigm that has been prevalent for more than a century is backward looking and needs to evolve to facilitate this change — to ensure that the public interest is protected.

The exponential growth in technology renders the future inherently unpredictable except that we know profound change is coming.

As Irving Wladawsky-Berger recently wrote on fostering innovation, it is not just technological but also organizational and cultural. According to Wladawsky-Berger, "[m]ost everyone agrees that such a culture entails five key behaviors: tolerance for failure, willingness to experiment, psychological safety, collaborative and non-hierarchical."

If these qualities are difficult to nurture in competitive businesses, they are almost completely absent, and often actually prohibited, for regulated utilities.

A new regulatory construct

We need to reimagine the regulatory construct to embrace innovation and become more tolerant of uncertainty by adopting more flexible frameworks that value learnings over time.

The utility industry is not unique in facing this challenge. Recent and growing volatility in financial markets has led regulators around the world to rethink their approach.

Simply applying old structures to new technologies without a sophisticated recognition of nuance and the value of learning will no longer work. As a consequence of this shift in perception, financial regulators are discussing and implementing a concept called the regulatory sandbox.

While there is no precise definition of a regulatory sandbox, it is, broadly speaking, a framework within which innovators can test business ideas and products under regulatory supervision, without fear of penalties.

Innovators are provided direct access to regulators in the sandbox framework where permissions can be granted in minutes as opposed to months of litigation. The sandbox concept allows innovators to experiment without the usual regulatory impediments and long delays associated with traditional oversight.

Regulatory sandboxes encourage innovation by creating a relatively low-risk testing environment that allows experimentation and reduces legal uncertainty, they also expose regulators to new business ideas and products, and learnings that can help fill in gaps in existing regulatory understanding.

The first modern regulatory sandbox was implemented by the U.K.'s Financial Conduct Authority (FCA) in 2016. Under FCA's sandbox, projects that fail do not result in disastrous consequences for regulated entities but instead become the basis for conversations with the stakeholders about how to improve.

Since then, regulatory sandboxes have been implemented in several countries, including Abu Dhabi, Australia, Canada, Denmark, Hong Kong, Malaysia and Singapore.

In 2017, FCA released a report on its findings. The program was generally considered very successful.

A majority of participants and ideas, many of which received significant private investment, are continuing toward a wider market launch. A majority of the firms ultimately received full regulatory clearance and complied with consumer safeguards following the program.

An energy sector sandbox

The concept of a regulatory sandbox is also taking hold in the energy sector, led by the U.K.'s Office of Gas and Electricity Markets.

Three projects were granted sandbox safeguards, and another 22 were evaluated. The projects pursued innovations in both regulatory and technical advancements, from exploring peer-to-peer energy trading to tariffs supporting smart home technologies. Singapore is also firmly considering a regulatory sandbox in the energy sector.

Although the U.S. has been slow to embrace the concept of regulatory sandboxes, with a number of agencies outright opposing it, some states have encouraged innovation through more traditional means.

In Illinois, we recently approved the development of two of the most advanced microgrids in North America in Champaign-Urbana, adjacent to the campus of the University of Illinois, and another in the Chicago neighborhood of Bronzeville, which will be the first clustered microgrid when paired with one on the campus of the Illinois Institute of Technology. Illinois will also have 100% deployment of smart meters and related communications infrastructure to nearly 6 million customers by the end of this year.

On the policy front, Illinois is taking the lead nationally in establishing new rules to level the playing field between information technology investments in cloud-based and on-premise systems. We are also a pioneer in competitive wholesale markets. But the value of all of these investments would be exponentially increased by more nimble and creative application of new technology.

During a time of rapid technological development, in order to maximize consumer benefits and the public interest, prudency requires significantly more utility investment in innovation.

Given the momentum behind the concept of sandboxes and the positive results of experiments around the world, now is the right time to consider implementing this regulatory technique by state public utility commissions.