Today’s modern society requires a reliable electricity system. Anyone who has lived through a major blackout, such as occurred in the Northeast in 2003, knows that when the lights go out, life shuts down.

Users of electricity also want power to be affordable, of course, and at the same time, policy makers increasingly are demanding that more of the nation’s energy come from renewable sources such as wind and solar.

Meeting these competing challenges won’t be easy. Creating a cleaner, more reliable and less expensive electricity grid is going to require new ideas and a great amount of technological development to lower costs. Unfortunately, these are things that run counter to the incentives of regulated utilities.

In a regulated system, government agencies make basic production and grid-access decisions, and set electricity rates in a way that guarantees utilities a certain rate of return on capital investments and other approved costs. Because utilities’ profits are a function of how much they spend, there is little incentive to cut costs and increase efficiency.

The other option is to rely as much as possible on market forces. While no one has figured out how to completely deregulate the electricity market, “restructuring” clears the way for competition in certain segments, such as generation, that aren’t natural monopolies.

In restructured markets, investment decisions are made by entrepreneurs and engineers instead of government lawyers, and companies spend money only if they believe there is a market for the products they are building. Innovation happens more quickly because it doesn’t have to run through the gantlet of regulatory review.

The push to create a cleaner portfolio of generation resources is just one challenge that calls out for the further use of market forces.

Electricity for the most part can’t be stored, meaning supply must nearly match demand at all times or the grid could come under stress and crash. The problem with renewable power such as wind and solar is that it operates when nature allows, not when grid demand calls for it.

This is already causing problems in California, which has invested heavily in solar energy. The supply of solar power declines rapidly in the late afternoon—right when people are coming home from school and work. The result is tremendous stress on the electricity grid, as a large amount of electricity capacity must be “ramped up” at significant cost to supply power to the system in a short period.

To address this, California—which has a competitive wholesale electricity market—is joining forces with utilities and grid operators in neighboring states in an “energy imbalance market” that allows for transfers of power among participants. A free flow of electrons across the West will allow grid operators to balance supply and demand at a lower cost, and allow California to permit increased penetration of renewable energy sources without building unneeded capacity paid for by consumers. (The market already has saved California ratepayers more than $100 million since late 2014, according to the California electricity-grid-market monitor.)

As technology advances in other areas, power markets will have to adapt further. For example, many are looking forward to an era of electric vehicles, but their widespread use could put more strain on the grid, threatening its reliability. One solution is “time of day” pricing that incentivizes drivers to charge up at night, when power usage and wholesale prices generally are lower. An innovation like that isn’t likely to happen quickly—or at all—in a system where regulators set pricing rules.

Of course, many supply-and-demand challenges could be solved if the cost of storing electricity was brought down to economical levels, allowing it to be implemented in the grid. But again, that is going to require a great amount of technological development to lower costs, along with market rules that allow competitive firms to connect storage facilities to the grid. These are investments that regulated monopolists have little incentive to make.

Restructuring hasn’t lived up to all of its promises. Business customers in restructured states appear to have benefited more than residential customers in terms of lower power prices, perhaps because they have more incentive to shop around.

But in terms of keeping the cost of energy production down, restructuring has worked. Most important, no restructured state is repeating the mistake of spending billions of dollars on nuclear power, which low-cost natural gas has made a money loser.

New technologies, innovation, green power and competitive markets all go together. To create a cleaner, more reliable and less expensive electricity grid, it is time to escape the dictates of government officials and free up competitive forces.