Always Be Prepared: Lessons Learned From 2011 Blackout Help CPS Energy Customers, State Grid

Texas’ electric grid came perilously close to rolling power outages during Monday morning’s icy temperatures after two power plants tripped offline, grid officials said.

“I think we were close,” said Dan Woodfin, director of system operations for the Electric Reliability Council of Texas, which operates the state’s largest grid. Had another large plant stopped supplying power, he said, ERCOT officials would have had to order a rolling blackout.

But thanks to lessons learned from Feb. 2 of 2011, CPS Energy was prepared for this round of cold weather. Increased investment in freeze protection at all its plants, tighter winter preparedness protocols and excess capacity allowed the utility to sell some power into the system on Monday, which helped stabilize the state grid — and earn roughly $2 million in additional revenue, a portion of which flows to the San Antonio community.

It was a different story in 2011. A deep freeze gripped Texas for three straight days, knocking out more than 150 power plant units, leading to forced blackouts, a disrupted economy and a slew of investigations into what led to the failures.

Like utilities across the state, CPS Energy, as a member of the ERCOT grid, had to shed load, rotating power outages among customers while avoiding hospitals, military bases and the airport.

But the emergency didn’t run smoothly, with eight of the utility’s own units tripping offline more than a dozen times, albeit for short periods of time. Rapidly changing conditions exacerbated communications shortcomings by CPS Energy to the media and its customers.

In the wake of the 2011 event, CPS Energy President and CEO Doyle Beneby, who had been on the job for less than a year, ordered a thorough review of all related processes, resulting in a number of improvements to equipment and weatherization procedures.

“Power plants are complex machines, and weather is tricky to predict,” Beneby said. “We’re addressing these factors by putting good processes and controls in place, but this business can humble you in a minute.”

For example, close to $6 million was spent on freeze protection at all the plants, with new heat tracing — thin heating elements that run the length of the pipe — and insulation, plus better testing to locate failures in the system.

That testing can be critical. In 2011, Spruce 2 was brand new — yet one small, undetected failure “brought the whole thing down,” said David Herbst, who oversees power plant management.

Winter (and summer) plant readiness checklists are now more detailed, the completion of which must be certified by each plant manager. A state-of-the-art alert system was also created, with specific preparations pegged to falling temperatures and times.

“We also plan farther ahead,” said Paul Barham, who runs CPS Energy’s market operations center. “So even though we had a number of units out on routine maintenance this time, we were in really good shape.”

For example, he said, CPS Energy’s two natural gas peaking units, Braunig 1 and 2, which are used largely in the summer to provide power during the hottest summer afternoons, were fired up in plenty of time to provide power as Monday’s freeze approached.

Communications has also improved, thanks to a customer alert system CPS Energy put into place in the wake of the 2011 emergency. Just a week after the event, the utility was able to send out 250,000 emails warning customers that freezing weather could once again bring rolling outages.

Today, the alert system sends texts, emails or phone calls to customers for both widespread outages and whenever an outage of more than five minutes occurs at their home or business, letting them know the estimated time of restoration. It also sends a follow up message confirming power has been restored. Customers can even choose their preferred method of communication.

The new system is already earning kudos from customers like Sam Talley, who posted about his experience last week on Facebook.

This morning at 6:11AM our power went out…15 minutes after it going out I received an email from CPS stating they were aware of the outage at our home and neighborhood and crews were looking to fix it by 8:15 AM and to call if we had any questions or emergency situations. … The power came back on at 8:15 and we also got an email asking us to call if it was still not back on or if we had questions. AMAZING service from a power company! No more being left in the dark wondering if I should call and report it or how long it will take to fix it….Thank you CPS Energy.

“We learned a lot of lessons in the wake of Feb. 2,” said Cris Eugster, chief generation and strategy officer. “But they’ve helped us improve tremendously, to the point that during this freeze, we were able to cover our native load and help the state of Texas meet its power needs.”

Tracy Idell Hamilton manages and writes Energized, CPS Energy’s blog, from which this post has been republished with permission. She’s a former San Antonio Express-News reporter who covered energy and City Hall, among other beats. Follow her on Twitter @tihsatx.

Full disclosure: The Arsenal Group conducted a four-month review of CPS Energy communications for the utility starting in June 2012. Monika Maeckle, a former member of the The Arsenal Group and wife of Robert Rivard, now works at CPSE as its Director of Integrated Communications.

Related Stories: