“`html

In recent times, power failures triggered by severe weather or substation intrusions have highlighted the fragility of the electrical grid. For the country’s military installations, which rely on this grid, being prepared for outages is crucial for national safety. What could be a better method to assess preparedness than to cut power?

Lincoln Laboratory is exactly doing that with its Energy Resilience Readiness Exercises (ERREs). During such an exercise, a base is detached from the grid to evaluate the effectiveness of backup power systems and the capability of service members to manage failure. Lasting up to 15 hours, every exercise simulates an actual outage scenario with minimal advance warning to the base population.

“No one anticipated that this type of real-world evaluation would be welcomed. We have now executed it at 33 locations, affecting more than 800,000 individuals,” remarks Jean Sack ’13, SM ’15, who directs this initiative alongside Christopher Lashway and Annie Weathers at the laboratory’s Energy Systems Group.

Per a Department of Energy report, 70 percent of the nation’s transmission infrastructure is nearing its end of life. This deteriorating framework, along with rising power demands and interdependencies, poses risks of cascading failures. Consequently, the Department of Defense (DoD) has intensified its emphasis on energy resilience, or the capacity to anticipate, endure, and recover from outages. Within a base, a power failure could hinder essential operations, create openings for physical or cyber intrusions, and disrupt water supply.

“Threats to this already-sensitive system are escalating. That’s the reason this work is so vital,” Sack states.

Safely disconnecting power

Prior to an exercise, the laboratory team collaborates closely with base leadership and infrastructure personnel to meticulously strategize how to safely disconnect from utility power. Through multiple site visits, they scrutinize each building and operation to comprehend power capabilities, ensure health and safety, and formulate contingency plans.

“We gather individuals who might never have conversed before, yet rely on one another. We often refer to it as ‘connecting mission owners to their utility providers,’” explains Lashway, a former electrician turned energy-systems researcher. “The planning phase serves as a tremendous learning opportunity, allowing us to resolve issues prior to the outage.”

On the day of the outage, laboratory personnel are present to guarantee the process proceeds smoothly, yet the base is expected to manage the exercise. Since its inception in 2018, the ERRE initiative has reached significant installations, such as Fort Bragg, a U.S. Army base in North Carolina that witnesses nearly 150,000 individuals daily, as well as locations as distant as England and Japan.

The objective is to avoid restricting its breadth. All facilities and operations, especially those deemed critical, must be included, and service members are tasked with troubleshooting problems. To enhance the exercises as a readiness assessment, some are tailored with scripted scenarios reflecting real-world incidents. These situations challenge personnel to address a cyberattack on control systems, the shutdown of a backup power facility, or a rocket launch during an outage.

“We can conduct countless tabletop exercises, but when you actually pull the plug, the crucial question becomes, what truly unfolds?” former assistant secretary of defense for sustainment Robert McMahon remarked at a joint House Armed Services subcommittee hearing regarding initial exercises. “Perhaps the most significant lesson I have observed is the lack of appreciation and insight by our senior leaders at the installation level, all the way up to my level, regarding the disparity between what we assumed would transpire versus what actually happened, and then being able to apply those lessons learned.”

Illuminating challenges

The ERREs have illuminated prevalent issues across bases. One significant concern is reliance on fragile or defective backup systems. For instance, electronic devices experience a hard shutdown if not supported by a backup battery to navigate power transitions. In several cases, these battery systems have failed or unexpectedly drained because of age or generator complications. “We observe a massive communications room failing, followed by phones and computers not functioning. It underscores the necessity for redundancies,” Lashway comments.

Generators also pose challenges. Some malfunction because they aren’t regularly maintained or replenished throughout the extended outage. At times, personnel mistakenly assumed a generator would support their entire facility, necessitating reconfigurations afterward. Air conditioning systems are frequently excluded from generator-supported emergency circuits, yet rooms filled with computers generate significant heat, and overheated equipment rapidly shuts down.

The exercises also uncovered interdependencies and chain reactions. In one instance, a fire-suppression system inadvertently activated, covering a hangar in foam, triggered by a pressure drop coinciding with a switch reset.

“Conducting an operation at this scale underscores how each of these components need to operate cohesively and effectively to ensure that the base, and ultimately missions, stay functional,” Lashway asserts.

Beyond rectifying technical concerns, the exercises have proven beneficial for honing coordination and adhering to chains of command. They’ve also exposed social challenges associated with functioning through outages. For example, some DoD regulations limit the use of generators at childcare facilities, prompting parents to coordinate care while fulfilling their missions.

Post-exercise, the laboratory consolidates all findings into a report for the base. It details timestamps of significant occurrences by building, identifies correlations between problems, and summarizes widespread issues across the site. It subsequently provides recommendations for addressing vulnerabilities. “Our aim is to offer as much justification as possible for the base to secure the resources necessary to remedy a problem,” Sack explains.

The researchers also seek to aid bases in preventing challenges and avoiding costly repairs. Recently, they’ve utilized power meters to gather electrical data before, during, and after an exercise. These monitoring instruments unveil power-quality challenges that typically remain concealed.

“Not all power is equivalent, and regulations must be adhered to in order to guarantee that equipment, particularly specialized military apparatus, functions appropriately and doesn’t sustain damage over time. Power metering grants insight into that,” states Lashway.

Igniting resilience for the future

Lincoln Laboratory’s ERRE initiative has led to legislative action. In 2021, Congress enacted a law mandating each military branch to conduct at least five ERREs, or “Black Start Exercises,” annually through 2027. This law was recently reauthorized until 2032. The team has transferred the ERRE procedure to two private enterprises, along with the Air Force and Army, to conduct exercises in the years ahead.

“It’s exhilarating that this has captured Congress’ attention and has expanded throughout the DoD,” states Nick Judson, who oversees the portfolio of energy, water, and natural hazard resilience initiatives within the Energy Systems Group. “This concept began as a method to drive change on DoD installations, encompassing many tough discussions about shutting power to critical missions, and now we’re witnessing substantial enhancements to the readiness of bases and their operations.”

It may even be fostering healthy competition across the services, according to Lashway. During a recent regional event in Colorado, three U.S. Space Force installations competed to extend the scope and duration of their exercises.

The team’s attention is now shifting towards related analyses, such as water resilience. Water and wastewater systems face vulnerabilities to disruptions beyond power outages, including equipment failures, sabotage, or depletion of water sources.

“We are conducting tabletop exercises and workshops to unite stakeholders around the importance of water and wastewater systems in facilitating missions,” states Amelia Servi, who leads this effort. “So far, we’ve witnessed great engagement from groups managing water systems seeking funding to repair these aging infrastructures and from missions that previously took water for granted.”

They are also exploring long-term energy strategies to create installations that are less dependent on the grid. One approach involves implementing microgrids, which are self-sustaining systems capable of utilizing stored energy. According to Sack, microgrids are highly tailored and complex to operate, so one goal is to devise a standardized system. The team’s recent power-metering data is yielding useful foundational inputs for such a design.

The researchers are also contemplating how this endeavor could bolster energy resilience for civilians. Large-scale exercises may not be practical for the general public, but they could be carried out in areas crucial to public safety, or in regions reliant on military resources. During one exercise in Georgia, city residents partially depended on a base’s power plant, prompting collaboration with the city to ensure its resilience during the outage.

“`