Reading between the clouds: New lightning data improves early warnings

The National Weather Service, NASA and the military are gaining new insights into the development of severe weather by leveraging the cloud.

Not just the Amazon cloud, which helps deliver lightning data gathered by Earth Networks to the interested agencies, but the clouds above where most of the world’s lightning activity takes place.

Recently developed lightning detection networks operated by private companies are able to provide information not only about lightning strikes to the ground, but also about inter- and intra-cloud flashes that account for as much as 90 percent of all lightning — and which was, until recently, unavailable to forecasters.

“It’s an order of magnitude greater amount of information compared to cloud-to-ground lightning,” said NWS meteorologist Peter Roohr. “It has the potential for giving us a heads-up for any kind of severe weather.”

That could mean a significant advance in the ability to protect lives and property, not just from lightning but from severe weather in general.

From 1981 through 2010, the United States averaged 54 reported lightning deaths a year, according to the National Weather Service. Tornadoes alone accounted for more than 500 deaths in 2011, however. That was an unusually high number (more than three times the 30-year annual average, the NWS says), but with severe weather becoming more common, the ability to look inside storm systems and track their development is becoming more important.

That ability is being offered by what meteorologists call Total Lightning Information: data not only on ground strikes, but also lightning activity in clouds.

The NWS last August chose Earth Networks as its primary provider of Total Lightning Information, and in January NASA’s Wallops Flight Facility in Virginia signed on to receive lightning data for range safety during aircraft and spaceflight operations. The agencies will be getting near real-time information about lightning activity from the company’s Total Lightning Network of sensors around the world, displayed through the StreamerRT visualization tool, as well as access to archived historical data for research.

Newly available data on lightning in clouds also has the potential to offer large amounts of new information on winter storms and off-shore storms, such as last year’s Superstorm Sandy, weather systems for which lightning information – in the past based only on ground strikes – has been lacking.

Data gathered from sensors around the country is sent via the Internet to mirrored data centers hosted by Amazon on the East and West coasts, where it is analyzed in near real time. Results and alerts are then sent out to customers.

Because most of the 700 lightning sensors now in place in the United States are in urban and suburban areas, there is little backhaul needed to Internet backbones, said Bill Callahan, Earth Networks’ vice president of federal programs. “The vast majority use landline Internet connections, although there are some cellular links,” he said.

NWS traditionally has provided the backbone of the nation’s weather observation systems, with around 2,000 surface stations, many located at airports, feeding data into the Advanced Weather Interactive Processing System. AWIPS data is distributed to government forecasters, as well as to private meteorologists and others who use the data to develop their own forecasts and products.

Despite this history, “the government did not have our own lightning detection networks,” Roohr said. “Private industry jumped ahead, and they have the expertise.”

So rather than duplicate that effort, NWS, NASA and military agencies are contracting with private industry for this information. This shift from government-owned to private data sources is a result of increasingly tight budgets; an increase in the amount of severe weather; a growing demand for weather information from government, industry and the general public; and technological advances in the gathering and transmission of weather data.

“The bottom line is, there is a change in mindset about how the weather enterprise does its business,” Callahan said.

Miniaturization of electronics and improvements in communications allowed Earth Networks to begin deploying its own weather observation stations about 20 years ago, gathering data on 20 common variables, including temperature, humidity, wind direction and speed, and precipitation. The company now has 8,000 stations in the continental United States and another 2,000 in other countries.

Customers, including businesses, schools and local governments, buy the monitoring stations and in return get access to the network’s data as well as to customized weather alerts delivered through local warning systems and, online, through desktop and mobile applications.

Among the data collected by the stations are lightning strikes, which schools and other customers use to warn of the danger during outdoor events.

Lightning detection is not new. A handful of companies have been providing information on ground strikes for the past two decades. “The Weather Service has been getting cloud-to-ground data from Vaisala Inc.,” which operates the National Lightning Data Network, Roohr said. This data also is often included in television weather coverage as an indicator of storm severity.

What is new is the ability to effectively and efficiently detect lightning that does not strike the ground. Ground strikes are easy to detect. They generate long-wave magnetic signals that can be picked up over great distances. But the short-wave signals generated inside clouds are more difficult to detect and even harder to accurately determine their location.

Difficult, but not impossible. “The theory has been around since the 1960s,” Callahan said. The equipment was bulky and expensive and not scalable. A multi-million dollar installation would cover only a small footprint. NASA could afford to install a system at the Kennedy Space Center at Cape Canaveral in support of manned space missions, giving it more information and longer lead times for severe weather warnings, but the systems could not be widely deployed for weather observation and forecasting.

But advances in electronics and communications made it practical to detect the long-wave lightning signatures and identify their source, and Earth Networks began deploying the sensors in its observation stations about five years ago to create its Total Lightning Network.

The system uses time data from multiple sensors and GPS data to triangulate the location of the strikes. Callahan said the system can place a ground strike within about 200 meters with 98 percent efficiency. Cloud flashes are more difficult measure and only have about 80 percent efficiency. But the value of the cloud lightning data is in determining trends that can be correlated with other weather-formation data, so precision location is not necessary to make it useful.

Lightning data is delivered in near-real time to provide information about current conditions, but archived data also is available to researchers, who can correlate it with radar and other data from past events to identify predictive patterns.

To enable this analysis, the data is gathered in analog waveform, Callahan said. “That is important for scientists.” That means the data has to be converted from analog to a digital signal for transmission over the Internet. This is done in a data logger appliance at the site where the observation equipment is installed, in front of the Internet connection. At the data centers it is converted back to analog.

It will take a while for the NWS to take full advantage from the data now becoming available. Vaisala data on ground strikes now comes directly into the Weather Service’s AWIPS system for processing. But NWS forecasters and researchers have to access the new data from the Total Lightning Network directly on Earth Networks’ servers, using the StreamerRT visualization software. The sheer volume of new data makes integrating it into AWIPS challenging, Roohr said.

“A number of facilities are testing ways to directly ingest the data into AWIPS and to visualize it,” Roohr said. “We want to be able to look at it not by itself, but also compare it to radar” and other data.