NWS transition to cloud supercomputing could help fuel AI weather prediction

A National Weather Service move to bring its high-performance supercomputing operations into a cloud environment will give the agency newfound ability to adapt as advancements from technologies — such as artificial intelligence — are expected to change the weather modeling landscape, officials told FedScoop. 

As it stands, NWS has two operational supercomputers that serve as the backbone for all of the agency’s weather models. Those operate in data centers and on-premises (or “on-prem”) environments, and while that structure has thus far been successful, David Michaud, director of the NWS’s central operations, told FedScoop the agency is using the expiration of that equipment as an opportunity to modernize and prepare.

“We wanted to look beyond just trying to build the fastest system that we could or the largest system that we could,” Michaud said. “We were really looking at a balance between the computing that we have and the flexibility that we would get moving to the cloud.”

The transition of those supercomputing operations, known as the Weather and Climate Operational Supercomputing System (WCOSS), comes as part of broader cloud transition efforts within NWS. Simultaneously, the agency is also moving its weather data and resources to cloud-based platforms known as NWS HIVE and NWS CIRRUS that will give forecasters the ability to access crucial data away from home offices, via mobile devices. 

Taken together, Michaud explained, those projects are all examples of the agency’s push to modernize its overall technology footprint and create more adaptability. Putting high-performance computing into a cloud environment, specifically, will allow the agency to make changes as advancements in AI weather prediction arise — something that’s more difficult to do in a more fixed, on-prem environment.

Computationally, the footprints to run traditional weather models and AI prediction models are very different, Michaud said. If AI prediction capabilities develop quickly, on-prem systems with a finite set of compute might not be able to shift the balance to accommodate. Cloud systems, however, can shift that balance quickly, he said. That can allow the agency to address on-demand needs, such as those for severe weather, which manages its normal workload, he said. 

The agency recently announced Google Cloud as the primary provider of the high-performance computing (HPC) infrastructure for WCOSS. In an announcement at the time, Brent Mitchell, Google Public Sector’s vice president of go-to-market, likened its technology to a synchronized group of delivery vehicles on a superhighway as opposed to individual vans making their way through city traffic.

For the National Oceanic and Atmospheric Administration’s workforce, “the move to cloud HPC provides a more streamlined, flexible environment, enabling meteorologists and researchers to quickly adapt future modeling innovations,” Mitchell said.

Those innovations include AI prediction. While use of the booming technology in weather prediction is still in early stages and can be limited in extreme weather situations, it shows promise to help improve forecasts. 

Already NWS has multiple AI-fueled models, including a global model that uses less compute to produce a forecast and an ensemble global model that provides probabilistic information to forecasters. Those are known as AIGFS and AIGEFS, respectively. The agency has also combined its traditional numerical weather prediction model with a hybrid system that can provide a more accurate forecast in a model called Hybrid-GEFS.

The move to cloud will actually return that newer technology to its environment of origin, Richard Bandy, director of NWS’s Meteorological Development Laboratory, told FedScoop.

Some of the agency’s existing work with AI-based models was developed in the cloud on a solution from Google’s DeepMind, a research subsidiary of the tech giant, and then put into operation on the existing on-prem infrastructure, Bandy said. “We’re excited that this cloud flexibility will allow us to move back to the cloud environment that those models were developed to run on.”

NWS’s Warn-on-Forecast System — which is designed to improve the speed of warnings for tornadoes, severe thunderstorms, and flash floods — could benefit similarly. Bandy said that system was designed to run in the cloud and has been running in a demonstration mode.

“One of the challenges with that system and operationalizing was we didn’t have an operational cloud environment to put into,” Bandy said. “So, by us moving the entire production suite to the cloud, it solved that problem for us, and we now see a path in the future towards operationalizing those kind of development projects as well.”

The potential improvements are just one of the steps the agency is taking to improve the resolution of its models at a global scale. Bandy explained that the agency sees a future where traditional numerical weather models are combined with AI models to gain the benefits from both.

“We’re not going to ever probably go away from doing traditional physical-based modeling, but the AI models and the speed at which they can run will continue to grow, and they’ll become a component of a future hybrid solution where we’re leveraging the best of both capabilities,” Bandy said.

NWS also aims to consolidate its model suite and one day have its global model operating at a high enough resolution — or degree of granularity — that the agency won’t have to run regional models, per Bandy.

As far as next steps, the agency is taking a phased approach.

In an effort to reduce risk, Michaud said, the agency has been working with Google to establish a smaller-scale early access environment and plans to run performance tests to get a better sense for what implementation will look like at final scale. He said NWS expects to have an at-scale environment by the start of 2027.

“I feel very confident about our approach, even though it’s a major transformation,” Michaud said.