AI weather models can now outperform the best traditional forecasts

A new machine-learning weather forecasting model called GenCast can outperform the best traditional forecasting systems in at least some situations, says a paper by Google DeepMind researchers published today in Nature.

Using a diffusion modeling approach similar to artificial intelligence (AI) image generators, the system generates multiple predictions to capture the complex behavior of the atmosphere. This occurs with a fraction of the time and computational resources required by traditional approaches.

How weather forecasts work

The weather forecasts we use in practice are created by performing multiple numerical simulations of the atmosphere.

Each simulation assumes a slightly different estimate of the current weather. This is because we don’t know exactly what the weather is like around the world right now. To know this, we would need sensor measurements everywhere.

These numerical simulations use a model of the world’s atmosphere divided into a grid of three-dimensional blocks. By solving equations that describe the fundamental physical laws of nature, the simulations can predict what will happen in the atmosphere.

These simulations, known as general circulation models, require a lot of computing power. They are typically operated on high-performance supercomputing facilities.

Learning the weather by machine

In recent years, there has been an explosion in efforts to build weather forecast models using machine learning. Typically, these approaches do not take into account our knowledge of the laws of nature, as is the case with general circulation models.

Most of these models use some type of neural network to learn patterns in historical data and create a single future prediction. However, this approach leads to predictions that lose detail and gradually become “smoother” as future prospects increase. This smoothness is not what we see in real weather systems.

Researchers at Google’s DeepMind AI research lab have just published a paper in Nature describing their latest machine learning model, GenCast.

GenCast mitigates this smoothing effect by generating an ensemble of multiple forecasts. Each individual prediction is less smooth and more similar to the complexity observed in nature.

The best estimate of the actual future is then obtained by averaging the various forecasts. The size of the differences between the individual forecasts provides information about how great the uncertainty is.

According to the GenCast paper, this probabilistic approach results in more accurate forecasts than the best numerical weather prediction system in the world – that of the European Center for Medium-Range Weather Forecasts.

Generative AI – for the weather

GenCast is trained on so-called reanalysis data from 1979 to 2018. This data is produced by the type of general circulation models we talked about earlier, which are additionally corrected to resemble actual historical weather observations to produce a more consistent picture of the world’s weather.

The GenCast model makes predictions on multiple variables such as temperature, pressure, humidity and wind speed at the surface and at 13 different altitudes on a grid that divides the world into 0.25 degree latitudes and longitudes.

GenCast is a so-called “diffusion model”, similar to AI image generators. However, instead of taking text and creating an image, it captures the current state of the atmosphere and uses that to create an estimate of what it will look like in 12 hours.

This works by first setting the values ​​of the atmospheric variables 12 hours in the future as random noise. GenCast then uses a neural network to find structures in the noise that are compatible with the current and previous weather variables. An ensemble of multiple predictions can be generated by starting with different random noise.

The predictions last up to 15 days, which takes 8 minutes on a single processor called a tensor processing unit (TPU). This is significantly faster than a general circulation model. Training the model took five days with 32 TPUs.

Machine learning predictions could become more widespread in the coming years as they become more efficient and reliable.

However, classic numerical weather predictions and newly analyzed data will still be required. Not only are they needed to provide the initial conditions for machine learning weather predictions, but they also produce the input data to continually refine the machine learning models.

What about the climate?

Current machine learning weather forecasting systems are not suitable for climate projections for three reasons.

First, to make weather forecasts for weeks in the future, one can assume that the ocean, land and sea ice do not change. This is not the case for multi-decade climate predictions.

Secondly, the weather forecast depends heavily on the details of the current weather. However, climate projections deal with the statistics of climate decades in the future, for which today’s weather does not play a role. Future carbon emissions are the larger determinant of the future state of the climate.

Third, weather forecasting is a “big data” problem. There are large amounts of relevant observational data that you need to train a complex machine learning model.

Climate projections are a “small data” problem because relatively little data is available. This is because the relevant physical phenomena (such as sea level or climate drivers such as the El Niño-Southern Oscillation) develop much more slowly than the weather.

There are ways to address these issues. One approach is to use our knowledge of physics to simplify our models so that they require less data for machine learning.

Another approach is to use physically informed neural networks to try to fit the data and also satisfy the laws of nature. A third is to use physics to establish “ground rules” for a system and then use machine learning to determine the specific model parameters.

Machine learning will play a role in the future of both weather forecasting and climate projections. However, fundamental physics – fluid mechanics and thermodynamics – will continue to play a crucial role.