back to article NASA, IBM just open sourced an AI climate model so you can fine-tune your own

Researchers at IBM and NASA this week released an open source AI climate model designed to accurately predict weather patterns while consuming fewer compute resources compared to traditional physics-based simulations. Developed as part of a collaboration between IBM and NASA with help from the US Department of Energy's Oak …

  1. Neil Barnes Silver badge

    While forecast is a classic case of learning from past observations

    How does this approach deal with changing climate or extreme weather events?

    1. b0llchit Silver badge

      Re: While forecast is a classic case of learning from past observations

      The historic data would already contain significant information about the changing of the climate. The model should be able to extrapolate if it at least includes concentrations of the different atmospheric gasses.

      But, remember, the results are just statistical inference and may be everything from spot-on to completely off the rails, with (probably exponentially) increasing uncertainties further into the modelled future.

      1. Jellied Eel Silver badge

        Re: While forecast is a classic case of learning from past observations

        The model should be able to extrapolate if it at least includes concentrations of the different atmospheric gasses.

        From a quick skim of the paper, I don't think it does, and that level of detail is something the physics-based models (try to) do.

        But, remember, the results are just statistical inference and may be everything from spot-on to completely off the rails, with (probably exponentially) increasing uncertainties further into the modelled future.

        Yep. From the paper-

        With all these caveats in mind, Prithvi WxC performs well to exceptionally well at very short lead times (6 and 12 hours), particularly for parameters like surface temperature. However, performance then decays and after about 66 hours Prithvi WxC falls below the performance of Pangu.

        So usual thing of picking the right tool for the job. Advantage seems to be in being a smaller/faster model, it allows more experiments to test various scenarios or assumptions, and then feed into the bigger climate models. Or just testing theories like these-

        https://www.bbc.co.uk/news/articles/cn5zx2zx5xvo

        Central Europe's devastating floods were made much worse by climate change and offer a stark glimpse of the future for the world's fastest-warming continent, scientists say.

        So perhaps in future, scientists might be able to say this with more confidence, once the observations have been integrated into the dataset. Then maybe answer questions like how much was down to 'climate change', and how much was down to say, the Hunga Tonga eruption that kicked a carpton of water into the atmosphere. That may have caused a couple of years extra warming, and those effects may be ending. But our dear 'leaders' can't tax volcanoes. Oh, and the Sun has been a bit weird lately, but again something that might be better for the physics-based models.

        Or just use this to model the potential effects of all that rainfall. Water is no longer in the atmosphere, but what comes down must go back up, so how much will evaporate and end up somewhere else. Could also do things like testing the cooling effects of all that water.

        1. LionelB Silver badge

          Re: While forecast is a classic case of learning from past observations

          Broadly agreed.

          The thing about a warmer atmosphere is that it can hold more water vapour and evaporation will be faster, so the cycle of evaporation and precipitation is intensified. Of course these effects are integral to weather and climate models (and transient effects like volcanic and solar activity may be factored in), so the effects of higher atmospheric temperatures may thus be predicted, to a degree of accuracy constrained by the quality and quantity of available meteorological and geophysical data, by experimenting with model parameters. This is not "in future" - it is already being done - but confidence will, as (I think) you suggest, improve with improved data.

          I would guess that statistical models might usefully augment - but not supersede - physics-based models, to an extent dependent on the time scale.

          1. Jellied Eel Silver badge

            Re: While forecast is a classic case of learning from past observations

            ...so the effects of higher atmospheric temperatures may thus be predicted, to a degree of accuracy constrained by the quality and quantity of available meteorological and geophysical data, by experimenting with model parameters. This is not "in future" - it is already being done - but confidence will, as (I think) you suggest, improve with improved data.

            Yep, but those are pretty huge constraints, eg the Met Office's poor station siting, or just the general sparsity of that data. Which is both a spatial and temperol problem. So there's a general lack of quality surface measurements historically, but the Global Warming hype has brought more money into improving that data. Plus since the satellite era, launching more observation satellites. But-

            https://arxiv.org/abs/2409.13598

            ..introducing Prithvi WxC, a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2)

            So uses more variables than most GCMs and CCGMs that are by necessity pretty crude. Plus a dependency on MERRA-2 and its update frequency. But there's also-

            Spanning from 1980 to the present day with spatial resolution of 0.5 by 0.625 degrees and temporal resolution of 3 hours (Gelaro et al., 2017), it is valuable for climate research and atmospheric studies.

            So higher resolution. Which perhaps leads to the ability to do more interesting 'localised' experiments, ie the effects of Boris (although probably not causation). Or given we're building massive solar and wind farms, the potential effects of those. Both effect the 'climate', so solar farms will cause albedo changes and localised warming, wind farms are already known to have downstream cooling and drying effects due to turbulence and vertical mixing.

            The paper is pretty cautious about how the model should be used, but the best thing is it makes this kind of modelling more accessable given it can be run on maybe $250k's worth of tin rather than the $100m+ supercomputers needed for full-scale climate modelling.

            1. LionelB Silver badge

              Re: While forecast is a classic case of learning from past observations

              > Yep, but those are pretty huge constraints ...

              Yes they are (for some value of "huge"), but quality and spatial/temporal resolution of atmospheric and geophysical data is improving rapidly, especially satellite data.

              > ... Which perhaps leads to the ability to do more interesting 'localised' experiments, ie the effects of Boris (although probably not causation).

              As far as local events go, as regards climate (i.e., medium/long-term averages of weather), "causation" can only be construed in terms of probabilities of, e.g., the intensity and/or frequency of events, not the occurrence and/or nature of specific events. Climate science is very clear on this distinction - popular media reporting, unfortunately, frequently less so.

              > Or given we're building massive solar and wind farms, the potential effects of those. Both effect the 'climate' ...

              I strongly suspect (but don't know for sure) that at present, and at least in the medium term, those effects will be negligible in comparison with the cumulative effects on climate of a couple of centuries of CO2 and other man-made emissions, and indeed in comparison with the effects of natural transient (geological and solar) events.

              1. Jellied Eel Silver badge

                Re: While forecast is a classic case of learning from past observations

                As far as local events go, as regards climate (i.e., medium/long-term averages of weather), "causation" can only be construed in terms of probabilities of, e.g., the intensity and/or frequency of events, not the occurrence and/or nature of specific events. Climate science is very clear on this distinction - popular media reporting, unfortunately, frequently less so.

                Yep, but then the whole point of models is to test theory vs reality, ie the model reanalysis work to refine forecast modelling. Which is where the ability to run a smaller/faster model allows more of that to be tested. Also why I think Hunga Tonga is interesting because it was one of those big events that's generated a lot of data, ie increase in water vapour and tracking circulation. Theory is that would create a few years of warming, then 'climate' would return to normal. Observations allow that theory to be tested, and models refined.

                I strongly suspect (but don't know for sure) that at present, and at least in the medium term, those effects will be negligible in comparison with the cumulative effects on climate of a couple of centuries of CO2 and other man-made emissions, and indeed in comparison with the effects of natural transient (geological and solar) events.

                This is where physics models collide with reality, and reconcilling the differences involves great leaps of faith. So the idea that CO2 is the dominant force, therefore any 'climate change' is man-made and we simply must give trillions to assorted grifters. Even though both physics and models show that stunts like 'Net Zero' make no measurable difference. But again it's where reanalysis helps refine that knowledge. So we know there has been past 'climate change', we know man-made emissions can't have driven observed events like the MWP, LIA etc.. So what caused those? Which is also where attributing those to natural events gets fascinating, ie solar variability and effect potentially exceeding cause.

                Especially when there are so many variables in play. So sea ice is a good example where there have been a lot of predictions, many falsified, or just falsely attributed like the 'Arctic melting by 2013' paper. We're at the end of the current Arctic ice melt season, it's not melting as predicted.. And if you squint a bit, observations may match an 18.3yr lunar cycle, and we know the Moon has a big effect on both weather and climate. But then there are the AMO & PDO cycles, Solar cycles and unpredictable (for now) events like earthquakes and volcanoes. With big quakes going as far as making the entire planet wobble a bit.

                Which is also why real climate science is fascinating, and the actual science is far from being settled.

                1. LionelB Silver badge

                  Re: While forecast is a classic case of learning from past observations

                  I am not a climate scientist, and nor, as far as I know, are you. I simply don't buy your assessment of the state of climate science.

                  And, of course, no science is ever "settled" - that's a feature, not a bug.

                  1. Jellied Eel Silver badge

                    Re: While forecast is a classic case of learning from past observations

                    I am not a climate scientist, and nor, as far as I know, are you. I simply don't buy your assessment of the state of climate science.

                    Luckily, unlike many celebrity climate 'scientists', I'm not trying to sell you anything. But one of the problems with climate science is pretty much anyone can call themselves one, and it cuts across a huge range of fields. So picking on a popular (or populist) one, the infamous Hockey Stick. A graph that purported to show our climate history-

                    https://en.wikipedia.org/wiki/Hockey_stick_graph_(global_temperature)

                    Which then became very highly politicised, or as wiki's infamous editors put it-

                    Arguments over the reconstructions have been taken up by fossil fuel industry funded lobbying groups attempting to cast doubt on climate science.

                    Which is simply incorrect because arguments have been taken up by mathmaticians/statisticians, historians, botanists, geologists and more. MBH98 (and MBH99) both relied on a basic assumption that tree growth, using wood density in tree rings was temperature driven and provided a reliable themometer. Much argument followed, but the basic idea was flawed. Growth is dependent on many conditions, not just temperature. But one of the main arguments was (and still is) whether the Medieval Warming Period existed, was warmer or as warm as present, and was a global event. And much the same with the Little Ice Age.

                    If true, then that is obviously a huge challenge for CO2-driven climate dogma because of the assumption that modern warming is 'unprecedented', not a natural oscillation in our climate, as has been seen many, many times in the past. Which then goes to misinformation involving things like black body temperatures, equillibrium temperature, and the way CO2 has perturbed this, threatening Thermageddon. Or just allowing non-climate scientists like the Bbc claiming that Hurricane Helene is proof of man-made global warming.

                    But a lot relies on simple statistical tricks. Main one being the arbitary decision to base 'Global Warming' and temperature 'anomalies' around a start date of 1850.. Which neatly coincides with both the end of the Little Ice Age and the Industrial revolution. If you pick a cold start, then obviously your trends are going to show an exagerated warming effect and 'anomaly', but can't prove causation. Erase the LIA, and that becomes simpler, or simple misinformation because there's a lot of historical evidence that the LIA was colder, and global with everything from temperature readings to crop & tax data to newspapers grumbling about the weather. Or things like the French Revolution or Holodomor being weather related, Napoleon's invasion of Russia leading to him losing a lot of his troops due to severe cold etc etc.

                    But again why real climate science is fun given it touches on so many fields. Plus I've been studying the debate for probably 30yrs now, so could probably claim to be a climate scientist because I've picked up a lot more knowledge and experience reading papers, learning about the way the world works and debating the subject.

                    And, of course, no science is ever "settled" - that's a feature, not a bug.

                    Sadly it's very much a feature of climate 'science', but then that's driven by politics and vested interests. Hurricane Helene is 'proof' of global warming and evidence of 'extreme weather' being the new normal. So obviously it makes sense to invest trillions in windmills and solar panels, which are vulnerable to those weather extremes.. right?

                    1. LionelB Silver badge

                      Re: While forecast is a classic case of learning from past observations

                      Sorry, I'm not playing your consipracy-theory/Gish-Gallop/misrepresentation/cherry-picking game. Been there, done that, it wasn't fun, and it wasn't enlightening.

                      1. Jellied Eel Silver badge

                        Re: While forecast is a classic case of learning from past observations

                        Been there, done that, it wasn't fun, and it wasn't enlightening.

                        It might have been, if only you opened your mind. I gave you a simple statistical post to look at, you refused, despite claiming to be a statistician. A closed mind is a dangerous thing for someone claiming to be a scientist. But then this is how the Climate-Industrial Complex has managed to scam billions form people too gullible to take a peek at how Hockey Sticks are actually made.

      2. LionelB Silver badge

        Re: While forecast is a classic case of learning from past observations

        > But, remember, the results are just statistical inference and may be everything from spot-on to completely off the rails, with (probably exponentially) increasing uncertainties further into the modelled future.

        Yes - but the same applies to purely physics-based models (albeit, at least for now, to a lesser degree). And the decay (exponential or otherwise) of prediction accuracy with prediction horizon is baked into the physics of weather and climate, due to its chaotic nature. The only real mitigation is more, and better quality data - and even then, it's diminishing returns.

        You might also consider dialling down the "just" there; statistical inference can be a powerful tool. My guess would be that statistical modelling might usefully augment, but not replace, physics models.

    2. Arthur the cat Silver badge

      "All models are wrong, some are useful" — George Box

      The big problem is knowing when a model switches from being wrong but useful to being plain wrong.

      1. Anonymous Coward
        Anonymous Coward

        Re: "All models are wrong, some are useful" — George Box

        When it exists solely to manufacture justifications for legislation, there's no switching involved. It's wrong before its first line of code is written.

      2. LionelB Silver badge

        Re: "All models are wrong, some are useful" — George Box

        No problem at all - in retrospect.

      3. LionelB Silver badge

        Re: "All models are wrong, some are useful" — George Box

        A more serious answer is: When you try use it to answer the wrong questions.

        Models are essentially simplified abstractions of a physical scenario - as they must be, according to another famous quote: "The best material model for a cat is another, or preferably the same cat” - Arturo Rosenblueth and Norbert Wiener. A model will in general explain/predict some aspects of the physical scenario well, and others badly.

        So, e.g., a weather model may predict the weather at some locality very well at a 24 hour prediction horizon, but very poorly at a one week horizon.

  2. Pascal Monett Silver badge

    "a relatively small cluster of 64 Nvidia A100s"

    So now you can have your own weather forcast for, what, only 6 A100s ? At $8,000+ a piece ?

    Or you might choose to rent them on a cloud platform for only $2 per hour.

    A bargain, I say !

  3. jokerscrowbar

    ‘If you can see that hill over in the distance then its going to rain…

    …If you cant see it then its already raining’

  4. TM™
    Big Brother

    The Scientific Method

    Climate science:

    Start with 1,000 versions of a climate model with different parameters.

    Check them against previous data and throw away the 750 versions that don't match.

    Run for a year. Throw away the 200 versions that don't match the new year's data.

    Run for another year and throw away the 40 versions that don't match that data.

    Use the remaining 10 versions to make public policy, take money from those who work for a living (but not those that don't) and restrict people's rights and freedoms.

    Paint anyone that disagrees with said policies or science as someone equivalent to those who would cover up the state sponsored murder of millions of people within living memory.

    N.B. This science can also be applied to the horse racing and the stock market.

  5. BasicReality

    This is great, make up your own fraudulent climate predictions just like the government does!

  6. FelixReg

    Why does this article contain the word, "climate"?

    This model is trained on 40 years of data. That's weather, not climate.

    Want to convert weather to climate conceptually easily? Think sliding window, 30 years long. Once you fill that window, the window's average tells you the "climate" right now.

  7. feral

    historical data is not what it was

    If anyone remembers the climategate dramatics, part of the problem is that historical data was "corrected" and 'anomalies' were normalised. This is to say, the historical record is not what it was, and that the apparent baseline data that this and other models use already has some modern assumptions baked in.

    1. Jellied Eel Silver badge

      Re: historical data is not what it was

      If anyone remembers the climategate dramatics, part of the problem is that historical data was "corrected" and 'anomalies' were normalised.

      That is still a problem, ie most of the UK Met Office weather stations are not sited in accordance with WMO standards, yet are still used to claim Global Warming records. Which then gets fed in to the training data used for climate models like this one. Or just used by politicians to claim that 1.5C warming is somehow an existential threat, and we must thrown billions at the 'renewables' lobby.

      But climate 'science' also gives us news like this-

      http://news.bbc.co.uk/1/hi/sci/tech/7139797.stm#:~:text=Arctic%20summers%20ice-free%20'by%202013'.%20By

      Scientists in the US have presented one of the most dramatic forecasts yet for the disappearance of Arctic sea ice.

      Their latest modelling studies indicate northern polar waters could be ice-free in summers within just 5-6 years

      Arctic sea ice supposed to have all melted by 2013, and yet it's still there.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like