The secret to better weather forecasts may be a dash of AI Climate and weather modeling has long been a staple of high-performance computing, but as meteorologists look to improve the speed and resolution of forecasts, machine learning is increasingly finding its way into the mix. In a paper published in the journal Nature this week, a team from Google and the European Centre for …
And even more forecasts for Trump to alter with his sharpie.
Will a future Trump Whitehouse demand that all forecasts relating to hurricanes heading for Florida avoid Mar-a-Lardo? A few simple rules would make sure that the residence of his holiness Trump is not affected by any sort of weather especially the hoax that is climate change.
Why do I get visions of a certain King trying to stop the tide whenever Trump and Climate Change is mentioned.
No they aren't, they are amazingly good - at least over the next few days.
Billions of $ and 1000s of lives rely on forecasting extreme weather systems
Saying forecasts are crap cos a 3min nationwide summary at the end of the news didn't tell you when it would rain in your garden.
> We all know weather forecasts are crap
No, we don't. Weather forecasting -- on time horizons of days, at least -- is actually pretty damn good. After that it's diminishing returns - that's simply a consequence of the chaotic nature of weather dynamics, and is unlikely to change any time soon.
What is crap is the manner in which weather forecasts are presented in the media. I'm looking now at the BBC weather forecast for my area (in the UK). It tells me that a week from today there will be a 67% chance of rain at 14:00*. That's obviously nonsense; we know that local weather cannot be forecast with anywhere near that resolution at that time horizon. Why do they do that?
*Also, what exactly does that 67% at 14:00 actually mean? Am I the only one who thinks that's not obvious?
Um... okay... so I guess "rain" there means more then 3 drops. Anyway, if that is it, why can't they just say that?
I thought it might have meant, for example, that if you find yourself anywhere in the given locality at exactly 14:00 hours, then there's a 67% chance that it'll be raining where you are (?)
It might help you to know that the UK uses a process called "Ensemble forecasting"
That means they run the same model multiple times, with multiple parameters randomly varied.
So your statement means "We ran the model 100x. Roughly 2/3s of the time it came up "rain" at 2pm in 7 days time"
Does that help?
That means they run the same model multiple times, with multiple parameters randomly varied.So your statement means "We ran the model 100x. Roughly 2/3s of the time it came up "rain" at 2pm in 7 days time"
Does that help?
Yes, but probably not in the way you were intending. If the parameters were really "randomly varied", and you came up with the same answer 66% of the time.. wouldn't that just tell you your model was wrong, ie you'd expect a random distribution of results, not one biased towards rain. The weather isn't a random walk, it's constrained and you're just repeating a mistake made by the infamous Hockey Team when they assumed the wrong kind of noise.
Luckily forecasting isn't quite that dire, although the Met Office is somewhat hampered by having around 70% of it's weather stations unfit for purpose. But for the downvoters who want a short explainer about mass-market climate science, watch this video-
https://www.youtube.com/watch?v=6Cs3Pvmmv0E
> If the parameters were really "randomly varied", and you came up with the same answer 66% of the time.. wouldn't that just tell you your model was wrong, ie you'd expect a random distribution of results, not one biased towards rain.
Actually, no, not necessarily. That would depend how you varied parameters (i.e., in what range and/or from what statistical distribution, which in the case of weather models is highly unlikely to be uniform). There is also, of course, the inescapable principle that All Models Are Wrong (but some models are useful). Or, if you prefer, The best material model of a cat is another, or preferably the same, cat.
Also, you can in fact run statistical ensembles without varying model parameters at all: you choose "stochastic initial conditions". I suspect that weather simulation may well do this too, since the "initial conditions" -- in this case current readings of temperature, pressure, etc. from weather stations and satellites -- are sparsely distributed and inherently noisy.
Ultimately, though, weather is a chaotic dynamical system, so that no matter how good your models, or how rich and accurate your atmospheric/oceanic data, forecast accuracy decays rapidly over time; and increasing model/data accuracy and coverage is an exercise in diminishing returns.
Actually, yes, necessarily. The quote I responded to was-
with multiple parameters randomly varied.
Think of it like rolling a pair of dice 100 times. You would expect a normal distribution. If you don't get that, then one or both dies are loaded. It would also be just a tad unscientific to randomly set parameters and hope for the best, especially when you're trying to figure out the relationship between those parameters. And around £2tn+ is being bet as part of 'Net Zero' on the rolls of those dice.
That would depend how you varied parameters (i.e., in what range and/or from what statistical distribution, which in the case of weather models is highly unlikely to be uniform)
But this is where pop science comes in. We are told that 'the science is settled' and 97% of climate 'scientists' prefer their funding to continue. If true, we would have 1 climate model which would be accurate and reliable. We would know to 97% certainty what the weather would be next Tuesday and on 29/07/2054. We're not even close, although we are closing in on CO2 sensitivity and that most of the early feedbacks/forcings predicted have been falsified.
Also, you can in fact run statistical ensembles without varying model parameters at all: you choose "stochastic initial conditions". I suspect that weather simulation may well do this too, since the "initial conditions" -- in this case current readings of temperature, pressure, etc. from weather stations and satellites -- are sparsely distributed and inherently noisy.
Kind of, but weather/climate isn't random, so you probably wouldn't want to choose stochastic initial conditions or parameterisation because those don't reflect reality. So for a UK model, you wouldn't start with an initial 0000z temperature of 35C and expect to get a skillful prediction. You could try starting it with say, CET data from 1970, and compare how the model is performing vs observed weather observations. But as you say, those observations are inherently sparse, noisy, or just plain untrustworthy or false. So a recent UK temperature record was set at RAF Conningsby when 3 Typhoons taxied past the weather station and took off. That's a correct temperature at that point in time on the flight line, but is garbage for any climate model because it's not weather. And if 70% of the Met Office's weather stations have an accuracy (wrt climatology) of +/-2-5C, then that data is useless for evaluating model performance as well. Especially when observations are sometimes 'adjusted' based on model results..
> Actually, yes, necessarily. The quote I responded to was-
>
> with multiple parameters randomly varied.
We may be at cross-purposes here: I am a mathematician/statistician, so for me "random" means "sampled from from some probability distribution" - but not necessarily a uniform distribution, which is how non-mathematicians tend to interpret "random". But note that in many scenarios -- including potentially parameter selections -- uniform distributions don't even exist (e.g., on the real line), so the "randomness" is of necessity constrained.
> Think of it like rolling a pair of dice 100 times. You would expect a normal distribution.
No you wouldn't. If the dice are unbiased, and the throws independent, you would expect a uniform distribution on the set of pairs of numbers between 1 and 6 :-)
> It would also be just a tad unscientific to randomly set parameters and hope for the best, ...
Sometimes if you're doing an "exploratory" scientific study, and have very little idea of the effect of parameter variation you might do just that to get a handle on the issue. More often, you will vary parameters by selecting from a constrained joint distribution. That's more or less the Monte Carlo method. I do this all the time in my work (which, amongst other things, involves developing parametric models for neurophysiological processes, based on neural data acquired through EEG, MEG, ECoG, fMRI, etc.).
> But this is where pop science comes in. We are told that 'the science is settled' ...
Are we? By whom? What science? I thought we were talking about weather. Did you surreptitiously switch to climate science (a distinctly different beast).
> ... and 97% of climate 'scientists' prefer their funding to continue.
I'd imagine that 97% of employed people prefer their funding to continue.
> If true, ...
Which it isn't
> ... we would have 1 climate model which would be accurate and reliable.
Climate models are pretty good, constantly improving, and make good predictions. Weather modelling, not so much - and we know that it cannot be, even in principle, since weather dynamics are chaotic. See my previous post.
> Kind of, but weather/climate isn't random ...
Please do sort out that confusion between climate and weather - simply not the same thing. So no, neither is technically stochastic, although at various temporal and spatial scales (and note that essentially it is the temporal scale that distinguishes weather and climate) they may appear so, because of the vast number of variables involved and our lack of fine-grained knowledge about the state of the system.
> ... so you probably wouldn't want to choose stochastic initial conditions or parameterisation because those don't reflect reality.
The point is that you don't know the (fine-grained) "reality" (see above). This is precisely why you choose stochastic initial conditions - it's pretty much the best you can do. And what you get, then, is a probability distribution over possible future states of the system. And of course you might well feed historical data into your models to help calibrate them. But again that historical data is coarse-grained, patchy, and (as you point out) noisy, and as such insufficient to accurately calibrate your model - hence the Monte Carlo sampling of parameters, so that your predictions reflect your lack of precise knowledge of model parameters.
Just to say that this stuff is pretty much my day job - in neuroscience rather than weather or climate science, but in fact many of the identical issues arise. Believe me, brains are dynamical systems at least -- quite likely more -- complex than weather systems, and again we have insufficient knowledge of precise system state (tricky with tens of billions of neurons and tens of trillions of synapses) and of how to calibrate our models.
This is my point.
The statement that "X is a chaotic system" is what Dr RV Jones called a "Doctrine of impotence," that discourages any attempts at improvement because we know they are ultimately doomed.
But IMHO we are a long way away from the ultimate point at which those limits apply.
When I read something like "Our thermometers are only accurate to 2c to maintain continuity with the data set" I think "So WTF don't you get a modern one next to it that can read to 0.1c, collect both data sets and see what happens to the model accuracy when you do that with most of your stations?"
What would have happened in Texas in Feb 2021 if they'd had more notice (and a better idea of the severity) of the coming state wide blizzard?
I can see the parallels between the global weather system and the brain. While near the surface of the brain might get reasonable resolution with IR sensors probing deep is going to be tricky. :-( . MI seems to be the way to go but even with 1mm cubes (is that even possible generally?) that's a lot of cells.
Actually there is an AI model that was originally developed to cope with the connected speech recognition problem in the face of background noise and high levels of uncertainty. It's called a Blackboard system
I don't disagree, although ultimately if you pick a fight with the maths, there's only one winner...
> But IMHO we are a long way away from the ultimate point at which those limits apply.
May well be so - depends on how many resources you want (can afford) to put into the problem in the face of diminishing returns.
> I can see the parallels between the global weather system and the brain.
Of course you can only record from very limited brain regions. Intracranial EEG (iEEG or ECoG) gives you very accurate spatial (and temporal) resolution, potentially at any depth, but even then you are actually recording an average local field potential (LFP) from probably tens of millions of neurons. And of course it's invasive (the only human data tends to be from severely ill non-drug responsive epilepsy patients). EEG is cheap and easy, but spatial resolution is poor (there is the problem of "scalp conductance" - the brain is wet!) MEG - my favourite - gives you better spatial resolution than EEG, and can record subortically as well, but even then the spatial localisation ("source reconstruction") is extremely tricky (mathematically it's a highly undertermined problem). fMRI gives you good spatial resolution, but terrible temporal resolution - and is compromised by the slow haemodynamic response. IR technologies are on the up, but I haven't personally seen any data from that.
And of course it's invasive (the only human data tends to be from severely ill non-drug responsive epilepsy patients).
I guess one of the good/sad things to come from the Ukraine conflict is there'll be a lot of patients who've taken non-lethal damage to their brains. One of the very nasty aspects to modern warfare is the sheer number of fragments and splinters flying around that will have caused damage to specific areas of the brain. There was a famous case from the Falklands conflict where a soldier lost (from memory) something like 30% of their brain mass, survived, and regained pretty good function.
Sadly, it is true more generally that a goodly chunk of neuroscience knowledge is/has been gained through studies on people with severe brain trauma or pathologies (plus historical studies performed at a time when, let's say, ethical considerations were somewhat more lax).
When I read something like "Our thermometers are only accurate to 2c to maintain continuity with the data set" I think "So WTF don't you get a modern one next to it that can read to 0.1c, collect both data sets and see what happens to the model accuracy when you do that with most of your stations?"
I think they mean the Met Office problem. So if the underlying dataset is only accurate to +/-2C, then for consistency their virtual thermometers should mirror that uncertainty or error margin.. Which is also a bit of semantics, ie the Met Office uses accurate thermometers, it's just that the measurements aren't accurate for for weather/climate modelling. So the RAF Connigsby fiasco is a great example. The measurements taken when the jets were taking off were accurate, but meaningless as broader weather/climate data because they were contaminated by the jet exhaust.
I can see the parallels between the global weather system and the brain. While near the surface of the brain might get reasonable resolution with IR sensors probing deep is going to be tricky. :-( . MI seems to be the way to go but even with 1mm cubes (is that even possible generally?) that's a lot of cells.
Yep, although MEGs are pretty awesome. Lots of SQUIDs peering into our brains to try and figure out what was going on. And agreed on the complexity, ie maybe 100bn neurons in the typical brain.. Which is far, far more than the number of cells in a typical climate or weather model. When I was a student, I did some work on this looking at controlling prosthetics, and quickly noped out of looking at the brain. Trying to measure nerve impulses and model those was challenging enough!
We may be at cross-purposes here: I am a mathematician/statistician, so for me "random" means "sampled from from some probability distribution"
We also don't know what the OP intended by their comment. But random should mean random. In weather/climate science, yes, it should be constrained by a probability distribution, but if random, should also produce results reflecting that distribution. So with truly random inputs, you wouldn't expect such a biased result. I'm but a humble engineer, and mostly worked with signal processing and feeding a model with random data is a standard test, ie you'd expect a 'normal' output as a result. If it's skewed by 66%, then something is wrong with the model.
Sometimes if you're doing an "exploratory" scientific study, and have very little idea of the effect of parameter variation
But we're not. This is the 'science is settled' argument. We've been doing both weather and climate forecasting for decades, so we should have a good idea what the effects of those parameters will be. Especially given they're also constrained by pretty well understood physical processes, ie gas laws, thermodynamics etc. Climate and weather models are similar, but different and as you say, mostly due to timescale, and also complexity. So climate models have more flexibility to test parameters and refine those models.
One of the key tests is, of course TΔCO2. Solving that is proving very, very expensive. So the UN wants $100bn a year to fight 'global warming'. The US just went bigger-
https://home.treasury.gov/news/press-releases/jy2504
But being so close to the magnificent Amazon is also a reminder that the transition to a lower-carbon global economy is also the single greatest economic opportunity of the twenty-first century. The transition will require no less than $3 trillion in new capital from many sources each year between now and 2050. This can be leveraged to support pathways to sustainable and inclusive growth, including for countries that have historically received less investment.
Which explains why there's so much lobbying from the Green Blob to get their hands on that money. But at $3tn a year, the science underpinning that investment opportunity better be absolutely rock solid, and it isn't. And I suspect nobody showed Yellen data from the orbiting CO2 sensors that show the Amazon is one of the biggest CO2 emitters on the planet..
Are we? By whom? What science? I thought we were talking about weather. Did you surreptitiously switch to climate science (a distinctly different beast).
Mostly the media, and activist climate 'scientists'. But weather and climate are very closely connected, ie climate is just weather averaged over the long term. If we can't forecast weather reliably, we can't forecast climate. Especilaly when the media also frequently connects the two, so 'extreme weather' or wildfires are evidence of global warming when they're just weather, or evidence of terrible land management. There are also presentation tricks used to play on the fear of weather, so the way weather forecasts like the Bbc now have angry red colours for what the historical temperature record were perfectly normal summer temperatures.
The point is that you don't know the (fine-grained) "reality" (see above). This is precisely why you choose stochastic initial conditions - it's pretty much the best you can do. And what you get, then, is a probability distribution over possible future states of the system. And of course you might well feed historical data into your models to help calibrate them. But again that historical data is coarse-grained, patchy, and (as you point out) noisy, and as such insufficient to accurately calibrate your model.
I disagree, but it depends on what you're attempting to achieve. If it's forecasting, then it's precisely why you don't choose random initial conditions. You know what those are, ie what the conditions are at the start of the forecast. So you start tomorrows forecast based on todays weather. And you do the same for climate forecasting, although you have more flexibility. So you could start a 100yr run based on the last decades weather data, and if it's started to diverge by the time the model reaches present, you know there is something wrong with the model..
But not the parameters. Because you're using historical data, so you know those are accurate. Which is the Met Office problem. If 70% of their weather stations are sited poorly, then that data is useless for both climate and weather forecasting because the data are contaminated by the location, ie your parameters are out by up to +/-5C. So unless you account for those error margins, you cannot accurately or reliably calibrate your start conditions, forecast or hindcast. Climate models do try to work around some of the unknowns, ie throw in a couple of volcanoes over the run that might raise or lower temperatures for a couple of years. But they can't account for 'anomalies' like a UK temperature record being set by some fighter jets taking off.. Which the Met Office still hasn't corrected.
But if you want to learn more about just how bad some climate 'science' is, read this-
https://www.drroyspencer.com/2020/06/cmip6-climate-models-producing-50-more-surface-warming-than-observations-since-1979/
During the period of strongest greenhouse gas forcing (since 1979), the latest CMIP6 models reveal 50% more net surface warming from 1979 up to April 2020 (+1.08 deg. C) than do the observations (+0.72 deg. C).
Someone will probably be along shortly to denounce Dr Spencer as a 'denier', but that's sadly normal in climate 'science'. The models are right, it's reality that's wrong. Especially if there's $3tn at stake.. Or my favourite-
https://climateaudit.org/2009/07/03/the-secret-of-the-rahmstorf-non-linear-trend/
Which is a little more technical, and I hope to your liking as it demonstrates the kind of bad math that can be rife in climate 'science'. Then when statisticians point out the climate 'scientists' are doing it wrong, they're again denounced as deniers etc..
> But random should mean random
Errm.. right? Do you mean "non-deterministic"? Or something else?
> In weather/climate science, yes, it should be constrained by a probability distribution, but if random, should also produce results reflecting that distribution.
I'm sure it is. That's just mathematics. How stochastic input transforms to stochastic output, though, is a different and highly non-trivial issue. Of course both weather and climate modellers are aware of all this. It's their bread and butter (as it is mine).
> So with truly random inputs, you wouldn't expect such a biased result.
"Bias" and "skew" are the wrong terms here; I would go for "extremely noisy". As a consequence of the chaotic nature of weather dynamics, error variance inflates rapidly over time; wait long enough and even low-variance input becomes high-variance output. It's simply the nature of the game.
Generally, I would caution against treating weather and climate scientists/modellers as idiots. This is their field. They know more - way more -- about it than you (or I) do. My original gripe was with the poor reporting of weather, leading to misinterpretation and false expectations by the lay consumer - rather than the scientists themselves. As a working scientist I would be (and occasionally am) severely pissed-off by clueless people from outside my field implying that I and/or my colleagues are incompetent/dishonest/corrupt. There is only one good old Anglo Saxon response to that. Don't be that <del>dic...</del> person.
> But we're not. This is the 'science is settled' argument.
That's a non-sequitur to me. I only raised exploratory studies as an example where more unconstrained parameter exploration is actually normal and useful. As far as scientists go, no argument is "settled". Weather science is fully aware of its limitations, as is climate science (see above).
> ... so we should have a good idea what the effects of those parameters will be. Especially given they're also constrained by pretty well understood physical processes, ie gas laws, thermodynamics etc.
Weather and climate scientists probably have PhDs in those, or closely related disciplines. It is their bread and butter. Again: see above. Don't be that person.
> If we can't forecast weather reliably, we can't forecast climate.
That is simply just flat wrong. It is quite commonplace in complex systems to be able to derive extremely accurate macroscopic models even when the underlying microscopic system is chaotic and/or highly stochastic and/or intractable to analysis (a common, but not the only approach is coarse-graining). The entire body of statistical mechanics bears witness to this.
> > I disagree, but it depends on what you're attempting to achieve. If it's forecasting, then it's precisely why you don't choose random initial conditions. You know what those are, ie what the conditions are at the start of the forecast.
You're missing my point by a country mile. Firstly, as regards weather, you don't know what the exact initial conditions are (that would be the precise state of the entire atmosphere, oceans, land masses, and probably also the sun and moon). All you have is a sparse and noisy sample of that initial state. Now combine that with chaotic dynamics - where even a tiny variation in initial state quickly blows up to a large variation in future state. As regards climate, it may well also be chaotic, but the time scales are much, much slower, so that trajectories do not diverge nearly so rapidly, small errors do not inflate so rapidly, and forecasts - albeit also based on patchy/noisy data - are consequently far more accurate. (It is also easier to quantify the error in model forecasts - which, of course, climate modellers do. Because they are not, in fact, clueless idiots - see above.)
(The rest is just politicised/conspiracionalist/cherry-picking flam, which is irrelevant to the original topic, and which I have no interest whatsoever in engaging with - see above.)
(The rest is just politicised/conspiracionalist/cherry-picking flam, which is irrelevant to the original topic, and which I have no interest whatsoever in engaging with - see above.)
Well, I suggest you try reading this-
https://climateaudit.org/2024/06/02/tracing-the-esper-confidence-intervals/
Which as a mathematician, you should understand, and hopefully shocked at just how bad some climate 'science' is. Especially as the paper referred to somehow passed peer review and ended up in Nature. Then the MSM picked it up and ran with it creating the meme that the summer of 2023 was the warmest in millenia.. But as McIntyre shows, that was 'fake news'. I especially liked this part-
In most walks of science, one expects that groups from one scientific institution will be arrive at more or less the same results from the same data.
You would think that, and hope that.. but as shown, it's not how climate 'science' works. Strangely enough, it's garbage like this that often turns believers into sceptics. As a peer, you probably assume climate 'scientists' know what they're doing, but this shows they do not, or intentionally manipulate their results. In most fields of science, that would be considered academic fraud.
Depends on what values you use for "accuracy" and "time"
In the 70's the SoA for models was a Cray 1 with 8MB of 50ns RAM. Today DDR5 ram cards can deliver 0.01953125nS access time (51.2GB/sec)
But IMHO there is a long way to go before we should just throw up our hands and say "We're doomed, the butterfly effect means we can't do more than 6 months reliably!"
Six months (if reliable) can give the whole world time to prepare for droughts, hurricanes, blizzards and other extreme weather events.
In the long run you're right.
But in the long run, we're all dead. I'll stay with the short run for now.
> It might help you to know that the UK uses a process called "Ensemble forecasting"
I did actually know that, but hadn't really thought through the implications (which is ironic, as my day job involves ensemble sampling of stochastic models).
> So your statement means "We ran the model 100x. Roughly 2/3s of the time it came up "rain" at 2pm in 7 days time"
Right...but at some prediction horizon (which for all I know may be about a week) that might just end up effectively being the probability that it is raining at 2pm on the 5th of August according to weather records for my area over the past 40 years (okay, maybe a little better if you can, e.g., at least predict the jet stream a week in advance). Which does stretch the concept of "forecast" somewhat.
> Does that help?
Maybe a little - hang onto that coat.
Agreed.
Actually it used to be thought 2 weeks was the absolute limit. At least that's what I'm reading from the says ECMWF
Turns out with better data collection in terms of data accuracy and resolution it can be extended 16-23 days over the course of a year.
If there is a systemic loss of accuracy (IE It's mostly 16 days in Winter and 23 days in Autumn) that would suggest the model needs revision (I don't know that's the case).
You can estimate probabilities for almost any future event - it's not nonsense to provide that probability, regardless of the resolution. 67% is a mid-range probability, i.e. a fairly vague statement that it's more likely to be raining than not at 14:00 next week. I would be more concerned if they predicted a more extreme probability or gave a definite start time for the rain, both of which do imply more certainty than the models provide.
> I would be more concerned if they predicted a more extreme probability or gave a definite start time for the rain, both of which do imply more certainty than the models provide.
He, he. Just checked again, and the BBC rain forecast for 5th August in my area goes: 14:00 - 67%, 15:00 - 27%, 16:00 - 68%
ECMWF is a 34-state European organisation, not an EU one, and is still headquartered in Reading. Its data centre was moved to Italy in 2017, unrelated to Brexit which I presume is what you're complaining about. ECMWF also operates the Copernicus climate change service for the EU and that, unsurprisingly, was moved to an EU country (Germany) in 2020.
I'm sorry but moving the computing (which requires a lot of scientists from all over Europe to work on location) to Italy was definitely because of Brexit. The UK has become a nightmare to get foreign staff working in the UK. We've had quite a disastrous meeting with the Home Office about getting some of our staff from Germany and Czech Republic into the UK. So bad that we subsequently decided to move our R&D lab out of the UK and all our planned facilities in the UK have been put on ice with a first plant now built in the Netherlands.
The Home Office gives a stock answer "we are working on training British people to fill those roles" but that is pointless. First of all, we need PhD level staff so you'd have to wait seven to eight years for someone to complete that process. Secondly, many of the relevant highly specialised engineering disciplines aren't even taught at UK universities (yet, perhaps ever?). Plenty of Media Studies and History grads but trying to find a dozen people with a PhD in gasification or High Temperature Air Combustion processes and you're out of luck.
Perhaps that will change with the UK population's more relaxed attitude to immigration and a new government that cares about business but it was too late for the ECMWF computing system.
You were promised "fuck business" your business is now getting fucked - what are you complaining about ?
I'm guessing you're referring to UK energy policy? So ECMWF is a large energy user. It's previous datacentre being slap in the M4 datacentre concentration was both power and space constrained..
For the hard of thinking:
https://www.ft.com/content/8075e68c-7857-11e8-8e67-1e1a0846c475
“Fuck business.” Never was the Brexit manifesto more succinctly captured than in Boris Johnson’s impromptu aside. As slogans go, it has everything. It surfs the populist wave of anger towards elites. It is easy to understand. Hell, it’s even shorter than “take back control”.
The UK’s foreign secretary apparently outlined his new business strategy at a private reception, when challenged about the clamour from Airbus and BMW over the threat to jobs and investment. Mr Johnson’s aides say the remark was aimed at business lobbyists. It makes little difference. (He has now fled to Kabul to avoid having to resign rather than vote with the government for a new runway at Heathrow. The foreign secretary had said he would lie down in front of the bulldozers. It turns out he preferred to lie low.)
In theory yes but it adds a lot of friction.
Partners can't get jobs/medical services/school places so you can't hire anyone with a family
97.3% of immigration won't know about the rule so you face threats and possible long legal fight every time you leave and re-enter the country
Was on similar NATO research visa to the USA and occasionally had to spend a day at the INS office having been picked up by some illegal.immigrant sweep that didn't understand the law.
And I'm white with a British accent.
It's also mostly using variants on correlation. If X happened before after Y, then that's a pattern that will repeat. Sometimes that's correct, but often it's not, and whether correct or not it ignores causation.
But lets face it, the whole AI label is purely marketing, aimed a bit at consumer buyers (all the latest AI assisted phones, MS Copropilot etc) but primarily at organisational leaders who are getting the full FOMO hose-down from shit-bag consultancies (SBCs) and tech snake oil sellers, and to those leaders investors/political stakeholders who are an audience who combine the unhelpful traits of greed and endless credulity. Blockchain was going to be a transformative technology, and billions of man hours were wasted looking down the back of the sofa for a use case and still failing. Likewise digital currencies. What's different about AI is that big tech companies stand to benefit in selling services and lots of hardware to blue chip customers, and there's a wholly unproven belief amongst organisational leaders that something will come along, that they can't afford to miss out on, either because it'll create exciting new products, retain customers better, or simply offer the chance to fire more staff. And the leaders of big organisations all have the SBCs wining and dining them, inviting them to important sounding conferences, crawling up their back passage etc.
If your CEO wants to know anything about AI, is he going to (a) go down to the IT engine room, and spend some time having the scruffy on-the-spectrum sceptics there explain that AI is simply the Emperor's New Clothes? Or is he (b) going to tell his PA to invite in his choice of SBC, picked from the likes of BCG, PWC, Accenture etc etc and look forward to a good lunch and some accomplished flattery?
Agreed, LLMs aren't Artificial Intelligence and all the research suggests they are a dead end in the quest to real AI. Mainly due to the weakness in Inference which doesn't do what researcher thought it would.
But, I get the impression that what the ECMWF uses is not LLMs (statistics based on a enormous amounts of data, "Stochastic Parrots") but rather actual Machine Learning.
That's the next logical step for google. They own Nest. My parents have the cameras. The webpage animation for those cameras changes to rain when it's raining and it's not using a weather forecast to do it.
Now with that information and all the other forecast information available it's only a matter of time before we get back to the future style forecasting albeit 10 years too late.
Weather forecasting has come a long way and is demonstrably more accurate than even ten years ago. The ECMWF is the world's most accurate medium range forecasting system, so accurate that even Americans in Hurricane Alley prefer it over their own American hurricane predictors from NOAA.
But, the ECMWF is a medium range forecaster. That means they don't do tomorrow's weather forecast but rather between 15 days and 12 months away. One of their current objectives is to accurately predict storms weeks(!) in advance.
Competition between international teams is a great way to improve on weather forecasting imho. Last year, the US's E³SM SCREAM model (3.25 km horizontal-resolution), running on Frontier (EPYC+MI250x), won the Gordon Bell prize -- against finalists from Japan (Fugaku) and China (Sunway) (all very impressive).
Europe is preparing its retort with the ICOsahedral Nonhydrostatic (ICON) EXCLAIM effort, targeted to run on the Alps Swiss-Cheese supermachine (GH200), with ECMWF in there somewhere too. Gotta wonder if these state-of-the-art systems will find a use for NeuralGCM (eg. to speed things up and maintain accuracy? -- if it does).
> But for the several km^2 area around? Pretty good.
Really? Last week Wednesday for this area was forecast pleasant and sunny when I checked the previous Friday. Same forecast on Saturday, Sunday, Monday and Tuesday.
Wednesday 9am the forecast for the rest of the day had changed to be cold, dull and windy, which indeed it was, for miles around. So much for inviting friends for lunch outside :-(
I find the five day forecast from our US National Weather Service to be reasonably accurate.
Of course, I live in Southern California where we don't have weather so much as climate (not that climate isn't changing at a rather alarming and disheartening rate).
In a previous life, I was a radio deejay and the forecast was almost invariably "Mostly fair through tomorrow with a high in the 70s and an overnight low in the 40s to 50s[1], late night and early morning fog and low cloudiness, northwesterly winds 5 to 15 miles per hour"
________________
[1] Yes, Fahrenheit.
It wasn't until I got to the following that I realized it was likely we weren't talking a about degrees of latitude (ie 60 nautical miles):
Meanwhile, at 2.8 degrees the team found the model was able to predict the average global temperature between 1980 and 2020
Even degrees of temperature aren't unambiguous C(K) or F/(R) or Réaumur?
2.8 degrees could with fewer characters be more clearly rendered 2.8°C.
It's spatial resolution indeed, in the paper: "We train a range of NeuralGCM models at horizontal resolutions with grid spacing of 2.8°, 1.4° and 0.7° (Supplementary Fig. 7)". The Figure 7, on page 25 of the 72-pages of supplementary PDF illustrates it nicely -- the 2.8° earth grid gives a visibly pixelated representation (leftmost), less so for 1.4 and 0.7.
The current local forecast in the US Gulf region is "A 20 percent chance of showers and thunderstorms before 9pm" ... predict weather like that and everyone is happy if it doesn't happen, so they never say anything like "An 80 percent chance of no rain or any thunderstorms before 9pm" ... So everyone just has to figure out whether it will happen (LOL).
It's too hot for a coat these days but wearing a hat while I walk, after reading the reports, keeps me happy.
A true SI unit for temperature would be eV, with Kelvins as a so-so runner-up. The Celsius scale is based on a poorly defined solid/liquid transition temperature for 0º versus the well defined triple point and the temperature at which the vapor pressure of water is about an arbitrary "standard" atmospheric pressure. Don't get me started on people using kilograms as a unit of force.
Anyway, the 2.8º refers to angular resolution, not temperature. If we want to get "scientific", the value would be 0.049.
>team found the model was able to predict the average global temperature between 1980 and 2020
They were able to predict the past? Awesome.
Here are three ways to predict the weather that should work great at large granularity:
1. Forecast tomorrow just like today
2. Forecast tomorrow as it was one year ago.
3. Forecast tomorrow based on average of last three years on given date.
Doesn't take Deep Thought to do those, either.
They were able to predict the past? Awesome.
This is actually an advantage over a lot of climate models. So this-
https://en.wikipedia.org/wiki/ECMWF_re-analysis
and this-
https://en.wikipedia.org/wiki/Coupled_Model_Intercomparison_Project
Which are kinda two different things. ECMWF uses RealData(tm), CMIP tends to work more with imaginary numbers. But also two different purposes, so ECMWF aims to provide accurate forecasts, CMIP to determine just how inaccurate the climate models are that conviince morons to throw soup at paintings. It's one of those "weather is not climate" things. If a climate model can't predict the past accurately when we know what the weather was, then any forecast of 11 ºC by 2050 is almost certainly wrong, but the Bbc will probably still run with it.
But for weather forecasting, past performance is a strong indicator of future performance. So atmospheric conditions that built up & changed over say, the past 100yrs are likely to repeat, if the models know what those conditions were and can see the same conditions occuring.
Here are three ways to predict the weather that should work great at large granularity:Doesn't take Deep Thought to do those, either.
Actually.. It does. Hence why so much money is spent on tin to run weather and climate models. It's one of those wicked problems. So break the planet down into cells, both horizontal and vertical and then run a lot of CFD and other physics calculates to try and figure out what the weather might be 15 days ahead for weather models, or 100yrs for climate models. For weather models, then run them say, 3 times a day because a lot of our economy relies on accurate weather forecasting. Then a large part of our economy also relies on currently inaccurate climate forecasting, ie the demands to prevent 11 ºC by 2050 by building windmills.
Climate forecasting is a wicked problem because it's trying to predict weather 50 or 100yrs into the future. So currently even though they need supercomputers, they have to do a LOT of number crunching. This means the actual physical models are very crude, ie large cells and maybe only 5 or 6 parameters per cell. So a GCM uses hepatoscopy to test some of these assumptions-
https://en.wikipedia.org/wiki/Keeling_Curve#
Many scientists credit the Keeling curve with first bringing the world's attention to the current increase of CO2 in the atmosphere
So a nice collection of CO2 samples showing CO2 increasing since 1958. Given we know this, and have some standardised assumptions wrt the way CO2 levels might increase, GCMs can attempt to answer the mulit-trillion dollar question of what is the relationship between CO2 and temperature? So plug those assumptions into a GCM, let it run forward 100yrs and publish your results so the Bbc's 'climate misinformation' team can spread more doom. Or you test those assumptions by running the model backwards and comparing its results to actual weather data from 1958 to date. So that's 'hindcasting' or model re-analysis. If the model can't predict know, past weather/cllimate conditions, then something in the climate models parameters or assumptions is wrong.. Which has generally shown that the climate is relatively insensitive to CO2, and a lot of the 'forcings' and 'feedbacks' predicted by some climate scientists were just plain wrong.
I think that gives us a pretty clear idea of your views, in case we were in any doubt.
Depressingly enough, it's in their job title. It's also in articles like these-
https://www.bbc.co.uk/news/articles/c1vd1d9r1wxo
Tuesday has become the hottest day of the year so far as a heatwave continues to affect large parts of the UK.
Met Office measuring stations in both Heathrow and Kew Gardens, south-west London, reached 32C (90F) earlier - exceeding the 31.9C recorded in the centre of the city on 19 July.
Scientists say that climate change makes hotter days more likely and more intense.
With $100b-$3tn a year at stake, they would say that, wouldn't they? But the 'hottest day of the year' was not from large parts of the UK, only very small parts of the UK. One a very busy airport where thousands of people are jetting off to holiday in even hotter places, the other from another urban heat island in London.
I'm sure other readers will treat them with the seriousness which they deserve.
Sure. You could go read the Climate Audit articles I left showing how climate modelling actually works. Or you could explain what you thought you meant by 'random'. LionelB had a go, but reverted to mean with this-
As a working scientist I would be (and occasionally am) severely pissed-off by clueless people from outside my field implying that I and/or my colleagues are incompetent/dishonest/corrupt. There is only one good old Anglo Saxon response to that. Don't be that <del>dic...</del> person.
Which is a combination of confimation bias, appeal to authority and wishful thinking. He is not incompetent/dishonest/corrupt, therefore his peers outside his field must be the same. Even though I provided a couple of examples where the incompetence was clear, and actually was his field. But that's climate science for you. It's the history, or future of average weather and relies very heavily on statistics.. Which is often wrong. And again, when actual statisticians point this out, they're generally branded as 'deniers.
But despite not being their field, some clue may be finally dawning with this comment-
You're missing my point by a country mile. Firstly, as regards weather, you don't know what the exact initial conditions are (that would be the precise state of the entire atmosphere, oceans, land masses, and probably also the sun and moon). All you have is a sparse and noisy sample of that initial state.
Except for both weather and climate forecasts, you do know the initial conditions. And you certainly don't start a forecast with random conditions, run an ensemble and then compare to the drying seaweed hanging up on the office door. It's why there are weather stations, weather balloons, sounding rockets, ocean bouys, satellites and more all feeding near real-time data to weather forecasting sites like ECMWF and the Met Office so they know what those initial conditions are, and can revise forecasts based on the data updates.
Climate modelling is much the same, just again average weather. They don't use random numbes either because CMIP is an attempt to standardise model characteristics to aid in comparing them. Which is where this comes in-
https://en.wikipedia.org/wiki/Representative_Concentration_Pathway
The pathways describe different climate change scenarios, all of which were considered possible depending on the amount of greenhouse gases (GHG) emitted in the years to come..
...In RCP 8.5 emissions continue to rise throughout the 21st century.[8]: Figure 2, p. 223 Since AR5 this has been thought to be very unlikely, but still possible as feedbacks are not well understood. RCP8.5, generally taken as the basis for worst-case climate change scenarios, was based on what proved to be overestimation of projected coal outputs. It is still used for predicting mid-century (and earlier) emissions based on current and stated policies
So RCP 8.5 grossly overestimated coal consumption, thus CO2 production, and thus warming. So rather than changing the scenario to more closely reflect objective reality, it continues to be used because Global Warming is such big business. Other climate models have previously been run based on only CO2 and the known physics and show a little over 1C per doubling. It's those pesky 'feedbacks' that can't be found reflected in reality.. Especially when most have been assumed positive, and amplifying the effects of CO2. But then you have to do that to convince people CO2 is a threat. Even though in our past, CO2 levels have been 5-10x higher than present, and life thrived in those conditions.
But such is politics. Don't ask awkward questions like "What happened to the MWP, or LIA?" or why climate 'scientists' chose 1850 as a start date for temperature 'anomalies'. The answer to that is connected, ie if you pick the end of the LIA, you're pretty much guaranteed to produce a warming trend that has no, or extremely weak correlation with CO2.
me> Firstly, as regards weather, you don't know what the exact initial conditions are (that would be the precise state of the entire atmosphere, oceans, land masses, and probably also the sun and moon). All you have is a sparse and noisy sample of that initial state.
you> Except for both weather and climate forecasts, you do know the initial conditions.
I'm really struggling to comprehend exactly how you are failing to understand what I said there. I can only assume that you have no idea what "initial conditions" mean in the context of dynamical systems - and by implication that your understanding of both weather and climate modelling (which are quite different) is non-existent. Taken along with your demonstrated lack of understanding of basic probability (that dice example, and more...) this does not encourage me to take anything you say seriously.
I'm really struggling to comprehend exactly how you are failing to understand what I said there. I can only assume that you have no idea what "initial conditions" mean in the context of dynamical systems
It's pretty clear you're struggling. This is also pretty normal for the "5 stages of grief", where you're still in the denial phase. This is also pretty normal for deniers who are on a journey to scepticism, cult deprogramming, or dealing with victims of con-tricks in general.
I know exactly what is meant by 'initial conditions'. Both weather and climate modelling start with initial conditions, and I'm still puzzled why you think forecasts would start with random conditions. But here's how it works-
https://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model
The data assimilation system combines the latest weather observations with a recent forecast to obtain the best possible estimate of the current state of the Earth system – known as an analysis.
And the Met Office, or any other NWS forecasts work the same way. Start with objective reality and predict what might happen from there, based on what you think should happen using the underlying climate model. Admittedly I have some advantage having lived around 500m from the ECMWF's site in Reading and had many interesting conversations over beers with their staff. They also publish a newsletter if you're interested in what they do, and how their stuff works.
Sure, it's not perfect because as you correctly say, a large part of the Earth doesn't have instrumentation providing data to form those initial conditions. But one of the good things about the Global Warming hype is it's allowed funding for more satellites and other systems to provide more, and better observations. Combined with more compute power, better understanding of our climate system etc ECMWF's predictions have become more skillful over time. Adding 'AI' into the mix may provide further improvements in forecast accuracy.
Taken along with your demonstrated lack of understanding of basic probability (that dice example, and more...) this does not encourage me to take anything you say seriously.
Denial again. But it's also an area where you seemed to be starting to realise the problems with climate forecasting. So take this story again-
https://climateaudit.org/2024/06/02/tracing-the-esper-confidence-intervals/
As discussed in previous article, Esper et al (2024) link, the newest hockey stick diagram, asserted that 2023 was the “warmest summer” in millennia by an updated version of “Mike’s Nature trick” – by comparing 2023 instrumental temperature to purported confidence intervals of temperature estimates from “ancient tree rings” for the past two millennia. In today’s article, I will report on detective work on Esper’s calculations, showing that the article is not merely a trick, but a joke.
You would hopefully understand concepts like confidence intervals, along with confidence tricks. But the Esper paper lead to articles like this-
https://www.bbc.co.uk/news/science-environment-67861954
The year 2023 has been confirmed as the warmest on record, driven by human-caused climate change and boosted by the natural El Niño weather event.
The Esper paper was 'peer reviewed' and appeared in Nature, so it must be true, right? Well, wrong, as you would learn, if you bothered to look at the article. You may be afraid of catching the scepticism bug, and then being branded a heretic or denier. Or you might learn something. Aside from the confidence trick, it also highlights a common problem with climate modelling you aluded to earlier, because Esper et al is a climate model. So it all starts with this-
Buentgen et al (2021) reported on what they called a “double blind” experiment in which they sent out measurement data from 9 prominent tree ring sites to 15 different climate science groups, asking each of them to respond with a “reconstruction” of Northern Hemisphere (extratropic) temperature for the past 2000 years. (Many of the nine tree ring sites are familiar to Climate Audit readers between 2005 and 2012: they include both bristlecone and Briffa 2008 sites, as I’ll discuss later.) The 15 reconstructions varied dramatically (as will also be discussed below). Buentgen’s takeaway conclusion was that the ensemble “demonstrated the influence of subjectivity in the reconstruction process”:
So the kind of science that would have Nyquist spinning in his grave. Only 9 locations used to create a claim that the entire Northern Hemisphere had the hottest year in the last 2ka. Not exactly great spatial resolution, is it? Then there's the basic premise that dendroprhenology can turn wood density into temperatures to 2 decimal places. This also touches on both the sparsity and reliability of temperature data used to calibrate tree rings. So if temperature data is subsequently 'adjusted', then the calibration should be re-run and any climate models based on those temperatures also re-run. But climate 'science' tends to overcome sparsity of actual temperature observations by just making them up. From memory, a couple of the tree ring sites used in Buentgen & Esper et al are in the Yamal Peninsula, and there's no temperature recording site within around 1,000km. But kridging can fix that.. right?
But such is climate politics. If you read into the subject, you'll learn something and may just become a little sceptical. Or it's just easier to close your mind, and brand sceptics as 'deniers'. Scientists are supposed to be sceptics though, hence the rebranding. Reality deniers can be a lil slow..
>> I'm really struggling to comprehend exactly how you are failing to understand what I said there. I can only assume that you have no idea what "initial conditions" mean in the context of dynamical systems
> It's pretty clear you're struggling.
As I said: I'm struggling to understand how you're still completely failing to grasp my original point.
> https://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model
Quote: "The data assimilation system combines the latest weather observations with a recent forecast to obtain the best possible estimate of the current state of the Earth system – known as an analysis."
Let me reiterate my point. Those "latest weather observations" do not specify the full state of the atmospheric, oceanic and other geophysical systems which are pertinent to forecasting (this would be a physical impossibility). They will also, inevitably, be inaccurate. Those observations may well be the best we have, but they do not pin down the initial state which mediates the future time evolution of those systems. And, given that weather systems exhibit chaotic dynamics, that lack of full knowledge of initial state, plus noisy observations, means that the simulated dynamics used for forecasting diverge over time from what the weather actually does. Of course, weather science knows about this; thus they will frequently run many instances of a model, with slightly different initial conditions, in order to get a handle on how sensitivity to initial conditions affects forecasting; to put error bars on forecasts, if you like.
Please see my reply to John Smith 19 - it goes into more detail.
>> Taken along with your demonstrated lack of understanding of basic probability (that dice example, and more...) this does not encourage me to take anything you say seriously.
> Denial again.
On your part, certainly. You demonstrated your poor grasp of probability theory. I called you out on it.
As I said: I'm struggling to understand how you're still completely failing to grasp my original point.
The original point was this assertation by John Smith-
with multiple parameters randomly varied.
And I've pointed out this is not how it works, and not how you should expect it to work in pretty much any kind of forecast. Which then got into a bit of an angels-on-a-pin thing about what random means. In which you tossed out one of the usual canards, the Monte Carlo method. Which is something that has been used in climate 'science' incorrectly. So if you have sparse data, your results will be poor. Like trying to claim 9 trees can accurately show 2ka temperature for our entire Northern Hemisphere. The other obvious one is if you would use Monte Carlo on non-deterministic processes, other than to show that they are, or might be.
Let me reiterate my point. Those "latest weather observations" do not specify the full state of the atmospheric, oceanic and other geophysical systems which are pertinent to forecasting (this would be a physical impossibility).
I haven't disagreed with that point. In fact I've agreed with it, and attempted to show you why this is a problem, especially for climate 'science'. So again the claim of 2023 being the hottest in 2ka. Seriously, go and read that article. It may enlighten you as to just how bad some climate 'science' actually is. Remember, this paper somehow ended up in Nature, and then got vomitted across the MSM because it fits the narrative of anthropomorphic Global Warming.
And, given that weather systems exhibit chaotic dynamics, that lack of full knowledge of initial state, plus noisy observations, means that the simulated dynamics used for forecasting diverge over time from what the weather actually does.
Or those observations are just wrong, and we know they're wrong, and the people responsible for providing accurate observations know they're wrong, just refuse to admit it or do much about it. So 70% of the Met Office's UK weather stations being poorly sited, so for forecasting/climatology purposes, have an error margin of up to +/- 5C. So although it allows the Met Office to claim record highs, and an accurate temperature of 32C for Heathrow, that observation has an uncertainty of (from memory) +/-2C due to it's poor siting and UHI contamination.
Of course, weather science knows about this; thus they will frequently run many instances of a model, with slightly different initial conditions, in order to get a handle on how sensitivity to initial conditions affects forecasting; to put error bars on forecasts, if you like.
Again I urge you to go look at ECMWF's website where they have a bunch of docs explaining all this stuff. But if you know temperature data from the UK is mostly garbage and only accurate to say, +/- 5C, then you might do multiple runs varying the initial conditions for UK cells to reflect that uncertainty. And the same for other locations where temperature data might be equally unreliable. But they're not random.
Of course, weather science knows about this; thus they will frequently run many instances of a model, with slightly different initial conditions, in order to get a handle on how sensitivity to initial conditions affects forecasting; to put error bars on forecasts, if you like.
Less so in weather science, far more common in climate science, and especially climate 'science'. The later doesn't usually bother with error bars and just rushes a press release out. But this is how proper climate science works, so varying conditions and assumptions used in climate model runs to test those models and assumptions. The trillion dollar question is climate sensitivity wrt CO2, and real science pretty much already knows the answer. The climate is pretty insensitive to CO2. But there are also other interesting effects, like theorised 'forcings' and 'feedbacks' based on increased amounts of CO2 that might amplify the physical effects of a humble trace gas. Again, the answer is generally 'No'. Especially when assumed positive feedbacks have been tested. Those naturally increase the amount of warming, but when hindcast (ie reanalysis to compare with actual weather/climate observations), they're shown to have very little predictive skill and diverge from reality quickly.
Hence why over successive IPCC reports, 'preventing 3C warming' became 'preventing 2C warming' to now 'preventing 1.5C warming' because for the more extreme cases, you just can't get there from here in our objective reality.. And then when it's only 1.5C warming, it's pretty much indistinguishable from natural variability, and below past warming periods like the MWP, RWP etc.. Which is also where science vs politics gets fun. HH Lamb wrote about both the MWP and LIA because there's a lot of evidence those existed, but we don't really know why. Climate 'science' has since spent a lot of effort trying to deny those because they can't be explained with CO2 dogma.
It's also why a lot of sceptics get rather frustated by being branded 'deniers' by people that don't have much clue. There are (in)famous examples of scientists like Richard Lindzen, professor of atmospheric physics with a very long publication & academic record being called deniers by climate 'scientists' who are just comp.sci grads. Much the same with Roy Spencer, who often points out the way models diverge from reality. Or Judith Curry, who was infamously hounded out of academia for daring to talk to sceptics and start challenging the dogma. Climate science has sadly become very, very politicised because there's so much money and status at stake.
From a starting point of 1980, the model was able to predict *forwards*, to a reasonable accuracy, the average temp from 1980 to 2010.
The maths is 'chaotic', meaning very slight changes in input (the butterfly effect) tend to get exaggerated and can mean very different outcomes after just a couple of interactions.
One of the techniques in the past was to run a model dozens of times, using slightly different starting values, then discard edge cases and create an average... hoping the answer wasn't one of those edge cases!
I drove from Scotland to North Wales a couple of weeks ago. The middle of summer. Got there, there were about 3 squashed flies on the number plate. If I'd made that journey 30 years ago it would have been covered in dead insects.
When the root of the food chain dissappears what happens to all the creatures that depend on that chain?
Sometimes reality is just so fucking horrid you decide to step outside of it and listen to the comforting voices who explain how all the scientists have got it wrong, except for these very special scientists that are funded by the petro chemical industry.
I drove from Scotland to North Wales a couple of weeks ago. The middle of summer. Got there, there were about 3 squashed flies on the number plate. If I'd made that journey 30 years ago it would have been covered in dead insects.
Yep, that's climate change in a nutshell. You made a single journey on a single route and then jumped straight to correlation is causation. CO2 has caused non-stick bugs to evolve!
When the root of the food chain dissappears what happens to all the creatures that depend on that chain?
Read Charles Pellegrino's Dust for one possible future. Is a fun bit of SF. Emphasis on the 'F', which is true for a lot of speculative fiction, which includes climate 'science'..
Sometimes reality is just so fucking horrid you decide to step outside of it and listen to the comforting voices who explain how all the scientists have got it wrong, except for these very special scientists that are funded by the petro chemical industry.
Ah, that old canard. The evil petrochemical industry that hasn't been making money out of 'renewables', or schemes like carbon capture to get cheap, bulk CO2 for enhanced oil & gas recovery, or just making money from storage. Also wrt insects, one side-effect of large windmills is the downstream turbulence & vertical mixing killing off both insects, and their predators like birds & bats, along with drying out the land. Renewables really can cause 'climate change', if only on the local level.
But the thing with science is that predictions can usually be falsified. So if there's a prediction that doubling CO2 will lead to 4C or more warming, that theory can be tested by comparing prediction to reality. Many such predictions wrt CO2 sensitivity have already been falsified, yet given the money or social policy changes that ride on the back of dire predictions..
Not quite: there is, I understand, little or no evidence for an increase in the frequency of severe storms, but some evidence that their intensity is increasing. (This is intuitive: if you put more energy into a system, it generates more extreme behaviour.) There is also some evidence that warming is leading to hurricanes moving more slowly, which means that they cause more damage over land.
None of the evidence is very strong at this stage, but watch this space.
None of the evidence is very strong at this stage, but watch this space.
And this space-
https://en.wikipedia.org/wiki/Accumulated_cyclone_energy#Atlantic_Ocean
Particularly the graph, and the Top 10 list. Coming in at #1, 1933. Oddly enough correlating with the US's Dust Bowl and a period where a lot of temperature records were set. But then reset because the past being warmer than present is an inconvenient truth wrt CO2 dogma. Also an area that can be rife with controversy, one big example being property damage as a proxy for severity & thus Global Warming. Roger Pielke Jnr debunked that one by normalising losses. Plus some common sense. More people and more expensive property located in hurricane landfall areas, thus more damage, although offset to some extent by building codes being revised to deal with hurricanes.
But also another fascinating area. We have better observations, so IR weather satellites etc, but obviously that's a fairly limited observation window temporally. The mechanics are also interesting wrt Global Warming, eg your comment-
This is intuitive: if you put more energy into a system, it generates more extreme behaviour.
True.. But there's also cause and effect. So there are known(ish) natural cycles like ENSO, AMO, PDO etc along with the general atmospheric and ocean circulation that transports heat polewards. If these cycles are real, and have different periodicities, it follows that at some point in time, harmonics means peaks will coincide and effects become more severe. Challenge, as always is reconciling natural events with AGW and CO2 dogma. That states CO2 radiates in a very narrow IR window, a random amount of which will make land or sea fall, and thus increase warming. But IR is not very good at penetrating water, so it'll hit the surface layer, then the normal heat/energy transfer effects get to work, so conduction, convection, evaporation and radiation. Physics of CO2 are very well understood, hence it being defined as a weak GHG. But so very, very profitable.
Then there's the general processes with cyclones. They build energy from warm waters. They can be tracked by IR weather satellites because they leave a trail of colder water. They convert thermal energy to kinetic, and transport heat up into the atmosphere, so cooling rather than warming events. And all part of the fascinating way our planetary heat engine works.
Whoever would have thunk that?
It should be obvious that any approximation suggests a failure to fully understand a phenomenon. That alone should be a Red Flag that this is an area that needs more research. Working to eliminate "approximations" should be a long term goal of all teams in this field. Given the amount of cooperation it should be possible to split the list of approximations up and divide-and-conquer.
Likewise It baffles me why people try to model an oblate spheroid Earth with rectangular voxels (Where in 3d-land, not flatland here. Like MI's and CAT scans). Fun fact the "waistline" of Earth is 21Km above the mean line of a sphere than the poles. That's about 2.3x higher than Everest. Likewise the Earth's gravity field has been mapped to a resolution of about 2000 points at any given ring around the earth. (by IIRC a joint German-NASA team).
And that "Oh it's the butterfly effect" line smells like BS to me as much today as the day I first heard it.
A massive PITA for all this is that we have a range of sensors that can deliver data at a range of different frequencies and repetition rates. A satellite in polar orbit can cover the whole globe every 24 hours, but can't give a spot reading now. Self contained weather stations can be deployed almost anywhere and give adequate readings at that point now, but without how that site has changed over time what is the "Heat island" effect surrounding it?
Tell me you've got a problem when those models a)Have no approximations (at least none for any phenomena that are near the size of the voxel) b)They match the shape of the planet they are trying to model c)Have a resolution around the 1Km mark in all dimensions.
AFAIK none of these models was built with Knuth's observation that "Premature optimisation is the root of all evil."
IOW (AFAIK) no one started with "Let's build a model that's accurate with the data we can get now regardless of how long it takes to run, then use that to drive
a)What data we need to collect to extend that period. NB this is a weather model. It's forecasting out 1day --> 1 month, not years, centuries or millennia.
b)What holes are there in the basic physics we need to fix (where we used fudge factors just to get it running in the first place)
c)Create it as a set of matched General Circulation models, atmosphere (including the very important cloud effects, which can have both temperature raising and lowering effects), ocean and soil and rock. All coupled together.
d)Then make it deliver on the operational time scale. IE "Fast"
> It should be obvious that any approximation suggests a failure to fully understand a phenomenon. That alone should be a Red Flag that this is an area that needs more research. Working to eliminate "approximations" should be a long term goal of all teams in this field. Given the amount of cooperation it should be possible to split the list of approximations up and divide-and-conquer.
Approximation is inevitable in many, if not most, scientific domains. Models (and almost all science is models!) are de facto approximate - cf. the famous aphorisms The best material model for a cat is another, or preferably the same cat, and All models are wrong (but some are useful).
There is a common misconception among non-scientists about what models are for. Scientific models are not necessarily (or even usually) veridical representations, down to the tiniest detail, of the phenomenon at hand ("the same cat"); rather they stand to abstract just enough of the phenomenon to be able to make reasonable predictions ("some models are useful").
> Likewise It baffles me why people try to model an oblate spheroid Earth with rectangular voxels
I strongly suspect (this is not my field, but I do know quite a lot about dynamical systems, modelling and simulation), that if you delve into the weather modelling literature (or, indeed, try it yourself!), you will be quickly unbaffled.
> And that "Oh it's the butterfly effect" line smells like BS to me as much today as the day I first heard it.
It most certainly is not (this is an area I do know about). It's a fundamental property (almost the definition, in fact) of chaotic systems that nearby trajectories diverge. Read about Lyapunov exponents, and sensitivity to initial conditions. And weather systems are chaotic systems. As I remarked in a previous post, you can pick a fight with scientists (hell, I am one, and we pick fights with each other, all the time!) - but if you pick a fight with mathematics there's only one winner...
The basic scenario in weather simulation and forecasting, is that the physics presents as a (huge) set of partial differential equations (PDEs), with many, many variables and parameters, which describe how the state of the atmosphere, oceans, etc., evolve over time. (The dynamics they represent are chaotic - see above.) Simulating PDEs is pretty much a science in itself, with a huge literature. It is hard. To solve a set of PDEs, you first need to know what the appropriate equations actually are. As mentioned, they have many, many parameters; some of these may be straightforward physical constants (like, say, the specific heat capacity of water), which are known to many decimal places. Others will have to be calibrated from historical weather/geophysical/solar measurement records. And there's your first problem: that historical data may be patchy and noisy, so calibration is inexact. You need to choose a spatial and a temporal resolution (if these are too low, then simulation may be wildly inaccurate; if they are high, the problem may become computationally intractable, even with your super-duper-supercomputer). You need to worry (a lot!) about numerical precision, and how precision error percolates through your simulation over time. Then... you need to have a set of initial conditions; the current state of all the atmospheric, oceanic, geophysical, solar, etc., variables. Oh, and their gradients. These should preferably be at your simulation resolution - but in practice they are not - not even close. And they are awkwardly distributed spatially, and inevitably inaccurate to some degree. Then, the chaotic nature of the dynamics represented by those PDEs quickly blows up all these inaccuracies (parameter estimates, initial conditions and numerical imprecision); yup, there's your butterfly effect, right there. Of course you have to quantify those errors, and how they evolve over time. Did I mention this was hard?
So I'd be circumspect about picking a fight with weather science: you have expressed some off-the-top-of-your-head, intuitive musings about how weather modelling/forecasting works, and the challenges it presents. Fine; but the scientists have spent decades-long careers studying the problem to depths you can only dream of. They tend to have PhDs in the relevant maths, physics and computational software engineering. If you really think you have something useful to offer to the field, there is only one honest recourse: get stuck in yourself, research the field (of course you will need strong maths, physics, thermodynamics, geophysics, numerical computation, etc.) and present your research to the community for peer review. Otherwise, I'm afraid, your musings carry little weight. Is this "argument from authority"? Certainly; but authority really is a thing, and expertise really is a thing. And they do not come cheaply.
I strongly suspect (this is not my field, but I do know quite a lot about dynamical systems, modelling and simulation), that if you delve into the weather modelling literature (or, indeed, try it yourself!), you will be quickly unbaffled.
Short answer: It's a wicked problem. For anyone interested in learning more, HH Lamb's books are a good start and pretty readable.
Longer answer: So to model the weather/climate, all you need to do is take an entire planetary ecosystem and simulate it accurately. There are a crapton of moving parts, not all of which are fully understood. Weather forecasting is 'easier' given it's 'just' attempting to make accurate predictions 3,5 or 15 days into the future. And then run those models say, 3 times a day in an attempt to provide reaasonably accurate forecasts. Which takes some of the biggest supercomputers outside of nuclear weapons development, which shows as a species, we have an odd sense of priority.
But an awful lot of economic activity depends on accurate weather forecasts.
Climate models are much the same, but have additional constraints that they're trying to make forecasts out to say, the year 2100. So this is an example-
https://en.wikipedia.org/wiki/HadCM3
HadAM3 is a grid point model that has a horizontal resolution of 3.75 × 2.5 degrees in longitude × latitude. This corresponds to a spacing between points of approximately 300 km and is roughly comparable to T42 truncation in a spectral model. There are 96 × 73 grid points on the scalar (pressure, temperature and moisture) grid; the vector (wind velocity) grid is offset by 1/2 a grid box (see Arakawa B-grid).[4] There are 19 levels in the vertical using a hybrid (sigma and pressure) coordinate system.
The timestep is 30 minutes (with three sub-timesteps per timestep in the dynamics). Near the poles, fields are fourier-filtered to prevent instabilities due to the CFL criterion.
And it also does ocean, and couples that to the atmospheric model. Which is.. a lot of calculations, so again needs supercomputers and still takes a lot of time for a full climate run. Oh, and back to using random initial conditions-
The slab model needs a calibration phase in which the ocean temperatures are held to climatology while it calculates the "flux correction"
Climate models are still bounded by objective reality. Unless they're being run in a 'test' mode, in which case reality can be rejected and replaced with something objectively false, like the RCP8.5 scenario. That uses an entirely unrealistic assumption for coal consumption and thus model runs based on RCP8.5 run hot.. Which is good news, if you want to announce a result showing say, 5C warming by 2100. Even though that's just garbage.
Again this is the probem of climate 'science' being highly politicised and inevitably leading to corruption of that science. Rather than evidence-based policy, we have policy-based evidence. This is dangerous, and can lead to a distrust of science. But this isn't to say that all modelling is bad. If you look at the evolution of the IPCC's Annual Reports, you'll see that modelling has lead to revised certainties and uncertainties. So run a climate model in a 'test' mode varying elements like CO2 sensitivity, emission scenarios etc and then use reanalysis to compare results to objective reality, ie what we know about past weather/climate. If that model can't predict the past, it's highly unlikey they'll be able to predict the future. Which is kind of what Google is doing. But models that seem able to predict the past reasonably accurately generally don't predict Thermageddon.. So those don't tend to get the publicity, which is again bad for science but is good news for humanity.
Longer answer: So to model the weather/climate, all you need to do is take an entire planetary ecosystem and simulate it accurately. There are a crapton of moving parts, not all of which are fully understood. Weather forecasting is 'easier' given it's 'just' attempting to make accurate predictions 3,5 or 15 days into the future. And then run those models say, 3 times a day in an attempt to provide reaasonably accurate forecasts. Which takes some of the biggest supercomputers outside of nuclear weapons development, which shows as a species, we have an odd sense of priority.
I won't argue with that, broadly.
As regards accuracy (the "how useful is my model?" part), the accuracy of weather models (at a given prediction horizon) depends on some factors I mentioned: spatial and temporal resolution of the simulation (you gave a real-world example), coverage and accuracy of calibration data, and coverage and accuracy of initial conditions (current atmospheric/oceanic/geophysical/solar data, plus gradients). Then there are computing resources.
Ultimately, though, because of the chaotic nature of the dynamics, it's diminishing returns: doubling the coverage of, say, temperature data -- or doubling computing resources -- does nothing anywhere near doubling your accuracy at a given prediction horizon, and nothing remotely near doubling your prediction horizon for a given accuracy. This is why weather forecasting performance, despite massive increases in coverage and accuracy of atmospheric/oceanic/etc. data, and massive increases in computing resources over the past decades, seems to improve at a snails pace.
Climate modelling is distinctly different, certainly in one particular respect: because of the much longer time scales and the smoothing effects of averaging weather over long periods, chaotic dynamics do not bite nearly as hard. There are other highly significant differences, such as difficult-to-predict and mutually-interacting factors which kick in over those longer time scales. These include human behaviour, ill-understood (and probably chaotic) ecological dynamics, natural geophysical cycles, volcanic activity, long/medium-term solar activity, etc., etc.
In some respects, you might conclude that climate modelling is more difficult than weather modelling. But in reality, the style of modelling, and the concomitant challenges, are so different that weather and climate prediction are really not comparable at all, and should never be conflated.
Which brings me to: the article we're commenting about, I should remind, is about weather forecasting. Despite your evident determination to do so, there really was no reason at all to bring climate into the discussion. It's simply off-topic. The Reg forums are remarkably tolerant about that, but I'm sure you can find a more appropriate forum to grind your axes in.
Climate modelling is distinctly different, certainly in one particular respect: because of the much longer time scales and the smoothing effects of averaging weather over long periods, chaotic dynamics do not bite nearly as hard.
They do, it's just as you say, they're smoothed out and generally lost in the noise, but also one of the main reasons why climate models diverge from reality so readily.
These include human behaviour, ill-understood (and probably chaotic) ecological dynamics, natural geophysical cycles, volcanic activity, long/medium-term solar activity, etc., etc.
But the combination of more/better observations are allowing greater understanding. Human behaviours are often macro-scale, so effects of deforestation, poor land management or cooling effects downwind of windmills due to vertical/boundary layer mixing, but those are macro-scale events. Volcanoes can be more fun. So modellers can chuck in a few VEI 4 or 5 events between now and 2100, varying the location. They're known to have 'climate' effects, but usually only on 2-3yr timescales, sometimes positive, sometimes negative. Which is also an interesting current event due to this-
https://en.wikipedia.org/wiki/2022_Hunga_Tonga%E2%80%93Hunga_Ha%CA%BBapai_eruption_and_tsunami#Climate_and_atmospheric_impact
One study estimated a 7% increase in the probability that global warming will exceed 1.5 °C (2.7 °F) in at least one of the next five years
So a model that can be tested against reality. And also why caution is needed claiming any recent warming is due to man-made climate change when it could just be transient warming as a result of this eruption. Time will tell. Less significant for weather models because they just use observations for atmospheric water content. Observations showed that increase, and are showing the decline. Whether that means we'll revert back to cooler weather is TBD and at least it's better than 1816 and 'The Year Without Summer' after Mt Tambora went <bang> or the 1883 eruption of Krakatoa that cause a volcanic winter. Again why real climate science is far more interesting & complicated than just charlatans attempting to attach blame for every weather/climate event to a humble CO2 molecule.
Which brings me to: the article we're commenting about, I should remind, is about weather forecasting. Despite your evident determination to do so, there really was no reason at all to bring climate into the discussion. It's simply off-topic.
Let me point you back at the lede-
Climate and weather modeling has long been a staple of high-performance computing, but as meteorologists look to improve the speed and resolution of forecasts, machine learning is increasingly finding its way into the mix.
And what I've been trying to explain to you. They're very closely coupled, ie a weather forecast has an underlying climate model... Problems with one tend to be reflected in the other, as can improvements as the ECMWF / Google collab is showing.. Now, will you go and read the articles about the 2ka claim?
> Climate and weather modeling has long been a staple of high-performance computing, but as meteorologists look to improve the speed and resolution of forecasts, machine learning is increasingly finding its way into the mix.
Oh, yeah - my bad.
> Now, will you go and read the articles about the 2ka claim?
Um, no. Your Gish-gallop, cherry-picking, conspirationalising and infuriatingly poor grasp of the basics has sapped my will to live. I think I may rather go and have a quiet argument with some religious fanatics or flat-earthers.
I think I may rather go and have a quiet argument with some religious fanatics or flat-earthers.
Sadly it seems you are already firmly in that camp. Just don't go throwing soup at paintings like your fellow travellers do.
But a mind is a terrible thing to waste, especially for someone who claims to be a scientist. The Esper article shows clear evidence of cherry picking, but from a supposed climate 'scientist'. But as one of the original 'Hockey Team', he has form for this. Pre-screening samples to select for Hockey Sticks is, of course a simple way to ensure you produce Hockey Sticks. 9 trees really can determine a 2ka temperature 'record'. I'm suprised more branches of science don't use wooden thermometers when climate science finds them perfectly adequate.
But projection is also common amongst reality deniers. There can be conspirationalising, but statements like Yellen's $3tn a year 'climate opportunity', as is the UN's demand for $100bn a year as part of COP.
You probably have enough clues to identify me. Do feel free to review my publication record.
But that would just be another appeal to authority. Why not instead just offer your expert opinion on the Esper climate model? That's your field, you may be able to show where McIntyre was incorrect..
> But that would just be another appeal to authority.
Actually, I am appealing to you you to assess my "authority" (or lack thereof) directly for yourself.
> Why not instead just offer your expert opinion on the Esper climate model? That's your field, you may be able to show where McIntyre was incorrect.
No, as I have repeatedly remarked, it is not my field. What I have said is that I have expertise in dynamical systems theory, statistics, mathematical modelling and simulation amongst other things - but in a neuroscience context, not climate science. Those domains may be relevant to both fields, but those fields are also funsamentally different. I do not have the background to expertly assess a climate paper. (Note that that would involve reading, and understanding all the relevant literature, arguments and counter-arguments - a major undertaking, and one for which I am not equipped. Research studies never exist in isolation; this is precisely what makes cherry-picking a vacuous and misleading exercise.)
On a point of principle, I have no intention of engaging with your Gish-gallop and cherry-picking.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
