Chaos - the making of a new science
Book, by James Gleick.
Mandatory reading, otherwise you'll have a major gap in basic understanding. Applies *directly* to weather forecasting.
And no, AI doesn't bypass it.
Meteorologists are starting to experiment with deep learning tech to predict severe weather patterns. David Gagne, a postdoctoral researcher at the US National Center for Atmospheric Research (NCAR), developed a simple convolutional neural network model to forecast the chances of hailstorms. In the last decade, severe storms …
This post has been deleted by its author
Have an upvote for the book on Chaos. Superb work. I read it after getting stuck on the string theory bit of a Brief History of Time.
Can we get the weathermen to make it stop raiing. My back garden is a lake and its Easter and the grandsprogs will be here for 4 days from tonight (shudder). Their 'got to keep them surgically clean' mothers won't let them go anywhere near the huge puddle that is my garden.
I think I'll retreat to the shed on my Allotment with a few bottles of beer.
AI of course does not do any magic to deal with the chaos inherent in the weather: nothing can do.
But that's not the point. At the moment people use big physically-motivated numerical models to forecast the weather, but it's not clear they have to rely exclusively on them. What neural networks (ehem, 'deep learning AI' sorry) like is training data, and lots of it. And weather forecasting is an almost perfect match for this: if you run a forecast n hours in the future, then in n hours you can compare the forecast with what happened and use this to train your NN, as well as use the data as the initial conditions to predict another n hours further out. And this goes on for ever: every n-hour-out forecast can be trained n-hours later. Oh, and there's lots and lots of data: another thing NNs really like.
There's every reason to believe that NNs could become really good at forecasting the weather. The problem is (as with all such things) is that they're also really opaque: when a physically-motivated model screws up you can go in and have a guess about why it screwed up, and you can do a lot of clever tricks like running it with a bunch of different values for some physical parameter to see which one does best. But you can't really do that for an NN -- who knows what physics the weight between two nodes represents?
A good mix might be to use NNs to help train physical models by exploring their parameter space. but I'd guess that for short-term forecasts at least, NNs will win in due course.
(Disclaimer: don't work on weather, but in the same building as people who do.)
If the boffins explicitly design the AI system to allow for outputs such as "There's a front coming; it might go north or it might go south. Stay tuned and we'll let you know about 4 hours in advance. Best we can do, because - you know - it's Chaos", then fine. Another example is, "There's a lovely high pressure zone coming on Friday night. It's a thousand km wide, we're smack in the middle, and there's precisely zero percent chance it'll miss us. It's going to be a lovely weekend."
The basic problem with AI today is that humans design it, and apparently they're uneducated idiots.
Researcher "Why do you think this storm will have dangerous hail"
NN "Well because the others with these characteristics did."
But what are the key factors? What are the factors that are less important?
And BTW that's 88% correct against existing predictions.
So it's 12% less accurate that the existing predictions.
So how accurate are they to begin with?
Generally on this side we get severe weather advisories several hours out to may be a day out. This is not the problem in reality. It is the propensity of some of these storms to have either microbursts, very fast, linear winds, or tornadoes in very localized areas. Often the warnings for these are less the 30 minutes as these spin up rather quickly during the severe weather.
Rarely does a rough thunderstorm directly kill, but tornadoes, microbursts, and straight line winds do kill with regularity. Tornadoes are notorious for their unpredictability. Those of us who experience them regularly have a healthy respect for them.
As I imagine anyone who's lived through being hit by would would also.
I once did a very rough calculation of the power of a Tornado.
It came out about 1000 GW, or 1 TW.
Top tip.
Getting into an argument with Mother nature will probably end badly.