Mystic Met Office abandons long range forecasts The Met Office has confirmed it is to abandon long range weather forecasts, finally acknowledging criticism. The most recent forecasts were so inaccurate, that even the BBC is reconsidering whether to appoint an alternative supplier, such as Accuweather, after 88 years of continuous service from the 1,700-strong MoD unit. The …
Let's get rid of all these pluviometers and tech thingymajiggies. Surround all weather stations with the right combination of frog ponds, rook-and-cone filled pine trees, and sundry other tools, and let the forecasting begin.
Probably quadruples the necessary staff, good news in these times.
When are they going to admit their short range forecasting is pretty crap too?
As an example I told my son on saturday that we wouldn't be going out on sunday because the forecast was for heavy rain. There were even weather warnings in place. I watched the late forecast and they were still forecasting heavy rain for sunday. So eight hours later I got out of bed to glorious sunshine. Checked the forecast and it said sunny periods. No mention that in eight hours the forecast had totally changed. Later on TV one forecaster had the decency to mention the error, except that he said that the weather had done something unexpected. So it seems that it's never the met office's fault when they get it wrong, it's the fault of the weather for not doing what they expected it to do.
We've had snow forecast at least three times in the last few weeks only for there to be no snow. And then this week they forecast a day that was apparently going to be precipitation free for the whole country. Guess what. It snowed.
Seasonal forecasts? Monthly forecasts? They have trouble getting the weather right a few hours in advance.
What's that you say? They have statistics that show how accurate their forecats are? Well sure they do, but that's easy. Here's a little excercise for you: Nip to www.meto.gov.uk and look up the forecast for your home town for next tuesday. Pop back at least a couple of times a day until next tuesday and notice how the forecast changes. Keep a record and I'm sure you'll find that one of the variations turns out to be accurate. So if the Met Orifice keep a record of how many days weather they managed to forecast accurately then they can probably chalk up close to 365. However what if they kept a record of the number of inaccurate forecasts made?
...But I don't actually care what the weather is like personally. I like being surprised too! Well I guess if I actually did pay attention to weather forecasts I would of been surprised as they got them so wrong!
Mines the one hanging next to the broken barometer.
That statement is in the manner of a stopped clock being better than a clock that runs fast or slow. All three of them tell the right time in some part of the world. Which is the sort of thinking that the MetOffice demands of its TV presenters. They don't know enough to come in out of the rain and can't afford umbrellas.
This magazine has had frequent reports on the bias and other problems of the hugely expensive MetO computer. But before all that the national weather organisation, whatever it is, is being cut to ribbons by closures and redundancies and automation. It has been for the last decade.
On top of that; until recently most model runs were intercepted for correction by expert hands. These days the whole shebang goes through untouched. Check out uk.sci.weather. The old posts that is. Most of the present stuff has died the dearth.
All weather models are compared to the "climatological average", which is what I think you mean by last years weather.
The better forecast than this is "tomorrow will be the same as today (slightly adjusted to climatological average)". This has a 50% chance of being right in Britain and Ireland.
You don't bother giving a forecast unless it has a 60% chance of being 60%, as a rule of thumb.
(Most weather services these days are using Ensembles, a collection of forecasts with slight variations in conditions as well as a main forecast. This gives a spread of probabilities, and helps spot cases where the main forecast might not be right. and avoids a Micheal Fish moment. You'll notice there haven't been many of those in the last 20 years ..)
These days we measure the quality of the forecast based on how many days out the forecast will be right > 60% of the time. Its about 5 days for the ECMWF model, the best in the world at the moment. The Met Office Unified Model is in "competition" with ECMWF, but the MO is a member of ECMWF and when good ideas are implemented in one model, they get copied and implemented in the other fairly quickly.
"These days we measure the quality of the forecast based on how many days out the forecast will be right > 60% of the time. Its about 5 days for the ECMWF model, the best in the world at the moment. The Met Office Unified Model is in "competition" with ECMWF, but the MO is a member of ECMWF and when good ideas are implemented in one model, they get copied and implemented in the other fairly quickly."
Fascinating if accurate. You're saying that at more that 5 *days* in the future reliability drops to <=60%.
Not 1 week, 1 month or (OMG) 1 year, but 5 *days*.
I've read various comments over the years about it being a chaotic system, butterfly effect etc.
To me this sounds like a doctrine of impotence. When a model has a resolution where *whole* cities might not cover enough of 1 pixel to rate inclusion I profoundly doubt all relevant factors are taken into account at the resolution of the variable necessary to give correct effects on the outcome or at the spatial resolution needed to be worthwhile. My suspicion is that like turbulence prediction problems the phenomena take place at several different scales, making effective modeling *very* difficult.
I'd love to find a listing of the parameters the ECMWF model includes, as well as what they'd *like* to include but can't, don't or won't. Donald Knuth's point about premature optimization being the root of nearly all evil i definitely at the top of my list of why this don't work.
Mine's the one with a well thumbed copy of "instruments and experiences" in the side pocket.
The details of the ECMWF (and other) model _methods_ are public, published in the scientific literature. The code is a different matter: only a handful of the _models_ are publically available, the best being those such as GFDL, etc. from the states; you can review those.
In science the best checks against error are independent _implementations_ of the experiments (in this case, weather codes), not just reviews of one piece of apparatus (eg one program). Rather than rely on code reviews (those those happen) we re-implement the methods from the papers, using different test cases. (The fixation on Phil Jone's code, etc. for example is a canard: other people have got the same results independently with their own data analyses)
There are different attitudes in met services about releasing code: in the US its mostly considered that "public tax paid for this, it should be free to the public". In the Uk, the attitude is Thatcherite : "the Met Office should operate as a business and sell what it can to keep its tax draw low", and so treats its code as prioprietary. This is why sites like metcheck.com use US models like the GFDL.
Personally I think it would be best if the codes were freely available to the public, but this has problems. Its one thing for NCAR or NOAA in the US to release their code : they have one funding source and no competition ; ECMWF is funded by 35 or so countries, who pay in to get access to the best weather forecasts and the code. If the code was freely available, why would Russia pay its fees ?
Also, be careful not to get _weather_ and _climate_ models confused. Give me a fair coin and I cannot predict heads vs tails (weather), but I can predict the _average_ over 1000 tosses ( climate). Seasonal forecasts are an intermediate case: while I cannot predict the state of the _atmosphere_ more than 2 weeks in advance, I can predict the _ocean_, which is predictable over years, because it changes much more slowly. So it "biases" the weather. I may not be able to predict that April 10 is sunny/wet, but could predict the percentage of wet days in April, by running an ensemble of weather models based on the expected _ocean_ conditions. This is the cutting edge of research, and depends a lot on increasing measurements of ocean conditions that we are now getting.
Modern models, by the way, use resolutions of 1-2 km for weather prediction regionally (eg the UK), 10 km globally, so we do resolve cities. Climate models have coaser resolutions, but we do test them with very high resolutions (extremely expensive computationally) to check for exactly the issues you describe (not resolving cities), and its not a problem; minimal gains appear above ~25km resolution for climate models.
Well they don't include the effects of turbulence just for starters. Those equations require a fine resolution to make any sense and are probably the biggest, most important factor not currently modelled. In my view the current models and methods are as good as useless for predicting more than 4 or 5 days ahead. It makes you wonder how the hell the models can accurately predict 100 years ahead! We keep being told the long very term forecast has a 90% certainty!
"In place of observation, or new scientific thinking, the Met has a new £33m supercomputer. Alas this is used for climate modelling - an area that has proved even less accurate over the past decade than seasonal forecasts."
Got some kind of justification for that out of the blue claim, Andrew?
You mean other than fact it's true! That's the problem with the Met or anyone else trying to pretend they can predict the weather any more accurately that you or I. No-one's ever been able to forecast any weather of any severity to any decent level of accuracy, now with that in mind what do you think anyone really knows about climate change?
My bet is we know the same about climate change as we do about what the weather will be like tomorrow, 50/50 odds of being hotter or colder!
De facto positions often give way to sloth. Have you seen the code for their Unified Model?
Or does its quality have more to do with having a bar within the main building right across from the offices, selling pints for 30p? (I wonder if that survived to Reading...)
They may have predicted a 60% chance of a hot summer, but that was in the detail. Unfortunately they trumpeted a "barbecue summer" in their own headline, the media did not make it up they simply quoted the met office report. Furthermore the met office did not predict in any way that we would have a cold wet summer. So the report was essentially useless, and the met office have admitted that themselves.
The met office know they can't predict the weather with any accuracy months in advance any more than those silly old folk methods can do it. Predicting the weather in August from observed atmospheric activity in May is no more accurate than predicting the August weather by the date that whitethorn blossoms in spring.
Do the Met not mean that the UK is too *small* and strange to forecast? My impression is that continental forecasts are usually more accurate. In summer it's always hellish hot, and in winter it's always devastatingly cold.
Is the UK, a smallish island, subject to the vagiaries of oceanic weather, with its cyclones and occluded fronts piling in from the west?
Mere idle speculation on my part. Much like a long-range forecast, I suppose.
Large land masses are relatively easy to predict for because the air is more uniform and stable.
Here in New Zealand the weather is incredibly hard to predict because we have a tall (relative to the rest of the Pacific) mountin range along the spine of the country. Our weather tends to come in from the SW, end on to the mountain range. If it comes from a few degrees to the East then the area I live in gets cold, wet weather (even had light snow in the middle of summer). If the weather approaches a few degrees to the West then the western side cops the rain and we get hot, dry winds (35+C). The tiniest deviation in the approaching weather has a huge difference in outcome.
Strange how people are willing to rip into climate and weather models, with a proven, measured skill, and yet give the economists an easy ride ...
"Revert back to hunter gatherers". Prove it.
No, I mean it. Show a measure of your day to day accuracy in predicting economic results, and put them up against the skills of any weather forecast. For that matter, show me papers where economists measure there own skills as forecasters and reporters do it. Meteorologists do it all the time: go to a met. conference and its all about the events they got _wrong_, not the one forecast they got right.
Really ? Thats not a hypothetical question: the Met Office (and other met services) are loath
to make public there long-term forecasts, because they know they are only 60% likely.
But they will be ripped to shreds if you have a forecast for a wet / dry summer and don't release it, even if you're only 60% sure.
Lots of fields (farming, seasonal foods, etc) have to make decisions with no information, and plant / make ice cream based on guesses. Your guess is better than theirs, and they want to know even if you've only a 60% chance.
A lot of work is going on with long-range forecasts : we know (sort of) what we need to do to improve them, but its a matter of time and work (better ocean data assimilation, for example). They will always be probabiistic and less useful than short-term forecasts, but being able to say "Next month has a 70% chance of being dry; postpone planting" is worth a fortune to forresters, for example.
but their 5 day ones ain't too lucky, either
Looking at yesterdays (thursdays) forecast for the next 5 days, showed I could expect:
Thursday: Sunny Intervals
Friday: Sunny intervals
Saturday: Sunny
Sunday: Sunny
Monday: Sunny intervals
by today, that had all changed. We're now told to look out for
Friday White cloud
Saturday: Light rain
Sunday: Sunny - yay! one that (still) matches
Monday: Sunny intervals - strewth! two.
So not only did they change their minds about what fridays weather would be, with 1 days notice, but they've either messed up thursdays forceast for saturday, or fridays (or both) since the two conflicting forecasts can't both be correct. So not only is the 5 day forecast pants, but the 1 day forecast is, too.
The point about forecasting is that it shouldn't change if it's accurate. As soon as you change a forecast (or any other prediction, come to that) it means you think the earlier one was a mistake - or that you made it without considering enough information - neither being a particularly clever thing to do. As it is, today turned out to be sunny intervals (which is what they said it would be on thursday), rather than the white cloud they forecast this morning.
Maybe weather forecasting is like standards in I.T. If you don't like the one you've got, just pick another.
If you looked at the satellite images, available on the met office web site, they would probably show you why the forecast changed. Most likely it would be that a weather system that was looking like it wouldn't affect the weather where you were moved into a situation where it would do so, but then moved away again.
...they confidently predicted there would be no precipitation anywhere in the country last tuesday. Huge swathes of Yorkshire woke up on tuesday morning to a significant covering of snow.
Over the last couple of months they have predicted snow a few days in advance only for the forecast to be changed to rain nearer the event. So 100% accurate on snow? Nope.
Strongly suspect that the title of this piece is factually wrong. Almost certainly, the UKMO have not abandoned long-range forecasts - they've merely stopped public dissemination of those forecasts (ie they've returned to previous practice). They'll continue with internal generation of the LR forecasts and continue to inch along on the very tricky path of improving their accuracy.
"I do think that, of all the silly, irritating tomfoolishness by which we
are plagued, this "weather-forecast" fraud is about the most aggravating.
It "forecasts" precisely what happened yesterday or a the day before, and
precisely the opposite of what is going to happen to-day."
Doesn't look like things have changed since Jerome K. Jerome's day then.
where is the so called proof that weatheraction were wrong ?, i have followed their weather data for the last 12 months they do not say (like the met) these predictions will be 100% but that there is a good chance due to solar activity and climate models that the weather will be pushed north or south, the latest report can be found at http://www.youtube.com/watch?v=1eSD5bMqCoE
Whinge, whinge, whinge. Why all the hateful diatribe? The Met office are trying to help you and frankly I think their forecasts are pretty accurate most of the time.
So what if a few Daily Mail -reading cretins got their charcoal soggy? If you can't understand the meaning of 60% probability you don't fricking deserve a weather forecast.
...they think we're idiots or that they think they are gods?
Visiting foreign countries I note that often the forecasts give you a percentage chance of, for example, rain. That way you are allowed to make up your own mind (b) treated as if you have at least a reasonable level of smarts. In the UK we have the met office telling us that it will rain in this location at this time. Which neither allows you to make up your own mind or allows for the fact that you may have a mind to make up in the first place.
Do they really believe their forecasts are that accurate? Or do they think that we are so stupid we can't apply our own judgement?
"It's going to rain" means "Take your wet weather gear." Whereas "There's a 30% chance of rain" gives you some leeway in deciding what to wear. If you're wearing the wrong clothes in the first scenario it's the forecasters fault, in the latter case it's your own fault.
Take that forecast for last sunday that basically said "it's going to piss down all day". Given that it didn't piss down at all it's safe to assume that, if the forecasters are any use at all, the source used for the broadcast weather forecast probably included some sort of probability of rain. Even if that probability was 90%, telling us that would have made them look less stupid. Likewise the snow they confidently predicted was not going to happen two days later. Had they forecast a 10% chance of snow they'd have looked at least part way competent. But no, on saturday they would not allow that there was a chance of fine weather on sunday and on monday they would not entertain the possibility of snow on tuesday. That's a whole world of fail in just three days' forecasts.
Anything more than three days' forecast is a guess.
I've heard of a much more reliable "stick and string" method. (forgiveness if you've heard it before).
Jam a stick in the ground, tie a string at the top.
If the stick and string is wet, it's raining.
If the string is flapping, there's wind.
If the string is stuck out horizontal, it's a strong wind.
If the stick and string is missing, it's a hurricane.
I work outdoors, and have suffered through some of the coldest weather I have ever experienced in my 28 years in the last month.
One day in mid-January, the BBC forecast for my area was mild temperatures (only in single negative figures) and overcast with heavy snow. So i duly put on my waterproof clothes (not very warm, but dry) and headed off to work. It turned out to be one of the clearest days I have seen this year, with a rather balmy temperature of -18.2degC (average temp taken on our calibrated monitoring equipment at 3 stations across a 5 mile area). Gawd only knows what it was with windchill (I reckon less than -30degC).
I know it can be tricky to forecast in mountainous regions such as the highlands or Pennines, but I work only 20 miles south of Glasgow, and my nearest Met Office weatherstation (and MoD listening post) is only 7 miles away.
Sorry, but AccuWeather is the doesn't live up to the first part of its name. I run Forecastfox here, with source data from AccuWeather, for comedy value only. It currently says "Party sunny" when it's pitch black outside. The other day it said "Light showers" when there was a huge tropical thunderstorm happening that gave us over 70mm of rain.
I wouldn't worry about the Met Office not being good at long range forecasts when AccuWeather can't even do real time reporting.
Basically "In large flat ares with simple weather patterns forecasts are apparently quite accurate"
Which suggests a few interesting things.
The accuracy of weather forecasts out to a certain range is *not* universally bad.
Complicated terrain makes weather forecasting more difficult.
If this anecdotal evidence is correct it should be a *clear* sign that current models are missing things, reading them to insufficient accuracy or insufficient spatial resolution to give an accurate result.
When all forecasting the Met Office can claim "We are no worse than anyone else." If other places can do better why can't they?
BTW. Performance related pay?
If sensationalized weather forecasting is your bag, tune in Accuweather any day - but, for validation, don't forget tomorrow what they said today. Very flashy but, for those of us who go down to the U.S. Left-Coastal sea in boats, acting on the opposite of AW's forecasts works out nicely in all but the most gigantic storm conditions - that is, they have the model somewhat right (they can discriminate between a hurricane and a hiccup), but the interpretation, not so much. Methinks they purposely use only the extremes of the probability/prediction distribution as the middle is sooooo booooring. When things get tight in the short-term, they look out the window. Long-term is anyone's guess.
Curious. The Met Office is "finally acknowledging criticism". Why didn't you point out that the Met Office didn't want to do long-term forecasts, that they always acknowledged they were inaccurate, and that they were massaged by journalists anyway? Isn't that sort-of-fundamental to this story?
Here's a thought. The average reader of the Reg is probably smarter than the average reader of the Sun. If you're going to pretend to do climate "news", then it's probably better just to stick with the facts. Constantly getting your take on the story, rather than just the story itself, is just plain irritating to anyone who has an IQ greater than their shoe size.
Can't forecast next week with any accuracy.
Can't even generalise about the season.
But apparently they are still 100% right on climate change predictions.
Perhaps if they stopped using the shiny new super computer for running models where the data is "fitted" to the "correct" answer, they might be able to improve 5 day forecasts instead.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2024
