Just a wee bit arrogant?
Perhaps we, the public, should accept there is a lot the civil service do not want us to find out, such as 2/3rds of my local work force being civil servants supported by the reamining 1/3rds income?
The head of the government's website for the release of public sector data has said it is a challenge to ensure that users can understand the statistics. Cabinet Office official Richard Stirling, who leads the team that runs Data.gov.uk, said that if he was at the Office for National Statistics he would have concerns about …
If the government itself (like, oh, the DfT admitted just now) cannot manage to draw sensible conclusions from their own data, then while it's natural to worry others will cock up the same, they can't really be trusted with aggregating the raw data, can they?
I think it would be best if data.gov.uk kept doing what it said on the tin. Aggregation and making sense of the data is someone else's office. They should do their job, and preferrably do it well, too, of course. But that's not what this office is for. Richard Stirling, please be takink note.
That actually sounded like a reasonable and well thought out response from a senior civil servant!
How could this be allowed to happen? I mean if this sort of thing keeps up we might actually have some civil servants that we can trust! I demand this man be sacked for bringing the service into disrepute by actually knowing what he's talking about...
1 - full dataset of Raw and/or ordered data are available to the public
(advantages: independent data analysis or commissioned commercial analysis on publicly funded data; universities or other training establishments to run courses incorporating analysis of real data where appropriate)
2 - summarised form of data presented
(advantage: data analysis by ONS available for scrutiny when/where required)
There are two main strands - possibly three:
the source data
analysis compiled on source data
decisions influenced or based on analysis of source data
I can see the guys point - why spend public money putting raw data out there that very few people can really understand and be bothered to take the time to understand? That money has to come from somewhere.
On the other hand openess is good. Seems like they are looking to be open but aren't sure of the best way to go about it. Which is a good thing!
Cost permitting releasing both together would seem the best solution - their work can easily be checked by anyone with the relevant skill set and time on their hands, but those people for whom stats is an arcane and interminably boring art can just download the summary, knowing that it has been scrutinised.
Kinda like OSS really - some guys check the src, but most ppl just download the bin from the official site and rely on the few to check it's all fine and dandy.
The alternative of publishing summaries is frought with dangers. What was the methodology behind the analysis? Were the most appropriate statistical methods used? Who says so?
I say keep that raw information coming. Once its out, its out forever and and be checked retrospectively.
Despite the reservations, its cheaper too. A process to publish raw data can be created and run month after month, year after year. An analysis requires consideration, discussion and review, committees to agree the findings, etc. All very costly.