Data.gov.uk chief admits transparency concerns

The head of the government's website for the release of public sector data has said it is a challenge to ensure that users can understand the statistics. Cabinet Office official Richard Stirling, who leads the team that runs Data.gov.uk, said that if he was at the Office for National Statistics he would have concerns about …

[Martyns]

Just a wee bit arrogant?

Perhaps we, the public, should accept there is a lot the civil service do not want us to find out, such as 2/3rds of my local work force being civil servants supported by the reamining 1/3rds income?

[Anonymous Coward]

Misplaced concern

If the government itself (like, oh, the DfT admitted just now) cannot manage to draw sensible conclusions from their own data, then while it's natural to worry others will cock up the same, they can't really be trusted with aggregating the raw data, can they?

I think it would be best if data.gov.uk kept doing what it said on the tin. Aggregation and making sense of the data is someone else's office. They should do their job, and preferrably do it well, too, of course. But that's not what this office is for. Richard Stirling, please be takink note.

[lglethal]

I cant believe it...

That actually sounded like a reasonable and well thought out response from a senior civil servant!

How could this be allowed to happen? I mean if this sort of thing keeps up we might actually have some civil servants that we can trust! I demand this man be sacked for bringing the service into disrepute by actually knowing what he's talking about...

[Number6]

Understanding

It doesn't matter too much if most people don't understand the data, there will be enough people around who are prepared to interpret it who are not seen as government lackeys. The important thing is that it is freely available.

[Anonymous Coward]

A couple of important principles?

How about:

1 - full dataset of Raw and/or ordered data are available to the public

(advantages: independent data analysis or commissioned commercial analysis on publicly funded data; universities or other training establishments to run courses incorporating analysis of real data where appropriate)

2 - summarised form of data presented

(advantage: data analysis by ONS available for scrutiny when/where required)

There are two main strands - possibly three:

the source data

analysis compiled on source data

decisions influenced or based on analysis of source data

[Anonymous Coward]

About right

Just remember to include in 1 all data about collection methodology etc. that's required to make a proper analysis of the data.

[Ross 7]

Bit like OSS

I can see the guys point - why spend public money putting raw data out there that very few people can really understand and be bothered to take the time to understand? That money has to come from somewhere.

On the other hand openess is good. Seems like they are looking to be open but aren't sure of the best way to go about it. Which is a good thing!

Cost permitting releasing both together would seem the best solution - their work can easily be checked by anyone with the relevant skill set and time on their hands, but those people for whom stats is an arcane and interminably boring art can just download the summary, knowing that it has been scrutinised.

Kinda like OSS really - some guys check the src, but most ppl just download the bin from the official site and rely on the few to check it's all fine and dandy.

[Gannon (J.) Dick]

The biggest problem

"It ain't what you don't know, it's what you know that ain't so." -- Samuel Clemens (Mark Twain)

[Sirius Lee]

Keep on keeping on

The alternative of publishing summaries is frought with dangers. What was the methodology behind the analysis? Were the most appropriate statistical methods used? Who says so?

I say keep that raw information coming. Once its out, its out forever and and be checked retrospectively.

Despite the reservations, its cheaper too. A process to publish raw data can be created and run month after month, year after year. An analysis requires consideration, discussion and review, committees to agree the findings, etc. All very costly.