Posts by TheLioness

Biz prof disses Big Data as a fetish for info hoarders

'Big Data' not New Just Cheaper to Do

'Big data' (formerly and sometimes still known as database marketing on very large databases, [enterprise] data warehousing, combined with statistical analyses, no wait that's data mining, I mean analytics)... sigh! Every time a new buzzword comes up, it's usually because some vendor wants to put a new spin on their frankly old products and ideas to distinguish them from the crowd.

Not so long ago, because of the costs of storing and managing large volumes of data, we used to be very selective about which data we tried to collect. The continuing poor quality of much data collected in organisations makes that ever more important. Models built on this unstable mountain will be very unreliable indeed.

But, now that data storage costs have fallen (predictably in accord with Moore's Law), companies want to recklessly capture as much data on their customers as they can. I also believe it has something to do with what I call the Microsoft Market Effect, i.e. if Microsoft seriously enters a product market, it becomes a commodity (software licence costs fall and everyone wants one on their desktop).

As for keeping all data forever, this ignores the very real rules of Data Protection legislation (even in the US, Safe Harbour agreements often mean that large companies need to be cognizant of EU and other international laws) that forbid such a thing. Also, as the business evolves, very old data becomes non-representative of the current business. Retention requirements and archiving policy need to be evaluated on a careful (business) case-by-case basis, not simply because the storage/database vendors tell you can store as much as you want or at least more than your competitors ('mine is bigger than yours' springs to mind).

The real risk in all this is that as the fad grows, it exacerbates the demand for truly experienced technologists who have the gravitas to tell their decision makers, "No! This is how we do it the right way so that you spend less and get meaningful results quicker." If decision makers then really want to make automated decisions based on faulty data and even worse models, they risk turning loyal customers into sworn enemies--downward spiral begins. And oh, how I have seen this already beginning. Caveat emptor.

Queen unveils draft internet super-snoop bill - with clauses

Sign the E-Petition to Stop This Madness!

If you oppose this, please sign the official e-petition at: http://epetitions.direct.gov.uk/petitions/32400

There are at least two key parts to this 'draft' bill, both heinous IMHO:

(1) The surveillance/snooping/spying by the government listeners and the requirement for (UK) ISPs and mobile phone companies to make their log databases available for real-time snooping, supposedly with the caveat that Home Secretary Theresa May or a judge can sign a warrant to do so (certainly not comfortable with any Home Secretary being able to do this without having to pass a legal test). Note, only the body/content of the e-mail/call would not be real-time (if you believe government assurances).

(2) The entry and hearing of communications data evidence in secret closed courts (hence Justice Minister Ken Clarke's argument that the US is more likely to be willing to share secretly collected [via waterboarding at Guantanamo or rendition exercises?] evidence with the UK if it will not end up in open court.)

Also, they talk about "collection" and "retention" of these communications data details (header metadata if you will). What is the likelihood of the return of the giant central GCHQ database that the previous Labour government, and which, the Cameron-led Tories and Clegg-led Liberal Democrats opposed? How else would they be able to feasibly data mine through the massive volumes of data scattered across a rather diverse set of ISP logs and mobile phone company call details?

And how are they going to impose this on suppliers who store such data outside the UK, in countries where such rights to privacy are enshrined in their constitutions, e.g. Scandinavian countries? And what is the point of getting header data, unless you wish to

Lastly, if anyone has actually watched some of the MPs questioning IT industry executives in the Select Committees, you would realise what an abysmal lack of technical understanding these folks have. And we are to trust them to review any proposals?

The "terrorists" and "paedophiles" scare tactic is just that. Do not fall for it. Please sign the petition and let your voice be heard, before we allow then to take away our last remnants of privacy!

Hypocrisy Reigns Supreme

Note this is from the same government that is currently bemoaning the Freedom of Information Act. They can know every little personal detail about us, but we are not to know what they are up to with their special advisors, 'best men,' old school chums, and corporate donors.