back to article Why is information delivery so bloody hard?

Are you frustrated with how difficult it can be to find information? What makes this more irksome is that the vendor community seems to have just woken up to the fact that there's a lot of information 'out there' to manage. Translated, this means that some of the larger vendors have recently got their act together with …


This topic is closed for new posts.
  1. Goubert

    Cri di coeur

    should be "Cri du coeur".

    Unless it's a pun, in which case, don't mind me, I'm on my way.

    Mine's the one with the french grammar text book in the pocket.

  2. Robert Ramsay

    Why is information delivery so bloody hard?

    Because you have to have people at the other end who want to/can understand it. This kind of person is sadly lacking.

  3. Gareth


    I'd say it's so hard to manage information because digitization and computer management is the obvious solution to the problem and yet (and excuse me while I whip out my broad brush) most upper managers, the very people who hold/grip/hide and could use the information are the kind of people who, when presented with a sophisticated database of 3,000 pdfs, all neatly collated and OCR'd, would ask their secretary to print them all out for him because he "can't be bothered with reading all of that on a screen."

  4. Anonymous Coward

    Problems are twofold

    I work in a County Council. First problem is information sharing, it doesn't happen. Not "important" information regards children at risk etc, but more mundane things.

    Things like where people are and what training they have had. Where buildings are and which departments work in them. If an IMS was to be purchased, the Council would likely end up with 9 different IMS systems, all bought independantly by each department without the knowledge of anyone else, all incompatible with each other.

    And then the central human resources department will pipe up that they had purchased the Council a Standard IMS last year but had kept it secret.

    The second problem is that many IMS providers get into trouble if the customer wishes to have a system cutomised to their needs. We have found IMS providers and other software system vendors seem to have poor quality and continuity control and nobody who can actually write proper code that works.

    Information delivery is hard because the expertise is lacking. IMHO.

  5. Dave Harris

    Not that hard

    What do we know? Who knows that we know it?

    Easy in our place. We have replicated databases of our corporate knowledge in each of our locations. And we have a state of the art IM system through which we can find the relevant expert on any topic, even to the point of a skill tap to find out who the relevant expert is.

    Gah, you don't need me to tell you what it is, just to say that BillG likes it, or at least respects it so much, he hired the guy that invented it.

    And it comes in Yellow ( )

  6. Graham Orr
    Gates Horns

    Data. You mean the collation and delivery of data.

    You are discussing the collection and dissemination of data, not information. Information is what human beings create for themselves - at a given moment in time and for no one else - from the interpretation of data. When you open your gob and speak all you are sending is data, which will be interpreted by the recipient into their personal form of understanding. Ditto when you write an article to appear on the sheet/screen. Were this not so then the word 'misunderstood' would have no meaning.

    Actually, pushing the envelope of misunderstanding toward the end of the day where the ducks line up in a row against a clear blue sky of lateral thinking, there is no such thing as 'misunderstanding'. One always understands the data perfectly; either perfectly in the belief that the information is understood or perfectly in the belief that the data generates some information or perfectly in the belief that the data is unable to be interpreted as information.

    Oh and of course, information is only useful 'in this place, at this time, with these people, with these things'. Change any of these parameters and the information requirement differs. You can't store information, you can only generate and use it. Of course this viewpoint means that knowledge is not information but merely data, able to be recalled by a human (or possibly other sentient beings, say the dolphins) to be applied in the generation of information with which to act in a given situation.

    As for the untidy's called the internet.

    Bill, cos he knows 640kb is enough RAM for anyone......but sends a different message

  7. Alan Esworthy

    depends on nature of question asked

    The company I work for does a very good job of delivering information as long as it is directly doing production work or planning changes to or growth of that work. For so-called ad hoc information requests, on the other hand, or for one-off analytical undertakings, things get a bit more difficult. What has developed, and has done so largely without deliberate decisions, is a smallish crew of highly knowledgeable people who can turn oceans of data into the information needed.

    This is a bit risky. I am one of the smallish crew and if I were run over by a beer truck it would take at least weeks if not months to find the right replacement and get him up to speed. In the meantime a few dozen highly-trained (and paid) business planners, mathematical modelers, and statisticians would be very significantly less productive.

    (When I say "oceans of data" I really mean it: among other things I manage to receive, process, and store...let's call them "event records" the tune of about 100,000,000 per day.)

  8. Anonymous Coward

    Don't even go there

    We had to wait a while earlier this year whilst all of our data providers (banks) reviewed the security on their data provision, one (who used to send over ftp) now sends us data encrypted CD's via courier with the password on a post it inside the package.

  9. Matthew Ellen


    Learn to proof read.

    I was the weird kid whose room was a pig sty, but knew perfect well where everything was. Luckily, with age, I have come to accept that I'm not the only one who needs know where my stuff is.

  10. Pierre

    "information management solutions"

    This expression completed my BS Bingo grid, so I stopped reading there, obviously.

  11. Kevin Bailey

    Will we ever be able to get it right?

    We've tried to move from paper based to computerised systems in a couple of generations.

    And it takes something really special to get it right.

    The current situation isn't good enough; Offices full of crashing PC's, spreadsheets and amateur Access databases full of errors, shit and chaos all the way up to the big stuff.

    Sure, we can get some tech stuff right - but it could be *so* much better.

    So - will we ever be able to get it right?

    I feel we need to make one important change. All software should be open source.

    The difference is between a world full of crashing Windows PC's using badly written and bloated software from Visual Studio.

    Or rock solid Linux PC's accessing LAMP/MVC applications.

  12. Jeff Wolfers


    Because it's a million separate problems, each with it's own domain space and use cases.

    What question are you really asking??

  13. Anonymous Coward
    Thumb Up


    Nice distinction data / information.

    This article really is about data, though I find information is at least as interesting as data.

    I don't absolutely agree with you over 'misunderstanding'. Don't take it personally, but treating understanding as a logic problem, no matter how complex, is too crude. I read your post several times and am sure that I got some, not all, of what you thought you meant (as you will of mine - if you read it).

    - Claude Shannon

  14. Anonymous Coward

    It's marketing people

    99.9% of marketing and sales people will put data into an Excel spreadsheet period. If you show them a copy to Crystal Reports they run away or say it's NOT simple, and ask the IT dept to fix it. Their time is too precious for "training".

    When the IT dept suggests a CRM the whole company (except IT dept), or just the top two directors will then vote on which CRM to use. Generally none of them understand CRM or database projects, they hire consultants who just repeat what has been said in some textbook, and in the end some idiot decides to vote themselves project manager and make correct decision as nobody knows what they are doing. (<PISSTAKE COMMENT>Of course anybody apart from the IT department or IT developer will become the project manager, as they know nothing about this sort of thing, generally it's the latest YTS trainee</PISSTAKE COMMENT>).

    Years later, when they have accumulated 10,000 spreadsheets or MS Access databases they will eventually encounter a shiny database salesman who will sell them yet another Microsoft Access database who will port everything in, which eventually grinds to a halts and crashes every other day. The IT Department will then be commanded to "fix it".

    ... and round and round it goes. People muddle themselves, not data.

    Then again if a company has a IT Director or an IT manager that has any control, that's knows their salt, that convinces people that they need training (or has an understanding boss who tell people they need training), then the job just gets done.

    I'm not bitter!!!

  15. Graham Orr
    Thumb Up

    @anon coward aka Claude Shannon

    I cannot agree that my use of the term ‘understand’, in terms of data, is not fit for purpose. How else is understanding to be treated if not a matter of logical deduction or inspirational (perhaps not in many cases) induction?

    As I stated earlier there are three possibilities; you are right, this distinction is crude. And I delight in its simplicity – that is what you meant is it not? You did not mean rough or unfinished by any chance? <grin> There still remain three possibilities: belief that the data can form information, belief that the data can form some element of information and the belief that the data forms no information. The second possibility clearly implies that some of the data may not be intelligible whilst some has formed information in the mind of the creator. And you will have realised that I hold belief = knowledge. Is that what you alluded to – the notion that belief is not knowledge?

    To relate these three possibilities to the subject of this forum, it looks to me that the IT jocks favour the first, pragmatic managers look to the second and harassed workers firmly follow the third when consideration is given to the use of data management tools. Users too busy to use/enter data on a data management system ought to be lionised, not pilloried, for the truth is that they are the information system. The distain often shown reflects apparent lack of applicability of, or lack of intelligibility of, the data held in the data repository (book, file, computer etc). I use the term apparent deliberately, there must be cases where the data repository does reflect the needs of the user. But then, as has been said, SOPs are for the guidance of wise men and the obedience of fools.

    Organisations get more out of training their employees to think than in foisting a Taylorist notion of ‘efficiency’ in the guise of a data repository with menacing electrons.

    Thumbs up cos you made me think.

  16. Stewart Haywood

    We are getting worse.

    Many years ago a company of any size had something called a registry. This was run by a registrar. This is where all the company data and information was stored. Any project/job/activity generated data in one form or another and this was stored in the registry. Every project/job/activity had time, hence money, allocated to producing the final reports and data. The registrar made sure that this all worked and that all the data was correctly referenced etc etc.

    Along came electronic storage and the firm belief that it would all take care of itself. It doesn't. It doesn't matter how clever or automated the system is, it requires a responsible human being to make sure that it works and that everyone behaves correctly. This of course costs money. Try persuading accountants that you need a registrar or a librarian to look after company data and information when it is all stored on hard drives and not in filing cabinets or on book shelves. The storage is invariably a mess and difficult to search.

    Searching data and generating information from it takes time, skill and a knowledge of the subject matter. Many years ago I remember a young engineer being sent from one department to another to find some urgently needed sky hooks. He was gone for a whole day before he came back to ask what a sky hook looked like. My wife tells me that young nurses were sent to the stores for a long stand. Companies are now full of people doing just this but to afraid or stupid to ask a person the all important question. The point is, if people are searching for something that doesn't exist or that they have no hope of recognising when they do find it, any system will fail.

    So, we need people to make sure that the filing and storage works and follows a set of rules. We also need the old grey hairs that can tell people what they are really looking for and guide them in the right dirrection. If we didn't need this we wouldn't need librarians and index systems in libraries and we wouldn't need teachers. Just pile the books up in a room, give the students the final exam papers and say go for it, you've got three years.

  17. Matt W

    We have all your data right here

    But information requires undertsanding, and a knowldedge of business drivers and indicators. Anyone can do a simple project report, but realistic and useful reports, i.e. information, require people with knowledge and undertsanding to process the data.

    It's hard because you need people who have those things, are articulate, and handy with query/report tools. They should have a deep knowledge of the business, some finance and IT literacy. Even then, they may get let down if management doesn't know what it wants.

  18. Alex

    Darwin did it

    SME's and even Blue Chip firms organically acquire systems and retain the most effective for business purposes which are not necessarily scalable with the growth of activity using theses systems.

    Cutting the BS. systems evolve: someone somewhere sees a problem which an information system could solve and having attended the company events listened to the geeks no sorry IT dept (my cupboard) and read the management memo that says stop tell the IT dept so we can spec develop and deploy it to a production environment with resilience, performance scalability and interoperability, proceeds to spend twice as long as a dev team evolving the business by knocking out an excel vba or access system which cant grow with the business or talk to the oracle database used elsewhere in the business because of how they hacked sorry developed it and then IT are told to fix it. Its not just marketing people ;) further complicating the issue is the fact the big firms have various separate business depts each with their own IT and thus various Operating systems and development patterns... so when we are told to take a joined up approach we look at the continental plug and the British socket scratch our heads reach for the converter leave it in on loads its not designed for and moan when it catches fire because the budget to start again with a unified approach does not exist.

  19. Andrew
    IT Angle


    There is a qualitative difference between reasonable understanding and perfect understanding. If the latter implies a data source containing no signal, semantic content or whatever, then it is pointless (unfalsifiable) to say the data has been understood. Perhaps your belief / knowledge fig-leaf did cover up the original sin, but I believe you were just messing around with all that stuff.

    People, like the grey-beards, usually have little trouble interpreting data to a reasonable degree of understanding, or fidelity to the intent of the originator. And some do better in general or on specific tasks than others. This could be tested empirically, but it is surely intuitive anyhow.

    What we need is systems that can do as good a job as those folks, but on the larger quantities of disparate data that are now prevalent in the workplace making it unfeasible to manage everything using dedicated or even time-sliced expert staff. Getting a good schema is the tricky part. I propose that a schema must emerge out of ordinary use of data and its interpretation to information. Entropy, not logic, will be at the core of such a system.

    IT? stands for Information Theory ?

  20. Graham Orr
    Thumb Up


    There you go now see? There's a perfect example of the second possibility in your first paragraph. I understand all of the words and believe, having interpreted the data, that it forms some element of information. Perhaps you missed the point of the third possibility: if a source of data is not able to be interpreted as information THAT is the perfectly understood element: it does not contain disernable information. Where the fig leaf comes from is a mystery unless your allusion is to the Garden of Eden in the belief that belief must, of necessity, relate to the numinous. My definition of knowledge is a source of data, personally held, which can be recalled either voluntarily (Polanyi's explicit dimension) or involuntarily (the tacit dimension) to form information when combined with external (to me) data. Belief is what I think I know (explicit) and it is not what I do not know I know (tacit). And I know I am scared of spiders but have no real idea why.

    Most, if not all, 'knowledge management' (KM) projects are a mixture of tidying up the messy boy's room and the wish to control the workforce: If only we knew what THEY know to paraphrase Polanyi. But since the World is a constantly changing environment such an aim is the equivalent to trying to nail down a swarm of eels; might get a couple but the rest.........

    People, not machines, can deal with black, white and shades of grey in terms of data interpretation: machines deal in black and white only - when was the last (first!) time a computer gave 'maybe' or 'depends' as the result of a query? Oh and fuzzy logic does not allow a machine to deal in shades of grey, it just means slice the black and white up faster and into smaller bits (or should that be take smaller bytes faster?).

    It is a great pity that Data Technology was not used to define computers instead of Information Technology, after all the automobile is not commonly described as Journey Technology. Or am I boxing myself in a corner here - after all the data technology does allow a human to create information <grin> Ah but wait, JT does not come ready packaged with predetermined routes, only a KM system trys to achieve that and, if successful, places the organisation on rails.

    Perhaps we ought to extend the analogy: data technology is an essential tool (automobile) that allows an organisation to go from A to B without necessarily prescribing the route; KM on the other hand is a railway; in many ways more efficient than automobile (more can get there for less cost eventually, once development expenditures are absorbed) but with less flexibility. Organisations are probably better advised to develop the abilities of their staff to think rather than spend money on defeating the natural disorder of things.

    BTW if the data source contains no signal it is not a data source. But you knew that <grin>

    Thumbs Up cos you also made me think.

  21. Andrew
    Thumb Up


    We seem to be at cross-purposes, but enjoying ourselves nonetheless. Probably nobody else is reading this thread by now.

    I may be missing something, but I keep coming back to the (mis?) understanding that all three of your exclusive and complete alternatives neglect to reference anything of the sender's intent or meaning. What I meant by signal in the data. your three possibilities only seem to reference the receiver's understanding of what they think they have received in the data. If this is a correct characterisation of your argument, I cannot agree with your definition of understanding. Indeed it is so patently wrong (to me) that I really did think you were just joking around. I do not believe that now.

    Perhaps you are not familiar with the work of Shannon, dubbed Information Theory. Entropy is not fuzzy logic, but it does describe a kind of grey. Entropy is actually uncertainty. The lower its value the less uncertainty the receiver should in truth hold about the data being signal rather than noise or vice versa. It was originally described in terms of a communications pipe (coming from Bell Labs), but I believe it can apply to any application of information transfer. We agree on most of the other things you've written here, so if we can get over my second paragraph in this post then you'll probably also see why entropy is applicable to this thread. Of course, you might see dealing with uncertainty as dancing with the devil. BTW don't read more into my allusions than common English idiom.

    I am slightly less cynical about the intention of organisations to want to do information / data / knowledge management. This may be just down to our different workplace experiences.

    On the whole it is human nature to accentuate differences. One thing that you and I and Polanyi seem to have in common though is the importance of the personal or subjective perspective. Though for you, this seems to be be all important, whereas I do believe communication is a two person (or machine) activity in which something real and valuable is transferred. Information is an artifact of both communication and (as you rightly said earlier) of interpretation.

    A simple but useful place to establish agreement might be over the term 'exformation'.

    - Andrew AKA Claude Shannon

  22. Graham Orr
    Thumb Up


    Interesting. Yesterday I sat, in my office in the cellar, coffee at the ready, fag dribbling ash and smoke, and commented, aloud, upon your argument, offering counter argument.

    Regarding understanding of a message sent being only in the gift of the recipent, and this is an extreme example, I think I rest my case.

    But let us return, if you wish, to the more mundane aspects of the original article predicated upon the notion that if one corrals the data all will be well. You sound like someone who has had some experience in this area. My own experience is limited to a number of middling to large projects which I found to be useful in their beginning but which quickly atrophied into the 'system', never to be tinkered with, or the 'system' to be circumvented at every opportunity. It really seemed to depend upon for whom the electronic bell tolls - the less autonomy allowed to the user the greater the benefit (SOPs being for the obedience of fools and the guidance of wise men - you will understand of course that I do not really mean all that obey are fools - it is just an idiom).

    As Bacon is alleged to have stated: Knowledge is Power. So a KM system is doomed from the start, particularly in an economic crisis, if what I know is the measure of my personal indispensability. However if what is being discussed is a better way of making data available - even something simple, say a bus timetable - then data management systems can add value if they are used and, somewhat perversely, also destroy value. Ever been at the receiving end of a script-constrained call-centre operator with a barely comprehensible Indian accent? And why was that Little Britain sketch about the mortgage agent ("Computer says no") so funny?

    It has been my experience that such a Taylorist approach demands a Lean environment with regard to the generation and passage of 'information' (the common understanding of the term). Left to themselves people will use a communication system for all sorts of things that merely create 'noise' - how many emails have you had recently about Sarah's new baby? A KM regime often demands a regimented approach to the allowable use of data distribution media - with some companies banning the use of email altogether and demanding that important matters be consigned to paper since so much time was being 'wasted' sending inconsequential (in terms of the business at hand) messages.

    I would be interested in your views on how data management systems can, or cannot, add value to an organisation. Or we can let the matter rest and I have enjoyed reading your arguments and look forward to reading more of what you post here, or elsewhere on the website.



  23. Andrew

    last post for me


    Agreed (to disagree). It is also time for me personally to switch the same gears in a more significant way. For the past couple of years, I have been thinking and working a great deal on Information Management in the abstract and I am currently in process of finishing a prototype to demonstrate a key principle that I want to proselytise. From here, I need to be focusing more on how to apply it to subject matter domain(s).

    You are right that I do have some experience, though latterly more as a generalist IT manager, than as a specialist KM consultant type. I have found that there are inherent tensions that a designer needs to acknowledge and work. The weight of any factor may change through a system's adoption life-cycle. As I reflect though, I find I cannot adequately summarise these. Here's an incomplete list:

    - predominant user types (early adopters; heavy SOP users; business managers)

    - user status (internal / partner / external)

    - homogeneity of users' language / business culture / social graphs

    - Scale of data / number of discrete sources / granularity of content / language

    - known unknowns / unknown unknowns

    - business criticality of content

    - homogeneity of data sources and content

    - Organisation's culture w.r.t. information (willingness to adopt new practices; functional/regional/business line demarcation; command and control / bottom up; acceptance of external data sources).

    Although I do agree with the adage about SOP, I also find it parochial. It applies to me and to many people I know. On the other hand, it seems to be culturally specific. I have worked in Japan and there I found the adage may hold but only in exceptional cases . It feels more like noise than signal in Japan, whereas in the UK the converse seems true. This is just anecdotal and subjective, but you asked for my opinion, so there you have it.

    - Andrew

This topic is closed for new posts.