back to article Real-time crowdsourced fact checking not really that effective, study says

Social media companies have proposed enlisting their respective audiences to catch the misinformation they distribute, or are already doing so. Facebook, now living under the assumed name Meta for its own protection, says, "We identify potential misinformation using signals, like feedback from people on Facebook, and surface …


    1. Eclectic Man Silver badge


      The same phenomenon was observed in fish by Konrad Lorentz. In his book 'On aggression'* he found that small fish which swim in shoals and are generally averse to risk (i.e., going into open areas) would follow a fish that had been operated on to remove part of its brain. This fish would just dart off to wherever it saw something interesting, without inhibition, and the rest would follow.

      Now, please note, that I am NOT in any way suggesting that your 'friend' is either brain damaged or stupid...

      * ISBN 0-415-13659-8. See also

  1. A Non e-mouse Silver badge

    Facts and truth

    The problem with moderating facts is that there are rarely yes/no evaluations. They usually have an element of truth about them but are stretched to breaking point. And you also open up a huge philisophical discussion about the difference between facts and truth.

  2. Trollslayer

    Does the word...

    Mobs mean anything?

    Icon is when someone gets burned at the steak.

    1. Jimmy2Cows Silver badge

      Re: Does the word...

      Mmmm... steak.... I'll have mine rare, with a rich peppercorn sauce, please.

  3. Twanky Silver badge

    Professional Fact Checker

    So, how do I get recruited? If we're 'professional' is there a governing body? Who selects the governing body? If you already have accreditation can you go freelance? What is the accepted ISO 9000 procedure for resolving a situation when two academics (history, science, language, whatever) disagree?


    1. jake Silver badge

      Re: Professional Fact Checker

      ISO 9000 just means you have fully documented that your documentation is documented.

      I always hated ISO 9000 ... chasing paperwork in the name of mediocrity.

      1. Twanky Silver badge

        Re: Professional Fact Checker

        Fair point.

        Perhaps a professional ISO 9000 consultant (insultant?) could verify that the Fact Checkers procedures have been correctly documented and checked for checking of facts against announced documented procedures?

  4. Anonymous Coward
    Anonymous Coward

    It's simply about avoiding costs and liability

    By letting users flag posts, the companies have a twofer. On the one hand, they get more information without having to pay people money to do the job, and on the other hand it acts as an excellent tool to ward off liability because they are doing "something" whereas the very use of this mechanism indicated they do not care one hoot about the actual outcome.

  5. Doug 14

    An aphorism for our times

    You are about as likely to find wisdom in a crowd as you are to find justice in a mob.

  6. yetanotheraoc Silver badge

    Wild extrapolation

    "... the whole idea of the 'wisdom of the crowds' literature has tended to be that you can aggregate across people who are not particularly good at a task (like guessing the weight of a cow) and, despite lots of variation, get a very good guess on average."

    So averaging independent guesses of a discrete number about a non-emotional objective fact leads to a reasonable approximation of the number, albeit with a huge variance. And how exactly does this relate to crowdsourcing of guessing about misinformation on social media?

    Still, tip of the hat to the researchers. "We knew it was bollocks, but we had to do the study anyway." Because it's not ranting that defeats bollocks, but facts.

  7. martinusher Silver badge

    History suggest that mob justice never works as intended

    Crowd sourcing is this year's version of "the mob" (small 'm'). We know from history that when mob justice prevails it invariably results in mob injustice -- the loudest, most strident, voices prevail, invariably drowning out quieter voices. This is likely due to it being a whole lot easier to appeal to emotions than intellect, especially if people are conscious of how to trigger and steer those emotions.

    Wikipedia has had a constant struggle with crowd sourcing of facts. This is easy when the facts are how to wire a RJ-45, material that is 'non controversial'. Once you cross a line towards opinion, maybe history, maybe biography, anything that's potentially controversial, then 'facts' become a whole lot more fluid and so editing has to be tightly controlled. Most of the facts we are thinking about with this crowd sourced checking are these 'matter of opinion' type facts so crowd sourcing is inherently unreliable, especially if professional reputation manager types are involved.

  8. david1024

    In other news...

    Also... Angry mobs sometimes 'get it wrong' too

  9. msobkow Silver badge

    How can you expect crowd sourced fact-checking to work when most of the crowd knows NOTHING about the facts and would often do ANYTHING to avoid learning them and upsetting their world views?

  10. amanfromMars 1 Silver badge

    Nation Shall Speak Peace Unto Nation via Remote Augmented Virtual Reality Control/AI Controllers

    In other words, with peopled peer groups in command and control of communications and computers .... and please excuse my French ..... Vous êtes fcukd .... and especially well so whenever defending the indefensible and inequitable and exclusively self-serving and despicable which is so terribly human and ubiquitous.

    And there y'all are, worrying yourselves senseless and terrifying all around you with wild speculation and constant fear-mongering about virtual machines taking over the decision making processes and presenting somewhat different and more accurate alternative and/or alternating facts crazily thought to be no more truthful than novel and noble fiction for practical realisation via the myriad multi-media channels and tunnels/programs and tubes provided to you for prime utilisation/leading facilitation ..... whenever your own serial efforts are so manifestly undeniably dire and destined to prove themselves worse than was ever thought possible before should there be no discernible fundamental improvements.

    Do you deny that simple observation and would you wish to defend it and claim it your right to propagate and maintain/retain and continue to seed and feed?????

  11. TheMeerkat Bronze badge

    Any “fact check” is just a form of censorship.

    The difference between “professional” and “crowd sourcing” is that in the firmer case it is the person who pays “fact checkers” who decides what is “true” and what is “false”, while with crowdsourcing it is just a majority opinion

    1. Jimmy2Cows Silver badge

      What about fact checks that confirm your viewpoint?

      Seems like the people who complain most about "censorship" are the ones whose "facts" invariably turn out incorrect when challenged.

      1. TheMeerkat Bronze badge

        “ turn out incorrect when challenged”

        We know perfectly well what king of opinion would “turn out to be incorrect” when challenged by government “fact checkers” in China.

        Accusation of “spreading false information” is the standard way dictatorships shut down opposition.

        1. Swarthy Silver badge

          On the other hand, the spreading of misinformation is the standard way for dictatorships to come into power.

          The Dilution of Truth (calling truth lies, calling lies truth, applying 'spin' and dubious interpretations to things that are observable) is the prime tool of those who wish to control thoughts.

          It seems to me the balance of these points might have something to do with a popular quote about "eternal vigilance" and it's relation to liberty.

      2. amanfromMars 1 Silver badge

        Fortunately, Leading Things see Most Everything Quite Differently

        What about fact checks that confirm your viewpoint? ..... Jimmy2Cows

        FailSafe Secure Safe Sound Advice then and there is always best most wisely to treat it as only possible as a temporary affair which appears to support and reinforce any stated opinion/leading momentary position ....... for both views and viewpoints mature and develop and can easily most radically change and be changed.

        To some who may be a few, and maybe also more than just a great many more with almighty powers to share and demonstrate/donate and champion, is that a constant worry and overwhelming human problem taking over and overtaking All Current Events?

        And that question goes to Glasgow COP26 to answer without any of that traditional babble and psychotic mumbo jumbo in reply to try mask an ignorance and indifference to be exploited and strengthened against wilful and wanton unnecessary defences.

        RSVP c/o El Reg for Everyone to know whether you know, or not ...... as the case can surely be whenever displaying Full Transparency. ........ for NEUKlearer HyperRadioProaCTive IT Vision with Blueprints for the Creation of Model Environments and Heavenly Spaces and Diabolical Places ..... a Hellishly Engaging and Almighty Addictive and Attractive Task to Test Both the Saint and the Sinner/Normally Very Good and/or Oft Terribly Bad alike in Equal Great Measures.

        Fair’s Fair ..... what’s good for the goose is good for the gander and very quickly highlights any obvious simple problems to address and correct whenever it is so easily made so ...... or not so should such FailSafe Secure Safe Sound Advice not be applied and deployed in those particular instances/peculiar places.

        1. amanfromMars 1 Silver badge

          Re: Fortunately, Leading Things see Most Everything Quite Differently

          And you know that all makes perfect common sense and reveals an abiding current situation from which there is no escape should just very much more of the same old nonsense be the collective answer to try to fool the masses in a fake guise for media to paint and print as being earnest progress.

          The emerging and rapidly expanding problem then/now for all existing conventional, hierarchical and evolutionary systems administrations though is, many relatively anonymous and quite autonomous in the masses are in possession of a much greater intelligence and are no longer able to be so easily fooled and enslaved/commanded and controlled ....... and they would be inclined to consider the exercise of sweet natural justice with side orders and desserts of relentless retribution as a logical course of communally supported action against any and all so discovered and uncovered to be actively engaged in their continued suppression and wanton persecution ...... and especially so now that such information on advanced intelligence services has been freely shared for greater universal disclosure.

          Take Care Out There. IT’s an Unforgiving Jungle which takes No Ghastly Prisoners.

    2. Eclectic Man Silver badge


      Would it be very wrong of me to point out that when you typed "firmer" you probably meant "former"? Or is that too close to 'fact checking pedantry'?

      I also disagree with your statement that the person who pays the fact checkers decides what is true. This is not the case. 'There Truth' as they say, is 'out there'. The fact checkers review statements to determine whether they accord with reality or verified and trusted recorded tests, and report on the results. I suggest that, if you can, you listen to BBC Radio 4's 'More or Less' radio program (available on the BBC Sounds feature on the web site) for some explanation of how they check statements for accuracy.

  12. captain veg Silver badge

    oh dear

    So Andrew Orlowski was right about something.



  13. Jellied Eel Silver badge

    Oh dear..

    ...especially before fact-checking sites like Snopes or PolitiFact have published their evaluations.

    But what if those are unreliable or biased sources? Both have got their facts wrong plenty of times in the past. It's a shame the paper doesn't list the sites it considered unreliable news sites. Would CNN or Fox be included? I may have missed the data, but it would have been nice to see the sites chosen, along with the stories. Partly so I could play at home, partly because in politics, facts aren't always what they seem to be.

    It does mention Covid, which was interesting because in the very early days, 'facts' were being 'checked' when it was highly unlikely the fact-checking services could have actually verified the facts. Some of which are stil unknown, ie the origin, or fun subjects like vaccine effectiveness. There is a reference to another paper, which has this in it's SI-

    With a couple of interesting inclusions, like the Express. Admittedly I haven't read an article from them in a few years. Also Zerohedge, which publishes a mix of facts and opinions. Sometimes it can be hard to figure out which is which, but such is the Internet. A lot of people on the left really don't seem to like ZH though. Then there's good'ol Infowars...

    But I'm happy to say I've never heard of a lot of the sites on that list.

    1. amanfromMars 1 Silver badge

      Re: Oh dear.. what have we just gone and done ‽ .

      In addition to their actions targeting misinformation, Facebook has taken a broader range of steps to reduce the circulation of clickbait and sensationalistic news (even if not false), and these actions may also affect the circulation of content ......

      Heaven forbid that you should learn of anything sensationalistic and true is the stuff of fascism that tickles a Goebbels fancy and gives rise to grand delusions of being able to wield exclusive elite executive command and control to power and energy and thus are you self-identified as a legitimate target of ESPecial Force Interests ...... and quite rightly so too.

      Heaven forbid that honesty ever be considered worthy of cloaking and covering in FUD. Imagine the Madness and Mayhem, CHAOS and Conflict unleashed to wreak IT's Epic Havocs then.

      1. Jellied Eel Silver badge

        Re: S/he who controls the facts controls the future

        Heaven forbid that you should learn of anything sensationalistic and true is the stuff of fascism that tickles a Goebbels fancy and gives rise to grand delusions of being able to wield exclusive elite executive command and control to power and energy

        Funny you should say that, power and energy being a topic of note as we slip into our renewable future. Here's a thing-

        Methane is responsible for a third of current warming from human activities - and it's one of the most potent greenhouse gases.

        Levels of methane in the atmosphere are continuing to rise.

        How is it emitted?

        Cows may spring to mind... but rice production and rubbish dumps are also to blame. Around 40% comes from natural sources such as wetlands, but the majority is now due to human activity.

        Which is a pretty bold claim from the fat-free Bbc. Cows may certainly spring to mind, when prompted, but wetlands? Isn't the EU restoring those, ie increasing them? Along with re-introducing the prospects of the ague to go with your Covid. Swamps around London got drained, ague/malaria was reduced, Now, we're building back better. Or something.

        Ok, so fact checking this claim is a bit more complex than just assuming the Bbc is a 'reliable source'. But as with CO2, CH4 is a natural process with some human contribution. Soil bacteria and simple organic decay produces far more than humans. This is in the IPCC's WG1, but still needs some care to dig into the facts behind that literature review.

        I guess for me, the most curious aspect of this report is that people actually try getting their news from FaceMeta, and not more reliable sources.

        1. jake Silver badge

          Re: S/he who controls the facts controls the future

          A "wetlands" in the North San Francisco Bay was "restored" a couple years ago, to great fanfare by the denizens of Novato. There was a huge song and dance about it. Now the very same green-and-granola people are bitching about the large swarms of salt-marsh mosquitoes eating them alive. And the smell of decaying plant matter and the odd fish whenever the water is low (it's tidal).

  14. Corporate Scum

    Will anyone make it this many pages in

    to read me asking if anyone else found it perfect that a Godel is listed on the paper? [probably no relation] I feel like the elder Kurt had some advice on the calculability of certain problems and statements of truth made on same.

    My guess is that this is right before the part where the managers of these projects start using interesting new phrases like "manufacturability issues". For those that noticed a funny red line on that one, it's not your spell checker, it's a predictive failure warning for future earnings. If you didn't see one, you might want to ask why your company is using the term so often, and who added it to your dictionary. That person is probably either valuable or a public menace.

    But hey, in this case it appears the fact that people "solving" this problem in industry are unaware of the contributions of a 115yo mathematician to their field kind of illustrates the core of the problem the paper addresses. Some problems aren't reliably or efficiently solvable by either crowdsourcing of current ML technology. Some problems require domain specific knowledge. Realtime content moderation embodies both.

    As some free advice to those considering making a career of this. It's harder than it looks... and you will fail.

    1. amanfromMars 1 Silver badge

      Re: Will anyone make it this many pages in

      Some problems aren't reliably or efficiently solvable by either crowdsourcing of current ML technology. Some problems require domain specific knowledge. Realtime content moderation embodies both.

      As some free advice to those considering making a career of this. It's harder than it looks... and you will fail. ..... Corporate Scum

      That is almost quite so and correct, for more than just a few can surprisingly succeed whereas corporate scum are always guaranteed to serially fail abysmally and/or spectacularly.

      Such is an important and valid distinction to be aware of so as not to be caught off guard and unprotected and undefended by services unable to recognise the critical systemic vulnerability being exploited and expanded upon and exported to regions and sysadmins elsewhere, both deep and dark webbed and bright and breezily trailed and tailed/remotely mentored and virtually monitored.

      1. Corporate Scum

        Re: Will anyone make it this many pages in

        ...Perfect. For this one brief moment, you are my plastic bag spinning in the breeze.


POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like