Real-time crowdsourced fact checking not really that effective, study says Social media companies have proposed enlisting their respective audiences to catch the misinformation they distribute, or are already doing so. Facebook, now living under the assumed name Meta for its own protection, says, "We identify potential misinformation using signals, like feedback from people on Facebook, and surface …
> There is no wisdom of crowds
Came here to say just that. Those are contradictory terms, crowds are as stupid (panicky, aggressive, egoist) as their stupidest member. Take a bunch of intelligent, educated people, let them form a crowd, and suddenly you have a brainless group of bleating sheep prone to easily induced knee-jerk reactions. It's really frightening.
"Crowdsourcing" is just another word for shearing those sheep.
"The wisdom of crowds" is real, but it's very much abused by people who see a chance to save a few bucks or headaches.
If you ask a crowd to guess the weight of a cow, then average all their answers, the final result will probably be pretty good. (Not nearly as good as it would have been 100 years ago, when more people had first hand experience with cows, but still pretty good for all that.) But that's asking people to guess at a number (easy to average objectively), that can itself be objectively determined.
If you ask people to spot "fake news", not only is there no clearly agreed definition of what they're looking for or criteria for judgment, there's also no objective way to convert to numbers and average their input. The "wisdom of crowds" assumptions are not even being acknowledged, much less fulfilled.
In a test setting it works.
BUT, in a test setting the participants are all individual humans and can't talk to, much less influence, the other participants.
IRL, the participants may have multiple identities and may not even be human. They actively try to influence others and can bring in others to participate.
Won't work. Can't be made to work.
A couple more: As any sysadmin worth his salt knows, ONE person can be utterly brilliant. However, groups of people are universally fucking stupid. From direct observation, it would appear that a crowd is automatically brought down to the intelligence level of it's lowest common denominator.
That "large group of people" are definitely not the kind of people that spend their days tweeting to all and sundry...
Those that study Rocket Science are generally not part of the Twitterati Trollers Community ( TTC for short)...
So what you heard was correct, it's just that they are not in the same league as the TTCs whose average IQ fails to pass that of the associated blue bird that they hail.
No, that was Sopwith Camel singing Monkeys on the Moon, "It used to be so peaceful in space, now there's hard rock all over the place" :-)
It was a large group of people in aggregate, yes. However, the very large,complicated project was split into many, many much less complex components, each of which was worked on by a much smaller number of people. The vast majority of them never came into contact with the rest.
A mob it wasn't.
And the original phrase replied to, and quoted, was "However, groups of people are universally fucking stupid", and as one might plainly see, contains no qualifier suggesting it is somehow restricted to some subset of groups of people.
Not to "mobs". Not "people that spend their days tweeting". Not those "split [...] into a much smaller number of people". Not even "kneejerk responders to something which they like to assume was said but wasn't". Given the other content of the original post, one could probably argue for an IT support context, but, curiously, this is not one of the subsets chosen as worthy of mention.
In the original phrase, "Often fucking stupid", or "frequently fucking stupid", or similar, would have been fine.
"Universally fucking stupid", however, is flat wrong. As we would hope, I suppose, being members of a *group* of people who post comments in The Register's forums, and would presumably - but perhaps optimistically - consider ourselves not "fucking stupid". Of course, the reader may judge whether this exchange points to one conclusion or the other. :-)
Around 20 years ago I read a report of research into the motivations behind the popularity of crowd sourced knowledge. A surprising finding was, not that it was viewed as more reliable than recourse to experts, but that it was considered to be more 'democratic'.
However, as was pointed out almost two millennia ago by Marcus Aurelius Antoninus (AD 121 - 180) "The opinion of 10,000 men is of no value if none of them know anything about the subject"
However, as was pointed out almost two millennia ago by Marcus Aurelius Antoninus (AD 121 - 180) "The opinion of 10,000 men is of no value if none of them know anything about the subject"
Hence, the appalling results of so many democratic elections.
The study in the article makes it clear that the time is right for a globe spanning Ministry of Truth, nothing can be published on any platform until it has had the approval and relevant certification of truth from the ministry.
I am sure it would be easy to find staff who would religiously ascertain the truth or not of anything that has been submitted for approval.
Now, where did I put that sarcasm icon?
more 'democratic'
The reported fact-checking of fact-checking actually reminded me of the famous gedankenexperiment on democratic decision-making: pick a contentious issue, frame it as a yes/no question suitable for a vote, conduct 2 referendums: one among the general voter population or a representative sample thereof, another - within 1,000 (10,000, whatever) of the most knowledgeable (most intelligent, highest IQ, best educated - take your pick) members of the society, compare the results.
If the results are statistically similar the conclusion must be that knowledge/intelligence/etc. does not matter. Bummer.
If the results are significantly different then the conclusion must be that knowledge/intelligence/etc. does matter, but the democratic process negates the benefits. Ouch.
Looks applicable to fact-checking if you ask me - without additional research...
What about the media who actively disseminates fake news under the "freedom of speech" umbrella? You can deal with a thousand quackheads preaching the Earth is flat or COVID doesn't exist, but having a major media outlet throwing "alternative coverage" of some events without a real mean to contend their "view" is a lost battle from day 0.
They will continue to preach tho their choir and will even grow it bigger. For every single post acknowledging a "mistake" they are forced to bury deep in the belly of their websites, they are spitting a hundred more reinforcing their "view".
To the general media out there "Fake News" is irrelevant, what's important to them is generating income from the ads shown to all the users reading the posting, whether it's fake or accurate. In general you see far more users reading fake news than real news so the average news site makes a lot more from stupid people than smart people.
"from stupid people than smart people."
Exactlly, all that the Stupid People wan't is a life without complication, it is the main reason that they are so easilly lead... It's just unfortunate that those who are prepared to lead them are a bunch of hungry Wolfs that no longer even need to wear sheep's clothing...
"no longer even need to wear sheep's clothing..."
Just $6,000+ Armani suits (with Thom Browne loafers, of course). Funny how the ripped Levis and torn tee-shirt clad denizens (wearing Crocs, no doubt) are the folks most likely to follow them, isn't it?
Hey, Trump supporter[0]: Do you really think the filthy rich former idiot-in-chief would give you the time of day if he passed you in the street? Like hell he would. When was the last time you saw him at a tractor pull, drag race, swamp buggy race, or boggin' in the back country? Seriously, trailer trash, you are the last thing he wants to be the neighbor of ... so why are you a supporter? Glutton for punishment?
[0] Substitute televangelist or "conservative" radio host, or YouTube "creator" of your choice. I won't mind in the least. Every single one of 'em are ripping you blind ... and you are so brainwashed you think you like it! The mind absolutely boggles ...
> Glutton for punishment?
Father figure I guess? Some people always seem to need somebody to tell them what's what, and what they should do or think. Life is so much easier that way, and then there is also that warm comforting feeling of belonging...
Also let's not forget it, there is the prestige of "celebrities": Humans are always fascinated by them, no matter their real value/achievements.
Who said humans are rational? That's utterly ludicrous: There isn't the slightest sign for it, but plenty of solid proof for the contrary.
Forget about the media.
What about the governments ? Aside from the clear-cut duality US, UK and friends = all true, China, Russia = all fake, how should we know we're dealing with fake news ? Remember those WMD of the Gulf war, we had absolute and indubitable proof but in reality they were fabricated a.k.a. fake news.
As someone used to say, government is not hiding the truth, they replace it with other truths we also need to know.
Post a loaded item, be it news or opinion, and who will care enough to report it? Those who already have a chip on their shoulder about it, that's who. If 60% vote X, then news which puts X in a bad light will get reported more than news which puts it in a good light. Truth? I dunno, wasn't that a 2015 film about Dubya?
The study opens with: Misinformation spreads rapidly on social media, before professional fact checkers and media outlets have a chance to debunk false claims.
And near the end concludes: Both models performed best when only using the evaluations from those with high political knowledge.
I wonder what sort of "facts" are best evaluated by "those with high political knowledge"?
a) Science?
b) History?
c) Philosophy?
d) Propaganda?
I know this doesn't work. I watched a game show where the person asked the audience, why does stuff weigh less on the moon than earth. a) lower gravity b) low air pressure. Something like 80 percent picked the wrong answer, so the contestant went with the "wisdom of the crowd" and lost. I mean, even if they didn't know the answer, they use low pressure chambers for medical and other uses on earth and the stuff does not start floating away!
Stuff doesn't go "floating away" on the moon either - it's not negative gravity.
Barometric pressure *does* influence the effective weight of things on the earth - indeed, this is why helium balloons float away. (Of course, lower pressure leads to reduced upthrust and hence higher net weight).
So there's no need to be quite so dismissive.
It may also depend on exactly how the question was worded. Did the question imply that there was a "lesser type of gravity" on the moon? No: the gravitational constant is the same out there. It's just that the moon has less mass than the earth, whilst maintaining a similar density (x0.6)
Look at a platform like Reddit where the limits on someone’s ability to upvote or downvote are very light and you’ll frequently see discussions which are not remotely accurate if you have any knowledge about the field. Worse, a knowledgeable person who says something the crowd doesn’t like will get downvoted to oblivion. Crowdsourced moderation cannot work if there’s no qualification step required for someone to have an opinion. I remember working with people in the past who would have an strong opinion on my work and my reaction was “you’re not qualified to have an opinion” and that’s the way it is. Complicated matters cannot be judged by an uneducated crowd because the more complicated the question, the lower the odds that a majority will produce the right answer. Worse, these groups will push ideas that are attractive to them if it avoids facing the reality of bad things happening due to making the wrong choices. The internet is many things, but a filter for dumb ideas is not one of them.
The problem with probably down to the fact that people seem to want to go with the consensus rather than their own decision.
As an example (which is true), I arranged a day out for a car club, when we met up I told everyone the lunch stop was less than 1/4 mile from a junction.
So off we go with me in front, we get to the junction and it fed onto a dual carriageway, at this point I kept my speed down so as to turn off at the right point.
One saw an open stretch of road and decided to open up the car, out of the 16 cars in the group - most of them then followed. Myself and 3 others kept the speed down and got to the lunch stop, the rest of the group had to drive another 15 miles to get back as it was 7 miles to the next junction.
When they finally arrived I asked them why - one person didn’t realise why I was driving slowly and the others just followed him…
Sheeple!
So crowd sourcing facts, not a chance of it being accurate because people don’t want to be singled out because they think it is “wrong”
The same phenomenon was observed in fish by Konrad Lorentz. In his book 'On aggression'* he found that small fish which swim in shoals and are generally averse to risk (i.e., going into open areas) would follow a fish that had been operated on to remove part of its brain. This fish would just dart off to wherever it saw something interesting, without inhibition, and the rest would follow.
Now, please note, that I am NOT in any way suggesting that your 'friend' is either brain damaged or stupid...
* ISBN 0-415-13659-8. See also https://en.wikipedia.org/wiki/Konrad_Lorenz
So, how do I get recruited? If we're 'professional' is there a governing body? Who selects the governing body? If you already have accreditation can you go freelance? What is the accepted ISO 9000 procedure for resolving a situation when two academics (history, science, language, whatever) disagree?
FFS.
By letting users flag posts, the companies have a twofer. On the one hand, they get more information without having to pay people money to do the job, and on the other hand it acts as an excellent tool to ward off liability because they are doing "something" whereas the very use of this mechanism indicated they do not care one hoot about the actual outcome.
"... the whole idea of the 'wisdom of the crowds' literature has tended to be that you can aggregate across people who are not particularly good at a task (like guessing the weight of a cow) and, despite lots of variation, get a very good guess on average."
So averaging independent guesses of a discrete number about a non-emotional objective fact leads to a reasonable approximation of the number, albeit with a huge variance. And how exactly does this relate to crowdsourcing of guessing about misinformation on social media?
Still, tip of the hat to the researchers. "We knew it was bollocks, but we had to do the study anyway." Because it's not ranting that defeats bollocks, but facts.
Crowd sourcing is this year's version of "the mob" (small 'm'). We know from history that when mob justice prevails it invariably results in mob injustice -- the loudest, most strident, voices prevail, invariably drowning out quieter voices. This is likely due to it being a whole lot easier to appeal to emotions than intellect, especially if people are conscious of how to trigger and steer those emotions.
Wikipedia has had a constant struggle with crowd sourcing of facts. This is easy when the facts are how to wire a RJ-45, material that is 'non controversial'. Once you cross a line towards opinion, maybe history, maybe biography, anything that's potentially controversial, then 'facts' become a whole lot more fluid and so editing has to be tightly controlled. Most of the facts we are thinking about with this crowd sourced checking are these 'matter of opinion' type facts so crowd sourcing is inherently unreliable, especially if professional reputation manager types are involved.
In other words, with peopled peer groups in command and control of communications and computers .... and please excuse my French ..... Vous êtes fcukd .... and especially well so whenever defending the indefensible and inequitable and exclusively self-serving and despicable which is so terribly human and ubiquitous.
And there y'all are, worrying yourselves senseless and terrifying all around you with wild speculation and constant fear-mongering about virtual machines taking over the decision making processes and presenting somewhat different and more accurate alternative and/or alternating facts crazily thought to be no more truthful than novel and noble fiction for practical realisation via the myriad multi-media channels and tunnels/programs and tubes provided to you for prime utilisation/leading facilitation ..... whenever your own serial efforts are so manifestly undeniably dire and destined to prove themselves worse than was ever thought possible before should there be no discernible fundamental improvements.
Do you deny that simple observation and would you wish to defend it and claim it your right to propagate and maintain/retain and continue to seed and feed?????
On the other hand, the spreading of misinformation is the standard way for dictatorships to come into power.
The Dilution of Truth (calling truth lies, calling lies truth, applying 'spin' and dubious interpretations to things that are observable) is the prime tool of those who wish to control thoughts.
It seems to me the balance of these points might have something to do with a popular quote about "eternal vigilance" and it's relation to liberty.
What about fact checks that confirm your viewpoint? ..... Jimmy2Cows
FailSafe Secure Safe Sound Advice then and there is always best most wisely to treat it as only possible as a temporary affair which appears to support and reinforce any stated opinion/leading momentary position ....... for both views and viewpoints mature and develop and can easily most radically change and be changed.
To some who may be a few, and maybe also more than just a great many more with almighty powers to share and demonstrate/donate and champion, is that a constant worry and overwhelming human problem taking over and overtaking All Current Events?
And that question goes to Glasgow COP26 to answer without any of that traditional babble and psychotic mumbo jumbo in reply to try mask an ignorance and indifference to be exploited and strengthened against wilful and wanton unnecessary defences.
RSVP c/o El Reg for Everyone to know whether you know, or not ...... as the case can surely be whenever displaying Full Transparency. ........ for NEUKlearer HyperRadioProaCTive IT Vision with Blueprints for the Creation of Model Environments and Heavenly Spaces and Diabolical Places ..... a Hellishly Engaging and Almighty Addictive and Attractive Task to Test Both the Saint and the Sinner/Normally Very Good and/or Oft Terribly Bad alike in Equal Great Measures.
Fair’s Fair ..... what’s good for the goose is good for the gander and very quickly highlights any obvious simple problems to address and correct whenever it is so easily made so ...... or not so should such FailSafe Secure Safe Sound Advice not be applied and deployed in those particular instances/peculiar places.
And you know that all makes perfect common sense and reveals an abiding current situation from which there is no escape should just very much more of the same old nonsense be the collective answer to try to fool the masses in a fake guise for media to paint and print as being earnest progress.
The emerging and rapidly expanding problem then/now for all existing conventional, hierarchical and evolutionary systems administrations though is, many relatively anonymous and quite autonomous in the masses are in possession of a much greater intelligence and are no longer able to be so easily fooled and enslaved/commanded and controlled ....... and they would be inclined to consider the exercise of sweet natural justice with side orders and desserts of relentless retribution as a logical course of communally supported action against any and all so discovered and uncovered to be actively engaged in their continued suppression and wanton persecution ...... and especially so now that such information on advanced intelligence services has been freely shared for greater universal disclosure.
Take Care Out There. IT’s an Unforgiving Jungle which takes No Ghastly Prisoners.
Would it be very wrong of me to point out that when you typed "firmer" you probably meant "former"? Or is that too close to 'fact checking pedantry'?
I also disagree with your statement that the person who pays the fact checkers decides what is true. This is not the case. 'There Truth' as they say, is 'out there'. The fact checkers review statements to determine whether they accord with reality or verified and trusted recorded tests, and report on the results. I suggest that, if you can, you listen to BBC Radio 4's 'More or Less' radio program (available on the BBC Sounds feature on the bbc.co.uk web site) for some explanation of how they check statements for accuracy.
https://tsjournal.org/index.php/jots/article/view/15/6
...especially before fact-checking sites like Snopes or PolitiFact have published their evaluations.
But what if those are unreliable or biased sources? Both have got their facts wrong plenty of times in the past. It's a shame the paper doesn't list the sites it considered unreliable news sites. Would CNN or Fox be included? I may have missed the data, but it would have been nice to see the sites chosen, along with the stories. Partly so I could play at home, partly because in politics, facts aren't always what they seem to be.
It does mention Covid, which was interesting because in the very early days, 'facts' were being 'checked' when it was highly unlikely the fact-checking services could have actually verified the facts. Some of which are stil unknown, ie the origin, or fun subjects like vaccine effectiveness. There is a reference to another paper, which has this in it's SI-
https://journals.sagepub.com/doi/suppl/10.1177/2053168019848554/suppl_file/appendix.pdf
With a couple of interesting inclusions, like the Express. Admittedly I haven't read an article from them in a few years. Also Zerohedge, which publishes a mix of facts and opinions. Sometimes it can be hard to figure out which is which, but such is the Internet. A lot of people on the left really don't seem to like ZH though. Then there's good'ol Infowars...
But I'm happy to say I've never heard of a lot of the sites on that list.
In addition to their actions targeting misinformation, Facebook has taken a broader range of steps to reduce the circulation of clickbait and sensationalistic news (even if not false), and these actions may also affect the circulation of content ...... https://journals.sagepub.com/doi/suppl/10.1177/2053168019848554/suppl_file/appendix.pdf
Heaven forbid that you should learn of anything sensationalistic and true is the stuff of fascism that tickles a Goebbels fancy and gives rise to grand delusions of being able to wield exclusive elite executive command and control to power and energy and thus are you self-identified as a legitimate target of ESPecial Force Interests ...... and quite rightly so too.
Heaven forbid that honesty ever be considered worthy of cloaking and covering in FUD. Imagine the Madness and Mayhem, CHAOS and Conflict unleashed to wreak IT's Epic Havocs then.
Heaven forbid that you should learn of anything sensationalistic and true is the stuff of fascism that tickles a Goebbels fancy and gives rise to grand delusions of being able to wield exclusive elite executive command and control to power and energy
Funny you should say that, power and energy being a topic of note as we slip into our renewable future. Here's a thing-
https://www.bbc.co.uk/news/live/world-59125188
Methane is responsible for a third of current warming from human activities - and it's one of the most potent greenhouse gases.
Levels of methane in the atmosphere are continuing to rise.
How is it emitted?
Cows may spring to mind... but rice production and rubbish dumps are also to blame. Around 40% comes from natural sources such as wetlands, but the majority is now due to human activity.
Which is a pretty bold claim from the fat-free Bbc. Cows may certainly spring to mind, when prompted, but wetlands? Isn't the EU restoring those, ie increasing them? Along with re-introducing the prospects of the ague to go with your Covid. Swamps around London got drained, ague/malaria was reduced, Now, we're building back better. Or something.
Ok, so fact checking this claim is a bit more complex than just assuming the Bbc is a 'reliable source'. But as with CO2, CH4 is a natural process with some human contribution. Soil bacteria and simple organic decay produces far more than humans. This is in the IPCC's WG1, but still needs some care to dig into the facts behind that literature review.
I guess for me, the most curious aspect of this report is that people actually try getting their news from FaceMeta, and not more reliable sources.
A "wetlands" in the North San Francisco Bay was "restored" a couple years ago, to great fanfare by the denizens of Novato. There was a huge song and dance about it. Now the very same green-and-granola people are bitching about the large swarms of salt-marsh mosquitoes eating them alive. And the smell of decaying plant matter and the odd fish whenever the water is low (it's tidal).
to read me asking if anyone else found it perfect that a Godel is listed on the paper? [probably no relation] I feel like the elder Kurt had some advice on the calculability of certain problems and statements of truth made on same.
My guess is that this is right before the part where the managers of these projects start using interesting new phrases like "manufacturability issues". For those that noticed a funny red line on that one, it's not your spell checker, it's a predictive failure warning for future earnings. If you didn't see one, you might want to ask why your company is using the term so often, and who added it to your dictionary. That person is probably either valuable or a public menace.
But hey, in this case it appears the fact that people "solving" this problem in industry are unaware of the contributions of a 115yo mathematician to their field kind of illustrates the core of the problem the paper addresses. Some problems aren't reliably or efficiently solvable by either crowdsourcing of current ML technology. Some problems require domain specific knowledge. Realtime content moderation embodies both.
As some free advice to those considering making a career of this. It's harder than it looks... and you will fail.
Some problems aren't reliably or efficiently solvable by either crowdsourcing of current ML technology. Some problems require domain specific knowledge. Realtime content moderation embodies both.As some free advice to those considering making a career of this. It's harder than it looks... and you will fail. ..... Corporate Scum
That is almost quite so and correct, for more than just a few can surprisingly succeed whereas corporate scum are always guaranteed to serially fail abysmally and/or spectacularly.
Such is an important and valid distinction to be aware of so as not to be caught off guard and unprotected and undefended by services unable to recognise the critical systemic vulnerability being exploited and expanded upon and exported to regions and sysadmins elsewhere, both deep and dark webbed and bright and breezily trailed and tailed/remotely mentored and virtually monitored.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2023
