What could possibly go wrong
So, based on an AI trained on 40 people, in a few years time we're likely to find people being "sectioned" by computer analysis?
Scary.
Machine-learning algorithms can help psychologists predict, with 90 per cent accuracy, the onset of psychosis by analyzing a patient's conversations, according to this research here. Psychosis can be a symptom of psychiatric disorders such as schizophrenia or bipolar disorder. People experiencing psychosis find it difficult to …
I had a few other examples in mind, given the coherency and delusion aspects.
"A lot of the wall was, you know, I'm good at this stuff. That's what I do. We renovated a lot."
"If you have a windmill anywhere near your house, congratulations, your house just went down 75 percent in value. And they say the noise causes cancer."
"Such as "Brexit means Brexit"?
Yes. This makes the quote "People experiencing psychosis find it difficult to tell what’s real or not... and are led to believe delusional thoughts." a pretty good summation of the current Tory party. The dangerous thing is one of these numptys is going to be the next PM.
Run away!
How many previous and current world leaders would love to "use for the greater benefit of all", these kinds of algorithms.
What use would this be to a repressive government? Autocrats aren't short of excuses for oppressing the populace, and a diagnosis of pre-psychosis says nothing about whether the subject is likely to support or oppose the status quo.
The published article itself (as linked by elreg) is open access, so we can read it.
Here are two quotes from the discussion section:
** "This work is a proof of concept study... "
** "This study had a relatively small number of participants. A well-known problem in these studies is overfitting and consequently poor generalization.38 In the current study, we took steps to guard against overfitting by limiting the number of predictive variables ..."
This article seems to be of a fairly typical type, i.e. "We tested out this idea on a small dataset, and it seemed to give promising results. Because of this, it is probably worth us doing a more comprehensive study in the future; but we are telling you all now because you might find it interesting".
I think any appendix, potentially stating something along the lines of "Aieee! Run for the hills, they're going to get you!" would probably have been deleted on the advice of the referees as being not in the correct scientific style. :-)
Here are two quotes from the discussion section:
** "This work is a proof of concept study... "
All very cogent, but then then we have a Police service trialling an unproven system of equally dodgy provenance - which probably had exactly this disclaimer but since it said 'proof' they thought they had it 'bang to rights' and assumed they could convict on it.
No, that isn't at all likely.
There aren't even enough beds in inpatient units for those who clearly need them. And people are discharged way too early.
And in the US, at any rate, you need two psychiatrists and a judge to sign off on an involuntary admission. (At least in every state I've ever heard of.)
So no, that isn't at all likely. At most this *might* be a tool, to help identify if someone needs further evaluation.
I can't find a date for when they started this study. If they had time to follow through and conclude that some got psychosis and some didn't, then it means that the study had to be done at least ten years ago. AI was not so prevalent a decade ago, so how did they use AI to bolster their study ?
The way that I read it was that they used filmed/taped interviews from however many years ago -- for which they already knew who had developed psychoses and who hadn't. Thirty of the subjects were used for training. They used the last ten to test whether the system would correctly identify which of them had developed psychoses and which hadn't. It apparently was correct in nine out of the ten cases. So, it wasn't a case of "run the software then wait ten years to see how it did"; they could check the system's conclusions against data that they already had.
At least, that's how I read it.
Or the current pretenders to be Prime Minister?.... heyrick
One surely hopes for a blending of genius rather than a surfeit of psychosis be rooted and anchoring to foundational ground zero bases for any SMARTR Governance Machine, heyrick.
In these rapidly changing days with exponentially expanding ways to share formerly held captive and secret and sensitive information about that which is Vitally Important and EMPowering, is there a leader amongst them brave and bold enough to recognise and embrace help whenever it is shared, and of which they would/should be made fully aware of. :-) Easily achieved with simple sight of Real Live AIBetaTesting Narrative.
How to Deal with the Future, is not a trick question. IT expects Answers ..... for IT is a Clear AI and Presenting NEUKlearer HyperRadioProACTive Danger to Fake News with Corrupted Mass Media Systems with an almost Universal Command and Control of Virtual Future Realisations with Sublime Autonomous Services for Servering to Almighty Futured Seeds and Feeds and Needs for such is surely Stealth InterNetWorking.
:-) Surely you weren't thinking Bletchley Park type Boffinry was just a Jolly Roger of an Experiment and/or a One Off Experience ..... whenever Master Moulds were Crafted there for Global Export/Import.
Casting Excellent Minds over Deep and Dark Matters is de Rigeur Default in Such, a Vast Advanced Proprietary Intellectual Properties Field. Another not a trick question is.... Is it a human confection or alien treat?
Methinks that sort of puts Brexit in something of a Harry Lime Light? :-)
So what's it to be? A Genius Blend or an Over Sufficiency of Psychosis for here lie Abiding Fields of Battle. Trick or Treat? Honest Truth or False Promise? What leads to where you have been and/or where you be thinking you might like to go to?
What can we do to get an interesting paper out easily?
That seems to be what this paper is about; a small sample of people overly weighted in one direction, no passable control sample and a questionable source of 'normal conversation' that isn't even conversation.
While the aims of the research are laudable the methods are laughable.
Well, that what the voices tell me.
Easy, just use https://pdos.csail.mit.edu/archive/scigen/. It will make you the paper you always dreamed about.
Imagine you're a boffin with a hypothesis. You can obtain enough funding for a pilot study with a small sample running over a short period of time.
Do you (a) abandon the hypothesis because your study isn't perfect, or (b) perform the pilot study and hope it generates enough interest and enhances your seniority in the community enough to win greater funding to take your research further?
Don't blame the boffins for the system they have to work in.
It may or may not. An awful lot depends on the level of correlation.
For instance, suppose you do a long term study because you suspect that a certain cell type in the liver is pre-cancerous. You already know the prevalence of liver cancer in the general population. You find 30 people with these cells, you prepare specimens and keep in touch. After 10 years, suppose 10 of them have developed the same kind of liver cancer and have no exposure to hepatitis, known carcinogens or high levels of drinking. That's a pretty solid result.
On the other hand, you announce that an in-depth study of eating habits shows that eating more than 70g of processed meat a day causes a 20% elevated risk of bowel cancer. That sounds pretty convincing...but it's likely that intake was self reported, and perhaps there wasn't much analysis around exactly what kind of red meat and processed food was being eaten and what other lifestyle factors might be involved. So it might be a useful statistical guide to populations, but not much use for individual risk assessment.
Small samples were discredited in many people's minds by Wakefield and MMR, but the point about his methodology was that, as in Alice in Wonderland, it was verdict first trial afterwards - identify a group of autistic children and find out if they had had the MMR vaccine. It showed a truly shocking lack of understanding of scientific method and he was duly struck off.
Of the 10 'test' subjects half went on to develop psychosis (so that's 5) but they claimed that the software was 90% accurate in detecting who would get pyschosis.
How do you get 90% of 5? Did one get better? If it predicted 4 instead of 5 then that's 80% accurate. If it predicted 6 but only 5 actually became pyschotic then that's 83% surely?
It's not worth your time, you're doing OK, you really don't need to understand that garbage, no one needs, there's nothing in there that needs understanding. REALLY!
Why start your day wasting time in an effort to take that garbage seriously? There are more productive ways to start your day.
I think it is less about the vocabulary and more about the density of meaning.
An expanded vocabulary empowers you to leverage synergies and blue-sky to obfuscate your dialog, putting lipstick on the porcine and metaforming the narrative. Or you can speak clearly and get your message across.
... by analyzing a patient's conversations...
Conversations with whom? I would think people change how they speak depending on the other participants, assuming there are any. For instance I find it helps to dumb it down when talking to political types but not my peers. Likewise, I would expect one to be more guarded, evasive perhaps, when talking to police than with a drinking buddy although semantic density may be low in both cases but for different reasons.
Conversations with whom?
As the article states, conversational data from NAPLS was compared to data from Reddit. For the latter, obviously, the interlocutors are other Reddit participants.1 For NAPLS, there's a link to the NAPLS site in the article, but I'll save you five minutes of reading and refer you to the NAPLS 2 symptoms paper, which shows that the conversational data available in NAPLS comes from researchers conducting clinical interviews per DSM-IV SCID.
NAPLS is a longitudinal study, so there's reason to presuppose its data is fairly varied and represents a decent distribution across the subject population.
While it's certainly true that the speech occasion and community have a strong effect on sociolinguistic and pragmatic aspects of speech, the strong correlation demonstrated by this - very preliminary! - study suggest that an SCID-type speech situation does not greatly distort the features that the model is correctly identifying as psychosis prodromes.
1Unless the subject is fully psychotic and having delusional conversations, as we sometimes observe here on the Reg forums.
That's when a customer phones up with a fault to report and they haven't got a clue how the thing works... Only their idea of how it should work as explained to them by sales.
High semantic density, engineers or specialists discussing the problem.
Low semantic density, management's understanding of the issue.
Makes me want to go postal most weeks.
By using Reddit for their 'normal' database, they're doing the right thing! Better a false negative than a false positive.
If they had taken their 'normal' database from a more reliable source of sane people, then it might have detected upcoming psychosis in people in between the two samples that actually were in no danger.
I can't wait until every single word the passes over the internet that has been captured in unrestricted dragnets is analysed by AIs to generate a personality profile for every single endpoint and given scores for dissent and likelihood of subversion! That will be such a wonderful future to live in :D
Plenty of researchers, academic and commercial, are routinely running vast corpora of linguistic data gathered from public online sources through various models. In some academic settings, the use of such data is restricted in certain contexts (via human subjects research rules enforced by IRBs and similar institutional controls). In corporations, it's a free-for-all.
You don't really need to wait. The difference between "most online language use is analyzed" and "all of it is" is minimal.
This is interesting stuff.
People with schizophrenia tend to make "loose associations," relating things which the average person would not connect. They see cause and effect where others see disparate events. They also tend to take events personally. As a real-life example, "I saw an ad for diabetes treatment" becomes "I have diabetes. They told me by showing me a TV ad about it."
It sounds like this research is looking for behavior patterns that might indicate loose associations.
This does not at all sound like it could lead to a diagnostic test (we already have diagnostic criteria), but more something that "could indicate elevated risk." This might enable families to become more educated about mental illnesses and begin to form support networks before there's a crisis. Given that symptoms often become problematic during puberty, there's the potential to help people through critical developmental stages, with huge improvements to quality of life.
This does not at all sound like it could lead to a diagnostic test (we already have diagnostic criteria), but more something that "could indicate elevated risk."
That's the whole point of NAPLS. They're looking for psychosis prodromes - characteristics that indicate higher-than-normal risk. NAPLS research publications include investigations into endocrinological and brain-imaging (cortical thickness) markers. I haven't looked at the results.