back to article Tech demo takes brain scan, creates a picture of what you're looking at

There's a lot of noise to signal in the machine-learning model world, but this demo is genuinely impressive – or scary, if you are given to recreationally climbing into MRI scanners. The new research is presented in a paper titled "High-resolution image reconstruction with latent diffusion models from human brain activity" co- …

  1. anderlan

    Interesting how those in the know (the paper authors) in their paper don't just hide the faces of the people in photos presented to those being scanned by fMRI, but their whole bodies from other data models that no doubt in future get input from their paper.

  2. Anonymous Coward
    Anonymous Coward

    nuclear magnetic resonance imaging

    Sometime in the 90's, whilst being wheeled about a hospital with a broken leg, we passed a sign for the "Department of Nuclear Medicine". Although "Nuclear Medicine" might have sounded pretty awesome, on the whole I was probably better off in Orthopaedics.

    1. Anonymous Coward
      Anonymous Coward

      Re: nuclear magnetic resonance imaging

      I had the experience of being injected with radioactive material to map blood flow. The radioactive material had a very short half-life, used to soon it would be too radioactive for safety, used to late it would not be radioactive enough for the scanner to detect.

      I can report that 9 years later I have not yet grow any extra limbs or organs. However, my waste line has increased which was obviously due to the exposure and nothing to do with to much time spent at the computer reading The Register.

      1. An_Old_Dog Silver badge
        Joke

        Re: nuclear magnetic resonance imaging

        However, my waste line has increased

        I first thought you were trying to somehow euphemistically write that your colon had increased its diameter; then I realized you'd meant to write, "waistline."

      2. Sean o' bhaile na gleann

        Re: nuclear magnetic resonance imaging

        Almost exactly the same here - radioactive juice injected for a PET scan, followed immediately by weight gain.

        (although it has to be said that my increase in mass was a work in progress since long before the scan...)

      3. d2

        Re: nuclear magnetic resonance imaging

        some are not as lucky as you,AC...

        Gadolinium Toxicity: The MRI Danger No One is Talking About -Jesse Cannone

        Chuck Norris and his wife Gena have spent more than $2 million treating her for gadolinium toxicity — a devastating condition they say was caused by the gadolinium contrast dye used in MRI scans she underwent several years ago.

        ...While the FDA has insisted for years that gadolinium-based contrast agents (GBCA) are only risky for patients with kidney disease, Gena had normal kidney function prior to her MRIs.

        And there are a growing number of other people with normal renal function who insist their debilitating chronic symptoms only appeared after they had contrast MRIs.

        These symptoms include brain fog, cognitive deficits, burning pain throughout the body, bone pain, joint pain, skin changes and hair loss. Some people also experience tingling and shaking.

        Most doctors know absolutely nothing about gadolinium toxicity. They don’t connect the symptoms to the contrast material used in MRIs.

        And the FDA has ensured them that gadolinium is harmless...

    2. Anonymous Coward
      Anonymous Coward

      Re: nuclear magnetic resonance imaging

      Many years ago my professional institutes local branch organised a visit to the local hospital's cyclotron; I don't think any of us were previously aware it existed. It gave the medical diagnostic teams access to short half-life radionuclides (of oxygen, if my memory is correct). Again, if my recall is correct (and the visit was about 30 years ago) they were introduced into a patient's air supply and could be tracked through the body.

    3. David 132 Silver badge
      Happy

      Re: nuclear magnetic resonance imaging

      > whilst being wheeled about a hospital with a broken leg, we passed a sign for the "Department of Nuclear Medicine". Although "Nuclear Medicine" might have sounded pretty awesome, on the whole I was probably better off in Orthopaedics.

      Where’s your sense of adventure? You could have left there with SIX broken legs!

  3. NoneSuch Silver badge
    Thumb Down

    Yeah, this won't be abused.

    US Border check points to scan your brain for "criminal activity" when travelling. The CCP will love this for scanning dissidents in mainland China. Not to mention HR using this as a pre-scan for job interviews.

    1. alain williams Silver badge

      Re: Yeah, this won't be abused.

      What was more scary was a paper that I read that showed different brain activity when the person was shown something that they had seen before. So: worry when the border guards ask you if you "have seen this person before".

      Sorry, I cannot find the link - maybe I need my brain scanning :-)

    2. chivo243 Silver badge
      Happy

      Re: Yeah, this won't be abused.

      I'm wondering if the George Costanza factor can be invoked... "It's not a lie if you believe it". Think happy thoughts!

    3. Evil Auditor Silver badge

      Re: Yeah, this won't be abused.

      Suddenly, what I, as a rightpondian, thought of as a rather silly question starts to make sense. "Do you seek to engage in or have you ever engaged in terrorist activities, espionage, sabotage, or genocide? - we'll answer that for you."

  4. Primus Secundus Tertius

    After this, what?

    So far, it is pretty pictures rather than sounds or smells.

    If the picture is a page of text, can it produce a Word file of the result?

    1. Jan 0 Silver badge

      Re: After this, what?

      Probably,, but not a file that your version of Word will be able to display.

  5. The Oncoming Scorn Silver badge
    Pint

    No Not The Mind Probe!

    When one is used on the Doctor it ends up being destroyed when General Williams refuses to accept it as truth. The Doctor talks of a similar Noodle Incident where he was captured by the Medusoids and broke their own mind probe. He was freed when the Medusoids ran out of mind probes.

  6. lukewarmdog
    Pint

    Finally

    A machine that can tell me what video I really want to watch on PornHub tonight!

    (other websites may be available..)

    1. Anonymous Coward
      Anonymous Coward

      Re: Finally

      "A machine that can tell me what video I really want to watch on PornHub tonight!"

      Isn't it always the same one .... !!!

      The one with the wet celery and the flying helmet .....

      :)

      1. DoctorPaul

        Re: Finally

        Upvote for the Allo Allo reference

  7. Anonymous Coward
    Anonymous Coward

    I wonder if it could be reversed to put images into the minds of blind people.

    1. Evil Auditor Silver badge
      Devil

      put images into the minds

      I imagine this is a rather simple process: place some electrodes on the brain parts doing the image processing and fire some voltage through them. Or did you mean some meaningful images such as a camera feed?

      1. mevets

        Elon

        Is that you?

  8. Anonymous Coward
    Anonymous Coward

    From the article

    "The scanners involved are extremely large (and this vulture can attest, having been in more than one, extremely loud) machines."

    You have been in one of those machines more than once? Sir, I am in awe of your bravery.

    I went in one of the things once. There is not enough beer on this planet to get me to go into one again. I don't know which is worst, the claustrophobia or the noise. To my eternal shame I pressed the panic button, but as it was almost at the end of the scan they didn't stop the fucking thing.

    Needless to say, I spent a fair amount of time in the pub afterwards, calming my nerves of course!

    1. Evil Auditor Silver badge

      Re: From the article

      I find these testimonies rather interesting. Never been in one myself (and it is not that my body wouldn't fit in!) but I did hear from a few people all of whom experience it more than once. Some described it pretty much like you, as rather close to horror. And others were more like "meh, it's a bit noisy."

      1. Inventor of the Marmite Laser Silver badge

        Re: From the article

        Had one a few years back and I can attest to the noise. It is, however a bit like a noisy a/c or traffic noise and your mind pretty quickly disregards it as baseline to be ignored. What I WASN'T warned about, however was the (almost) soothing baarrrp, baarrrp, baarrrp would suddenly change. There was I, just beginning to doze, in the audio cocoon of baarrrp, baarrrp, baarrrp when suddenly baarrrp, baarrrp, baarrrp became BARRR, BARR, BARRR and jerked me to full and sudden wakefulness.

        Have you ever tried laying inside a twenty inch pipe and suddenly sitting upright? It doesn't work. I was moved to suggest the staff warned people about sudden changes in the noise.

    2. G.Y.

      close eyes Re: From the article

      I fixed the claustrophobia issue by shutting my eyes -- and the technician thought I was asleep! Looks like some people can sleep there.

    3. Filippo Silver badge

      Re: From the article

      It's extremely subjective. I've zero problems with either the noise or the claustrophobia; I could easily lie there for hours, if it was just that. It's the remaining perfectly still that I find difficult.

      1. BeefEater

        Re: From the article

        Same here.

        I can't remember how many times I have been in one of those machines, at least half a dozen

        The noise is annoying, no problem with claustrophobia, but not being able to deal with an itch on my nose, or elsewhere, for up to half an hour is a torment.

        I was sure that there were gaps in the sequence when some movement would have been possible if only they had told me when they were.

        It was a bit amusing, and disconcerting, when one operator mentioned that any near-by metal could blur the results.

        I presumed that the consultants that booked the scan knew that the bar that they had put in my jaw would not cause such an issue, but the operator still thought that it would, but went ahead anyway.

        1. ThatOne Silver badge
          Devil

          Re: From the article

          > any near-by metal could blur the results

          Isn't it the case that any nearby metal could blur the patient, due to the extremely high magnetic fields the MRI generates?

          AFAIK the most important risks include inductive heating (burns), and metallic objects flying across the room to crush/decapitate/otherwise maim you...

    4. CrazyOldCatMan Silver badge

      Re: From the article

      You have been in one of those machines more than once? Sir, I am in awe of your bravery.

      Twice - once to see if my migraines were the result of a brain tumour (they weren't - just genetics) and the second time to check my heart.

      Not the most fun experience - you can't even read a book or listen to music. Just as well the scanner aparture isn't small enough to trigger claustrophobia.

    5. Liam Proven (Written by Reg staff) Silver badge

      Re: From the article

      [Author here]

      > You have been in one of those machines more than once? Sir, I am in awe of your bravery.

      Nah. I wasn't given a lot of choice in the matter. If you have no option, it's not brave.

      I was hit by a bus on my ZZR-1100 in 1994. Left leg broken in over a dozen places, right arm snapped off at the shoulder, left arm dislocated at the shoulder. Whiplash injuries to my middle ears means I have almost no remaining sense of balance. Extensive metalwork in left ankle, shin, thigh, hip, right shoulder, humerus, left shoulder.

      What was scarier was being put in an MRI scanner *again* 25 years later to try to work out why my left arm kept falling out of its socket. The thing is, while an MRI scanner vibrates hydrogen nuclei, it just heats metal, and I'm half steel. If they got it wrong, they'd cook me from the inside out like a microwaved pasty... _That_ was scary.

      It didn't. All 4 limbs are still present and working to some degree. I still ride bikes, albeit now pedal cycles. I've learned to ride a recumbent.

      I have more or less failed to learn cross-country skiing, though, and I am even worse on a snowboard.

      1. sofaspud
        Joke

        Re: From the article

        Ah-HAH! The Reg *does* use robots to write articles!

      2. This post has been deleted by its author

      3. This post has been deleted by its author

      4. This post has been deleted by its author

      5. This post has been deleted by its author

      6. 89724102172714182892114I7551670349743096734346773478647892349863592355648544996312855148587659264921

        Re: From the article

        I was minding my own business cycling one dark night, when a badger broke my wrist. The hospital refuse to remove the big titanium implant and screws they used to repair it with, despite titanium causing problems. On a motorbike at speed, I'd probably be dead.

        https://www.youtube.com/watch?v=iXdrFfpn3Is

        Fortunately I was only travelling at 12mph on my 2006 Dahon Helios P8 (250W Bafang BBS01B mid-drive) ebike conversion, unfortunately I was carrying 8 heavy pumpkins in my very large rucksack, plus another four in pannier bags. The jawguarded helmet saved my teeth, but my left wrist felt crunchy. Eventually one of the many passing cars stopped (I wasn't trying to flag any down, was considering calling a large cab [my bike folds]), and two witches returning from a seance (I'm not joking) drove myself, my bike and my pumpkins 13 miles home, then another 8 miles to hospital. After a 12 hour wait in A&E, they tried unsuccessfully to put my wrist back into place by forceful manipulation, using nitrous oxide to dull the pain of yanking and twisting; X-raying to see if it worked. It didn't. Three times, and each time the nurse doing the twisting was switched for a burlier one... the third time, my arm was being stretched by a nurse of truly gigantic proprtions bracing himself against the bed with his legs. The nitrous was very effective - I would have been laughing if my wrist was being sawn off. It was determined the fractures and breaks were too bad, and what approximates to a ball joint was too damaged for a solution so simple, hence this troublesome titanium. The surgeon was a keen cyclist, and had difficulty believing what had happened, until I showed him the video. He said he'd be more careful cycling in the woods near his home, where he's seen badgers.

        I couldn't find the badger, wasn't in any shape to search. I can't imagine it would have lived for long, as the combined momentum of myself, bike and cargo was substantial. The badger was quite large and stocky, looks smaller in the viideo because of the wide angle lens and poor sensor, which like all of the current crop of action cameras, is useless at night. It actually screamed like a small child, a moment before my front wheel hit it, which wasn't picked up by the action camera microphone. Unavoidable - it leapt out from a hole in the roadside hedge directly under my front wheel. I have been unable to find statitics about bicycle badger collisions.

        BTW These settings work well at night for GoPro Hero Black 7 or newer, to record vehicle number plates, for fun and litigation. Kudos to GoPro for including a good set of DSLR-like manual settings:

        https://vimeo.com/791920458

        ... flickering of LED streetlamps is revealed at 1/480th shutter speed, which is even better at capturing clear number plates (when they're not deliberately kept obscured with grime, to thwart speed cameras), but colour information is reduced:

        https://vimeo.com/795755447

        The good news is, action camera sensors and lenses are only going to improve.

        A bit too cold for cycling at the mo, but I have to run down the battery to a safe voltage for storage. I really should make make larger resistors to do the job.

  9. cantankerous swineherd

    looking forward to seeing this replicated.

  10. Dante Alighieri
    Headmaster

    Spin on article

    Well MRI works by interrogating the spin of atoms.

    What it *doesn't do* is spin a superconducting magnet around your body. The physics is phenomenal. TL;DR there is a gradient/shim field that alters the resonant frequency across the volume. Ping in some RF and you can read out the atomic relaxation.

    What does spin 2 tonnes (230 Adult badgers or 1 1/3 skateboarding rhinocerii) around your head or body is a CT scanner, using slip ring (ooer Matron) technology - as pictured in the article.

    It's my day job,,, and I'm licenced to play with the unsealed radioactive sources too

    1. Someone Else Silver badge

      Re: Spin on article

      What he said.

      When I was first shown the inner workings of a CT scanner, I remarked it was similar to taking a Mini Cooper and spinning it around at 5 revs per second.

      It's no wonder they test them in reinforced concrete bunkers...

    2. Anonymous Coward
      Anonymous Coward

      Re: Spin on article

      "and I'm licenced to play with the unsealed radioactive sources too"

      If your license expires while playing with unsealed radioactives, do you fry?

  11. mevets

    Astounding.

    It is an astounding result; but the editorial here seems a bit much. Unless I've misread the paper.

    Each test subject is scanned looking at each image several times as part of a training set.

    Then each test subject is shown some images again, and the machine determines the image they are looking at.

    If a given subject, subj01 does not participate in forming the training set, then the machine can't recognize what they are looking at.

    At least, the paper does not allude that it can, and I'm pretty sure it would have been in the title if it could.

    The paper also doesn't claim whether it is a per test subject training set, or one big one for everybody.

    While the core complexity wouldn't really change, the shear data, computational power and utility would be higher.

    Cool project though. I'm jealous.

  12. Zebo-the-Fat

    If it produces an image of what the subject is looking at, all the images should show the inside of the scanner!

  13. atropine blackout

    A less than obvious limit

    A few years ago, prior to a bone scan I was injected with Technetium 99, a commonly used and fairly lively gamma tracer.

    During the wait before scanning, I was chatting to the technician about research into decreasing the exposure time associated with short-lived radionuclides.

    She told me that one of the big problems with very short-lived isotopes was transporting the material to hospitals . (this is cleverly avoided with Tc99 by shipping it as Molybdenum 99 which then decays to Tc99)

    Apparently there are only a very few sites capable of making suitable short-lived nuclear agents, so you need fairly reliable shipping arrangements to avoid running out of half life.

    In one case the journey from the Netherlands to the UK, including a traffic jam meant that an entire batch of Gallium had to be binned on arrival.

    And for the curious - I measured my own gamma activity both at the time I was injected and 24 hours later - great happiness; the laws of physics held up.

  14. atropine blackout

    Somewhat off topic

    A few years ago, prior to a bone scan I was injected with Technetium 99, a commonly used and fairly lively gamma tracer.

    During the wait before scanning, I was chatting to the technician about research into decreasing the exposure time associated with short-lived radionuclides.

    She told me that one of the big problems with very short-lived isotopes was transporting the material to hospitals . (this is cleverly avoided with Tc99 by shipping it as Molybdenum 99 which then decays to Tc99)

    Apparently there are only a very few sites capable of making suitable short-lived nuclear agents, so you need fairly reliable shipping arrangements to avoid running out of half life.

    In one case the journey from the Netherlands to the UK, including a traffic jam meant that an entire batch of Gallium had to be binned on arrival.

    And for the curious - I measured my own gamma activity both at the time I was injected and 24 hours later - great happiness; the laws of physics held up.

  15. Anonymous Coward
    Anonymous Coward

    What it *thinks* your looking at

    The results give poor renditions of an image which comes from the same training series.

    So its not rendering what the subject saw, but a render of a 'match' image from its training.

    Examples

    Teddy has a different bowtie, and its a different teddy.

    Plane image has clouds

    Snowboarder is going the opposite way.

    So the results are correct as in 'a teddy' 'a plane', 'a snowboarder' etc. But the algorithms cannot depict your image, simply a version of an image they 'seen' that is similar according to its training results.

    Still impressive i guess

  16. Anonymous Coward
    Anonymous Coward

    I'd best not participate

    Were the scanners to read me mind, the scanner would stop working and the mice would grow hair!

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like