back to article Your phone may be able to clean up snaps – but our AI is much better at touching up, say boffins

Don’t worry if the lighting is a bit off in your photos – artificially intelligent software can fix that. Computer scientists from Nvidia, Aalto University in Finland, and the Massachusetts Institute of Technology in the US, have trained a neural network to restore images marred by flecks of noise. Computer vision algorithms …

  1. Deltics

    Am I the only one...

    ... completely fed up with the way that "AI" has become the go-to term for what we used to call "Data Processing" ?

    Sure, the volumes of data involved have increase dramatically and the complexity of the algorithms along with it, but essentially the technology is no different than it has been for the past 30-50 years.

    It's not AI, it's a computer. Dammit.

    1. Kevin McMurtrie Silver badge

      Re: Am I the only one...

      It means learning to create replacements for missing data the way you want it to. It's all good until it's applied to medical tests and law enforcement.

    2. Triumphantape

      Re: Am I the only one...

      It's just advertising and hype, however I do believe we will have true AI in my lifetime.

      God help us when we do.

      1. Anonymous Coward
        Anonymous Coward

        Re: Am I the only one...

        It's even better than that. Rather an outright lie:

        I can't wait for this ridiculous second dot-com bubble to pop.

      2. JohnFen Silver badge

        Re: Am I the only one...

        "I do believe we will have true AI in my lifetime"

        How do you define "true AI"?

    3. Tuesday Is Soylent Green Day

      Re: Am I the only one...

      At least they didn't choose to call it "nanotechnology". That one has been done to death.

      1. Anonymous Coward
        Anonymous Coward

        Re: Am I the only one...

        A friend of mine works in the nanotechnology department of his university.

        It's a huge building.

    4. Kristian Walsh

      Re: Am I the only one...

      It's not AI, it's a computer. Dammit.

      It is AI, because it's a computer.

      This uses a trained neural network. That's pretty much the classical Artificial Intelligence approach to problem solving. I can't think of any other correct term for this approach (it's not "data processing" - that would only describe the deployment stage, where the already-trained network is applied to the input, noisy, images). Call it Machine Learning if you prefer, but the academic field that does it has been called Artificial Intelligence since the dawn of computing.

      It's not the fault of people working in that academic field that its name has been hijacked by a generation of self-appointed "futurists" and bullshitters (same thing?) to describe their tech-utopian wet dreams.

      The other problem that besets AI is that as soon as an application of it is working and doing something useful, those same loudmouths stop including it in their definition of "AI", so the general public don't appreciate that AI systems are a. not something new, and b. not something rare: OCR, that X-Box Kinect camera, voice control of anything, high-definition ultrasound imaging are all excellent examples of Artificial Intelligence deployments.

      "Intelligence" here doesn't mean that the things can think. It means they can extract relevant information from a wide range of inputs that containing that information.

  2. Triumphantape

    It feels like having talent is going to be a burden when the AI can do it all, why practice develop yourself in any aspect when (eventually) AI will do it all and do it better.

    Humans devalued.

    1. Pascal Monett Silver badge

      And this will be defined by future historians of what is left of the human race as the exact moment the conflict started.

  3. Anonymous Coward
    Anonymous Coward

    Bu what if....

    ...I want a picture that is an accurate depiction of reality rather than one that looks clean? Isn't this just doing what magazine publishers have been doing to cover shots for decades?

    1. Anonymous Coward
      Anonymous Coward

      Re: No?

      If it is trained on "noise" and/or text defacement/watermarking then it will mainly remove that.

      It's like sanding down a rough edge. If you want a textured edge on a surface (grip on a hand held device), you don't sand it down. If you want a smooth edge, you sand it down.

      This is a tool, it could be used to fake things, it could be used to improve (actual data processing for sub pixels) the image. It's down to the user, as with most tools, to decide how to use it.

      1. DropBear

        Re: No?

        I do exactly this sort of thing rather a lot, while archiving my photos where my OCD compels me to get rid of the plethora of scratches dust and other stuff that seems impossible to get rid of no matter how much you clean before you scan. The thing is, the sharpen filters do an amazing job at actually making the pics strikingly sharper (and the ones I use don't even seem to induce unbearable light/dark contour shadows which are the hallmark of sharpening) - but one consequence is that a seemingly previously non-existent noise shoots up past the stratosphere in many cases.

        There is another photo filter that can get that phantom noise largely under control again and yield remarkably smooth photos (quite a lot like these actually), but here's the gist of this: it also tends to noticeably alter really fine detail, like the girl's knitted clothing; not completely mangle it, mind you, but still cause noticeable loss of detail at high zoom. And this thing kinda seems to have a similar problem, although I have to admit it's rather good at it. I'm rather curious exactly how much detail is lost here.

        As for removal of stuff from the image - I'd love to see how noticeable the "fudging" of data clearly not present in the image based on surrounding pixels is - as an artificial artificial intelligence, I can do that too when needed, but it's hard to do convincingly if there's any surrounding detail, and the filter tools I've seen do a remarkably shitty job of trying to "stich up" even around a simple scratch...

  4. Yobgod Ababua


    Do you notice how, at the end, they are all about removing superimposed text?

    That worries me a little, because it looks like a means to remove watermarks and branding, so you can use photos you don't have rights to while pretending you do.

    Or maybe I'm overthinking it.. no? yes?

    1. caffeine addict

      Re: Huh

      That was exactly my first thought. If they can remove text that's obliterating an image (interesting to know how they guess what should be underneath) then removing a subtle semi-transparent watermark should be simple.

      But... how does it know what is text "noise"? If you feed it a street scene, will it go and rip the names off all the shop signs?

    2. Kristian Walsh

      Re: Huh

      I though that too, but looking at the text examples, there's so much defacement, and the text is so small, that it's basically another kind of noise.

      I'd still like to see the output when fed a watermarked image, but my gut feeling is that it would leave it largely intact.

  5. Nifty Silver badge

    How about the audio equivalent?

    Much effort has been put into cleaning up old audio recordings. But I’ve yet to hear of any project that uses AI type algorithms that can restore or enhance missing sibilance in speech by learning from speech examples in the same language. Maybe even a similar idea with old orchestral recordings is possible.

    1. LDS Silver badge

      Re: How about the audio equivalent?

      Do old recordings have the required data, or you just have to "reconstruct" them? An educate guess may work and "improve" the listening, but it's not still the original data, it could be just a replacement, a good one maybe, but it could be the wrong one as well.

      1. Kristian Walsh

        Re: How about the audio equivalent?

        This system is a reconstructor for images. As it fills in pixels that have been obliterated in the input., you're never getting the original - all that you gain is the removal of signals that distract from what the original probably was.

        Old recordings to have quite a lot of data, but masked by very high levels of noise. If you consider that an old phonograph was basically a cutting needle coupled to a drum with very little processing between, there's quite a bit of information that could be pulled out of it.

        "Improving" the listening is all that's needed for academic study of old recordings, and certainly for entertainment purposes - I've listened to a few early recordings, and it takes a while for your ears to adjust to the noise and missing high and low frequency information. pre-processing with an AI could help bring these to a wider audience.

        For more serious work, just like the medical imaging examples, the post-AI version is used to get an overview, but where there's doubt, the original pre-processed file is still available for detailed study. This already happens in historical research: historians normally work from transcripts and translations of old documents, but in special cases they will need to examine the original book to resolve a query.

      2. Nifty Silver badge

        Re: How about the audio equivalent?

        There was a compression standard called MP3+ which replaced everything above about 11 kHz with bursts of white noise of the same duration of the original full spectrum sound. It turns out that our ears/brain are tone-deaf above a certain frequency, only caring about duration and phase if stereo. OK, trained musicologists mileage may vary.

        The codec was used by online radio stations for a while. Not sure if AAC+ uses similar tricks. Anyway the empirical result was very good.

        Hence I was thinking that a similar reconstruction would potentially work well. Concerning the info that is buried in old recordings: I don't just mean audio info, they have implied human language or knowledge about musical instruments. AI can cross-reference to language and music knowledge bases to help the reconstruction.

  6. Anonymous Coward
    Anonymous Coward

    Better use of De-noise

    I wish they had use the de-noise on the sound track. It was appalling

  7. Tony Paulazzo

    Every one's missing the bigger picture here, this means HiDef versions of all my old classic porn... er, I mean archive footage from the 70's :| all those old 40's / 50's scifi B movies lovingly restored.

    And yea, if it can't do that then it's not AI.

    1. caffeine addict

      For the love of god, don't let an AI "learn" about grumble flicks. It'll lock itself in a room and start creating artefacts of its own...

  8. Michael H.F. Wilkinson Silver badge

    The authors don't use the term AI

    which is reassuring. It might well be useful. I have given the paper a very quick read, and the idea of using statistical reasoning seems and learning seems reasonable. I would like to see how it performs on seriously big astronomical images compared to other advanced denoising methods (and goodness knows there are many of them). If it does the job efficiently, I will be happy, if others outperform it, so be it.

  9. Anonymous Coward

    AI in everything.

    Even the local Pizza place:

    "Powering our business with cutting edge technology, we use data and tech to put customers first. Artificial intelligence drives everything we do, from the recipes customers see on our website, to how we put the boxes together. We’re proud of our industry-leading product and service, which helps our customer enjoy the good food, they want."

    1. JohnFen Silver badge

      Re: AI in everything.

      "we use data and tech to put customers first"

      Phrases like that are a sure sign that a company is engaging in customer surveillance and data mining.

  10. jmch Silver badge

    Stating the obvious, much?

    "The model won’t cure all imperfections, however. It can’t restore objects that are just out of frame or reposition the photo to get the best angles"

    Objects out of frame or a picture taken at a non optimal position aren't 'imperfections'!

    1. DuncanT

      Re: Stating the obvious, much?

      Obvious to you , maybe. Anybody who's know in their job for being good with photoshop will tell you they've been asked to 'just turn that house around so I can see what's in the back garden' or similar

      1. Anonymous Coward
        Anonymous Coward

        Re: Stating the obvious, much?

        I'll raise you to a local councillor who, when watching some recorded footage of some vandalism could NOT grasp that despite asking, nay demanding to see what was to the left or right off camera that a recording is just that, a series of past events. He could not grasp this. This was a councillor, be afraid, be very afraid.

      2. Sam Therapy

        Re: Stating the obvious, much?

        "What was your husband's hair like?"

        "Well, you'll see that when you remove the hat."


  11. JeffyPoooh

    Spots seen on "...medical resonance imaging scans..."

    Oftentimes, one would rather have the spot removed by surgery, rather than just removing it from the image.

    [Extreme Example Alert]

  12. Simon Harris Silver badge

    Was it just me...

    or did the process make the child with the face-paint around her eye look like she was made out of plastic?

  13. Mike 16 Silver badge

    Ultimate goal?

    To make the typical bad detective show "Enhance" of photos into "reality" so convincing it can fool a jury.

    Like where they take 3x5 pixels from a dirty CCTV camera last serviced in the 1970s and get not just the license-plate number but an estimate of how long since the car was last washed and a list of roads it has traveled since.

    Could be a real boon to "parallel construction" unless someone grows a conscience at rats out Ofissah Plod.

  14. Bucky 2

    Old Movies

    I'd like to see a comparison of how it might clean up an old movie, compared to whatever procedure they do now.

  15. Anonymous Coward
    Anonymous Coward

    I do hope

    this doesn't turn out to be some vapourware ala Adobes anti blurring / shake "removal" debacle from a few years ago.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020