Proof-of-concept open-source app can cut'n'paste from reality straight into Photoshop using a neural network

We've written a lot about academic research, startups, and internet giants making use of artificial intelligence. Sometimes source code is shared, and sometimes it isn't, which can be frustrating – we feel that pain. For those of you thinking about toying with machine-learning in a practical sense, how about this interesting …

[jake]

OK, I'll bite.

If I take a photo of my physical desktop, which of the many objects on it are pasted into Photoshop? And if I take a picture outside my office window?

More importantly, for those of us who wouldn't run Adobe code if they paid us to run it, can I use the image editor of my choice?

[pmb00cs]

Re: OK, I'll bite.

It's open source, and the code is linked to in the article. You know you could always raise a pull request to allow it to support your image editor of choice if it doesn't already.

[JetSetJim]

Re: OK, I'll bite.

You could always exit it to save to a Google/One drive location with appropriate authentication controls added to the server

[JetSetJim]

Re: OK, I'll bite.

a) I would imagine that the code will rip out the object central to the focal point of the camera image

b) at some point the image will become a bitmap object in the code, so this should not be hard. I think that because Adobe supports remote commands it makes it easier to go via a server to your desktop. The AI bit is working on a server, too, but I wonder if it's simple enough to work on a phone?

Alternately, this would be a neat raddition to the "magic wand" select tools in most image editors.

[MiguelC]

Re: OK, I'll bite.

You can see it working on the demo, when he copies the book. That image also has a potted plant a little off-centre that's disregarded.

Well deserved pint!

[Pascal Monett]

Re: OK, I'll bite.

Well, the article does state that "Support for other imaging editing programs in the works, wee're told".

How long that will take, if it ever happens, is anyone's guess.

[Loyal Commenter]

Re: OK, I'll bite.

It's open source. If you need it, write it!

I'm sure it's not too much of a stretch to have it just stick the image into the Windows (or $OS_OF_CHOICE) clipboard and allow you to paste it yourself, I'm assuming Ctrl-V isn't too onerous for most.

[jake]

Re: OK, I'll bite.

"It's open source. If you need it, write it!"

I am aware of that. And yes, if I need it (doubtful) it'll probably be trivial for me to modify the source to suit my needs. But that's hardly the point. In the FOSS world, who writes code to lock you into a third partiy's code? Interoperability is part of the point. Especially when it should be easy to allow it to use any editor or viewer. Specifying Photoshop has got my jakey senses tingling ...

[Mike Moyle]

Re: OK, I'll bite.

"...for those of us who wouldn't run Adobe code if they paid us to run it, can I use the image editor of my choice?"

From paragraph 3 of TFA:

"Support for other imaging editing programs in the works, we're told."

[andy 103]

Impressive

That's actually incredible and well done them for producing it.

If you are going to slate it please give an example of something you've produced which is better. I certainly haven't.

[Doctor Syntax]

Re: Impressive

Or literally incredible? Or even literally actually incredible?

[jake]

Re: Impressive

Literally really truly actually incredible.

[jake]

Re: Impressive

"I certainly haven't."

Doesn't that make your commentary superfluous, by your own logic?

[andy 103]

Re: Impressive

Doesn't that make your commentary superfluous, by your own logic?

No. The point I'm making is that people often jump in with comments which are critical about something that's far more impressive than anything they have produced themselves. As you can see from my comments I'm saying it's both impressive and I haven't done anything better, therefore won't be criticising it.

[jake]

Re: Impressive

So in your opinion, you have to have built something better in order to offer up a negative review of any given thing? And that everybody else has to give happy-happy feedback, or not comment at all?

Do you have any idea how silly you sound?

[steamnut]

Unintended consequences

This is an excellent piece of work for sure but, I do see some "dark" applications for it. For instance, how about taking an image of your bosses signature and posting it into an important document? The process is so fast that you could do this by a quick visit to his desk while he is taking a comfort break. Yes, there are other ways of doing this but nothing like as fast as this is.

How about putting people into security images just before presenting evidence at a court case?

I think that this is just the start of some very scary technology......

[frank 3]

Re: Unintended consequences

'Taking a comfort break?'

Christ on a bike. This is an English publication, so let's not despoil it with ugly, squeamishly coy Americanism euphemisms for ordinary everyday things.

It's a poo, a dump, a crap, voiding ones bowels, emptying the chocolate whizzway, dropping some kids off at the pool, going for a shit and so forth.

And if you want to be really, really boring, you could call it 'going to the toilet'.

[User McUser]

Re: Unintended consequences

'Taking a comfort break?'

Christ on a bike. This is an English publication, so let's not despoil it with ugly, squeamishly coy Americanism euphemisms for ordinary everyday things.

As an American, I just wanted to say that have *never* heard this expression before today. But it certainly sounds very "Are you Embarrassed Easily" British.

Now if you excuse me, I have to go pinch a loaf, make an offering to the porcelain throne, take the browns to the super bowl, drop a deuce, unload some timber, and bust a grumpy. Then after all that I have to take a shit.

[JetSetJim]

Re: Unintended consequences

Don't forget to bomb China while you're at it.

Icon: for the curry after effects

[jake]

Re: Unintended consequences

Chalk me up as another Yank who has never heard the euphemism used in that way.

The closest I can remember is the tongue in cheek "morning constitutional".

[Anonymous Coward]

Re: Unintended consequences

It's a poo, [...]"

That toddler-speak is itself one of the most egregious euphemisms. It seems to have been given currency by people who are apparently infantilised by their society insisting that a 17 year old is still an undifferentiated "child".

[JetSetJim]

Re: Unintended consequences

If you can get your hands on a document (s)he has signed (eg employment contract?), you can just crop it out from a scan anyways and use it however you please. Apparently. So I'm told

[jake]

Re: Unintended consequences

People have been fraudulently transmitting signatures electronically since the early days of faxes. That would be in the late 1800s (before the telephone, even!) for you kiddies who seem to think the history of our modern world started with iFads and go ogle.

[fnusnu]

Re: Unintended consequences

It's 'bio break' nowadays

[Doctor Syntax]

Re: Unintended consequences

Upvote - but I think the start was a good while ago.

[matjaggard]

Re: Unintended consequences

I'm assuming the OP was referring to the Americanism of a "Rest Room" being a sensible location for a "Comfort Break" as opposed to a toilet, stall, crapper or whatever where you go for a piss.

However, the whole concept is ridiculous. If you stop people making things easy just because they might get used for bad stuff, you'll prevent every possible kind of progress in the world.

[ColonelDare]

An Open-Source app...

> "...and then transfer that image over the air almost instantly into Adobe Photoshop"

What about GIMP? Just asking.

[Anonymous Coward]

Re: An Open-Source app...

RTFA

[ColonelDare]

Re: An Open-Source app...

Sorry AC, I was just asking - can I use GIMP? Many of Photoshop's functions _are_ available in GIMP, but not having Photoshop (license/OS nor experience) I was wondering if I can use GIMP instead (?).

The article does not explicitly say so - so can you be more helpful please?

[JDX]

I've no idea if it's particularly useful

But what a fun, cool project. That must've been a great feeling the first time it all worked end-to-end.

[jake]

Re: I've no idea if it's particularly useful

Like building and successfully running a Rube Goldberg drawing. (You Brits can instead build a Heath Robinson drawing.) Useful for learning engineering skills, not much else.

[Mike Moyle]

Re: I've no idea if it's particularly useful

"Useful for learning engineering skills, not much else."

For a programmer, possibly; for a potential end-user, probably not. If it's faster/more accurate than manual masking then it's potentially money in the bank.

[Dave 126]

Re: I've no idea if it's particularly useful

Software that removes the background of images has been around since the 90s, though it needed a human to hold its hand. It cost hundreds of pounds. Removing the background of an image is very widely used as a matter of course - think of brochures, adverts, images inset amongst text in magazines - and so for many art departments it was just a matter of doing the sums; does this save us enough man hours to be worth buying? Of course the technique used by the software was different and the output required more human tweaking - just as we had speech recognition in the 90s.

Object removal and 'Backgroud aware' transformations are the main carrot that Adobe used to entice people to the SaaS Photoshop CC from its standalone predecessors.

So yeah, there has been money in it. This guy's decision to offer his work open source may have influenced by his understanding of what similar software is already available.

[pseudonymous]

Cut?

Err, unless they've also managed to advance our mastery of physics to Star Trek levels, that's *Copy* and Paste.

Cool project.

[JDPower]

The transferring it over the air is clever, not knocking that at all. But it'll take more than what's shown in that video to impress with the cutting and pasting element. Show it extracting from a more cluttered, non neutral coloured, area and pasting into something other than a white/neutral image in the exact same lighting/environment. That is incredibly tricky and if it can't handle that it has very limited usage. Fair play on the innovation side regardless though.

[Tessier-Ashpool]

Real world

I’ve only dabbled with Photoshop and the more reasonably priced Affinity Photo. But every time I’ve been asked to lift an image from a background, it’s involved a fair bit of work that’s much more involved than identifying an outline and cutting out the object. You can get spectacularly bad results, for example, if you try to remove the background from a model with flyaway hair unless you pay a lot of attention to what you’re doing.

I’m not so interested in the app > photoshop interaction as the ability of a neural net to convincingly remove a background. I imagine that in itself would be enormously useful to a lot of people if it turned out to be really clever at doing it.