The New World is Coming
Is this an example where journalists and political opponents will be identified while real criminals have the resources to use open weights?
When Anthropic disclosed earlier this month that it would be applying a statistical word-choice watermark to its Claude text output, the company noted that at least 190 AI providers have agreed to abide by Europe's AI transparency rules. One such company is Microsoft, which has been applying its own form of watermarking to …
Nothing will seep. We live in a world where we are already tracked 24/7 in every aspect of our life, and it's the new normal since cell phones became ubiquitous, because cell phones need our (coarse) position to work. Then came the whole lot of direct attacks to privacy, both from tech bros and governments. Now with AI we have reached "big brother" level of control. NO ONE CARES.
"will users grow up thinking that applications will inject tracking tags ... ?"
As an old fart, I can only hope so.
But in my limited experience with users currently growing up, they all know that everything they do is tracked, they just don't care. In some ways I am jealous of them.
Your headline has overshot the content. The images are linked to a UUID, but many document formats assign a UUID to a file. It’s a core part of XMP metadata (used in PDF and in lots of other media files edited by Adobe tools) and many programmers will have generated and store one in a file at some point.
The question is do MS keep a list of issued UUIDs and store them against a user ID? Or do they just generate a random 128 bit value. My money is on the latter. It’s good your source has asked the question to MS, but I’m not clutching my pearls just yet.
It doesn't necessarily have to link to a particular outright user id. All it has to do is link one document to another, and then if they find one document that they don't like they look for other linked documents. If even one of those has information like a name or signature attached or was first posted online with an identifiable account, or was sent through an email with a service like Gmail or Outlook (where the provider regularly scans your content), then they can identify which user created it. As they mentioned, it's similar to The watermark that is put on pages from a laser printer. The watermark indicates the serial number of the printer, and they may not have a record or proof of who owns that serial number, but if other pages with the same serial number embedded come from a known source, then they know who printed the suspect page or is associated with them. Or if a suspect's printer is confiscated they can tell whether documents were printed on it.
Headline: "Microsoft AI watermarks in Paint and Photos are linked to user IDs"
Body text: "If Microsoft associates each prompt with the user who sent it, then the company could in theory identify users by referring to the watermark in an image."
The headline is a fact, the body is a subjunctive. If it were "Microsoft AI watermarks in Paint and Photos could be linked to user IDs" I'd be in wild agreement.
The article refers to a 16 bit integer. That may contain enough information for a company and application id. Insufficient for identifying people.
The article links to a microsoft webpage. That refers to images/ sounds to embed. That may have sufficient additional information to id people. I'd expect more data but unsure what it is to be used for.
I feel a bit dirty for defending microsoft.
Why does the MS apps need to phone home to calculate a watermark? (*) That’s something that can be done extremely easily locally.
Ok, a locally produced one won’t be guaranteed to be unique but (a) it should be statistically unique enough and (b) does it matter anyway?
(*) yes, I know the answer is “because it’s Microslop and they like spying on you” but I thought I would ask anyway
I mean, I can't really see how else they'd do it. Put it in any headers and we'd just strip them off.
This does make one wonder how well such things would survive a quick trip through ffmpeg for example. Would it still be able to identify a marked image that'd been resized down and back again, or converted to a different colour depth and back?
Based on a quick read through of a few articles, it's better to think of it as shapes in the image. resizing and recoloring a photo of a woman showing off her wedding ring wouldn't change the shape of her face or its position in relation to her hands or the orientation of each finger in relation to each other finger.
However, using AI to convert it to an oil painting and then back to a photo will change this kind of information. That kind of trick is where the watermark data might actually be modified.