Apple's bright idea for CSAM scanning could start 'persecution on a global basis' – 90+ civil rights groups More than ninety human rights groups from around the world have signed a letter condemning Apple's plans to scan devices for child sexual abuse material (CSAM) – and warned Cupertino could usher in "censorship, surveillance and persecution on a global basis." The US-based Center for Democracy and Technology organised the open …
Someone needs to tell Apple that the US is not China.
And no, I am not in the screeching minority - whatever that is. I do, however, have a brain. And so do many others.
The idea of a private, constant, secret and for-profit US Department Of Suspicion - which, in my view, is exactly what Apple is trying to become - just does not sit well with a lot of folks, myself included.
I can't wait for the federal lawsuits that will be triggered by this moronic idea if Apple doesn't backtrack and forgets about it.
The problem is there are scores of people who've never bought Apple and they are going to be tarred with the same brush - Squeaky clean Apple/Apple users, but criminals the rest of them
Anyway, no surprises here - it's something that Apple have been cultivating over the years when it comes to movies...
"Apple won't let bad guys use iPhones in movies, 'Knives Out' director Rian Johnson says"
"Trying to figure out who the bad guy is in a movie? Look for the character that hasn't touched an iPhone."
https://www.cnbc.com/2020/02/26/apple-wont-let-bad-guys-use-iphones-in-movies-says-knives-out-director.html
Isn't it more a case of "Apple won't pay you for product placement if you let the bad guys use iPhones"?
It's not like every car manufacturer has to approve the use of their vehicles in films.
I know the 90s were a different time but I doubt Jaguar approved "For men who'd like hand jobs from beautiful women they hardly know." https://www.youtube.com/watch?v=XzyNPoI17rE
Someone needs to tell the US that the US is not China.
"The idea of a private, constant, secret and for-profit US Department Of Suspicion"
How about a not-for-profit group?
Erm, seems you don't know much about democracy. Also, wasn't Trump the one who was big buddies with Putin, Xi and Kim?
Presumably you conflate public good and safety and authoritarianism. You need to read your Constitution, as do Trump and the majority of his supporters.
To be honest, both parties are authoritarian. Even US Libertarians are authoritarian. Not that the parties have authoritarian elements, or leanings, they are straight up authoritarian. The main difference is flavor of authoritarianism: Corporate or Government.
There is no non-authoritarian element in US politics; Bernie is as close as we get, and he's not that close.
I can't wait for the federal lawsuits that will be triggered by this moronic idea if Apple doesn't backtrack and forgets about it.
Cat's out of the bag. Would you believe Apple if it said it changed it mind?
I can see potentially lot of pressure from LE and/or governments now that Apple has dangled this "backdoor" into the "unbreakable" iDevices.
Every image which goes into an MS Office document/note/presentation is subject to the same kind of checks because the image is uploaded to Microsoft’s cloud in order to add alt tags using AI analysis, even if you’re NOT using SharePoint or OneDrive to store the actual document. Microsoft uses PhotoDNA which uses a similarly broken algorithm and does NOT adequately disclose its use outside of OneDrive nor does it explain how it works.
If we decide it’s wrong when Apple does it then it is definitely wrong if Microsoft does it!
Sorry, but (like all other cloud providers and facebook and many others)
Microsoft scans for CSAM *on their servers*,
Apple wants to scan *on your devices*,
plus iMessage filtering and have Siri and Search detecting "unsafe situations" and "searches for CSAM-related topics.
To me, that's a real massive intrusion of privacy and security, which can and most certainly will be abused not only in russia and china.
(sorry for errors, I'm no english native)
If they scan for it on their servers, aren't they, because of strict liability, also guilty of posession?
Imagine if someone rents a flat and has some drugs hidden in the closet. Should landlord or agency be coming over every now and then and do a thorough search? In a way this is a server side scanning.
I guess users agree to that by accepting the terms of service.
Personal device, just like your own home should be off limits unless there is a warrant.
Some people do mental gymnastics and try to justify it by saying that Apple wants to maintain end to end encryption and scanning on the client is the only way to maintain privacy. But that just sounds like Orwellian doublespeak. If they really want to scan through people stuff and they cannot do it with end to end encryption (aka going to someone's rented flat), then perhaps they should stop offering such service altogether.
"Every image which goes into an MS Office document/note/presentation is subject to the same kind of checks because the image is uploaded to Microsoft’s cloud"
Even if the PC is off-line?
Not that I wouldn't put it past them if they got the chance. As I recall from the early days of W10 their T&Cs were artfully worded so as not to exclude them from almost anything.
"Now infosec bods are reverse-engineering the technology to see how it really ticks. If it can be shown that the image-matching algorithm can be fooled into flagging up an entirely innocent picture, such as via a hash collision, it will render the content-scanning tech of limited use for its stated purpose."
This article is slightly behind the Register article earlier today about someone already did that. It took less than a day after the public part of the algorithm was released.
Apple isn't this dumb. This is a 'think of the children' smokescreen to get us on the slippery slope that countries like the PRC are strong-arming them into, but trying to save face with this farce/smokescreen of a feature.
I found this article in the Washington Post about a parallel project and the finding that the technique is inherently dual-use, like a knife. Meaning it's nigh-impossible to prevent it being abused, especially by a sovereign power. As I've said before, I think the only reason Apple are announcing it now is because they are being pressured into including it: likely by China.
https://www.washingtonpost.com/opinions/2021/08/19/apple-csam-abuse-encryption-security-privacy-dangerous/
The German Governments Media Agenda committee said Apple should remove it immediately and:
“größte Dammbruch für die Vertraulichkeit der Kommunikation, den wir seit der Erfindung des Internets erleben” (The biggest dam burst for the trust in communications, that we have seen since the invention of the Internet.)
“Every scanned content destroys the trust of users, that their communications are not being monitored. Without trusted communications, the Internet will become the biggest surveillance instrument in history.”
Whatever politicans say, it's not worth a penny,
they change their minds more often than their underwear.
In this case I guess, as soon as the "Innenminister", police and secret services have been talking about the great opportunities to fight terrorism, drugsmugglers and other serious crimes (like violation of copyrights), then the rest of the "Bundesregierung" will accept this tech - of course with "Bauchschmerzen".
(I'm german, sorry for errors in my english)
re think of the children.
Humans will review all suspect photos? long queue of paedos applying for that job.
That will happen, as to whether we get to hear about it is of course up to Apple, who typically insist, when they are dragged through the courts, that the details remain private.
Coming at this from the angle that Apple is pushing, I think everyone with a functioning, modern moral compass can agree that harming innocents—children or otherwise—is a Bad Thing. But I personally do not think it is or should be Apple's responsibility to become the Arbiter of All that is Good. Companies and governments should stay well away from private citizens, even if it means that Bad Things can potentially occur. In addition, I believe the protection of our youth should be the responsibility of the parent(s) and the community around them, not a faceless international megacorp using proven less-than-reliable automated detection methods.
I also have reservations with the idea that someone dumb enough to store such unmentionable, illegal files on their iCloud would also be the type to actually produce such content. How will it help law enforcement find and capture active producers? Will their technology even help stop the harm of children? Given the sensitive nature of the matter, I doubt we would ever know should the technology be utilized for real. And that makes it even less trustworthy.
Also, in many cases, the parents either turn a blind eye or are themselves involved.
There was a case about 2 years back, where a couple was pimping out their sub-teen daughter.
Likewise, a couple of weeks back, there was a judgement passed on a mother of a boy, where her partner had been abusing him for years, and selling him out, and filming and uploading the abuse - this is the so called Schrebegarten scandal in Germany from 2019, where a ring was being run out of a chalet on an allotment. The mother found out her child was being abused, but didn't do anything about it.
So, leaving it up to parents isn't always a good idea.
Certainly parents should be responsible for a child's safety and for ensuring it knows the difference between right and wrong, instilling discipline (non-violently) etc. But there are plenty of abusive parents (and spouses, for that matter) out there, that the state and the police have to take action.
I downvoted because, while it's terrible that that happened, "think of the children" is not a valid excuse to allow private corporations such power. They already have enough as it is.
Pray tell, what would Apple's setup here provide? Again, I very much doubt that the perp would have uploaded the content to iCloud or similar, and even if he did, if the hash isn't indexed it won't be detected. In either case, there are other, less incriminating places to store such things that I would think someone daring enough to pimp out their own child would opt to use. And on that note I don't see how Apple's technologies are going to help stop face-to-face pimping of children.
I can't speak for Germany on what they need to do to help curb such behavior, but it ain't this. And no matter what one does, there will always be those that break the rules and get away with it long enough to commit atrocities. It is a part of life and we should not let emotional reaction to sensational incidents guide us to making poor decisions that affect a large percent of the populace.
Apple are proposing searching iDevices for known CSAM images.
Anyone abusing kids to create new images cannot be detected this way, by definition.
So one fairly obvious consequence is an increase in abuse as people try to create new, undetectable images by abusing more, new victims.
As a parent, that terrifies me.
That is actually a really good point. Make the usual stuff harder to find by busting the low-level consumers and possibly distributors, and there will be someone out there driven enough to fill the power vaccuum. I wonder if there is any research on that?
I absolutely applaud the intent behind what Apple is doing, but even the people who invented this system said that it would be very easy to adapt to less noble goals and shouldn't be used without significant safeguards in place. I'm not convinced, and it seems I'm in good company, that Apple has done this. Maybe at some point Apple will manage to convince us that their process is secure, but it kind of feels like some cattle rustlers already made off with the cattle, burned the barn down, and then paved over and built a high rise office park in the barn's place on that issue.
Fail icon because someone already used the mushroom cloud and I was torn between the two.
What if motor manufacturers built-in tracking systems? Great for tracking down speeders (good) but also great for who attended what rave or political meeting. Where data is banked against the time it might be 'needed' by some enforcement agency, there is no opportunity to defend the most obvious defence "My car may have been there but so what?" Joe McCarthy lives! According to our records (sorry citizen, you can't get a complete copy of what we (and our partners) hold on you) you're a heterodox recidivist. Your car can be as good as a fingerprint even if it is another driver. As AI gets more prevalent (I didn't say more sophisticated) there will be many more false connections and it will be up to YOU to come up with a defence against a fluffy snowstorm of 'evidence'.
Are you even paying attention to the outside world? As we speak your ISP is likely tracking your every page visit and selling that information to advertisers, and depending on your locale that data may be linked to your full name, email address, etc. It's 100% legal in the USA to track complete web history for I believe up to 3 months, ever since Ajit Pai wrecked the place and got rid of Net Neutrality.
And it is more than trivial to buy a matching record from yet another completely legal information broker, which contains not only your name but any past/married names, all the addresses you've ever lived at (i.e. registered with your governace), your family and their information, your pets, your job history, employment status, tax bracket, and who knows what else. I know this for FACT because I used to work for a company that had unlimited access to Acxiom's databases, and it had EVERYONE and I mean EVERYONE I ever searched for.
Your very life is just another record in a database, and that should scare you. Giving companies even more power to track us is not what we need. Of course your government likely has plenty of dirt on you, but at least there's some assurance that they probably don't want to share that trove of data with the likes of say Apple. God help us if there were a leak of that data and it got merged with other databases of information—millions of people would have their entire identity and history up for grabs, from top to bottom.
The problem is simple. People need to stop putting forth proposals to police and monitor the majority of people in order to catch a very small minority of perpetrators.
As with most social ills, many people take it personally (perhaps they have abducted family, for one horrid example), and have no issue with tromping over everyone's rights to get revenge and a "magic way" to stop it from happening again.
There are no "magic solutions" that aren't more reminiscent of 1984 than any free society history has ever dreamed up.
Other cloud companies have been dealing with this problem for years. They check what is loaded onto their servers, as it is loaded.
They don't break the sanctity of the local device.
Apple has just moved the discussion on from "the local device is sacrosanct" to "anyone can search for anything they want on the user's devices" (if they apply enough pressure).
Yes! Finally someone tells it like it is!
Apple users are abusing children left right and center. If Apple doesn't spy on them, it's failing in its duty to stop child abuse by its customers.
Why should Apple, even have to tell its customers about the scanning? Since when do you tell criminals they are about to be caught. It should just be free to scan their phones whenever it wants for whatever it wants to save the children from abuse.
It should be free to monitor their cameras, and record their microphones to protect children from being abused by Apple's customers.
If Apple doesn't search its customers phones, they will abuse children. Why, they are probably abusing children right now, why the delay!?
You are the one person here to speak the truth, that Apple customers abuse children.
It is exactly like the Taliban forcing women to cover up completely lest Apple users get excited and molest children. Or the Tellyban forcing OnlyFans users to cover up, lest Apple users get excited and molest children.
If Apple don't save the world from Apple users and their molestering, then the USA has failed in Afganistan.
Where is the line then?
In other cultures children may run around in their birthday suit on the beach until approaching puberty.
At least they used to back in my day.
My point is that US/UK hysterics about nakedness could result in thousands of completely innocent images being classified as "indecent", or whatever is the term.
The thing that struck me about this when I first heard of it is that they were scanning the uploads and see if they matched against a database of known images. Surely this idea will put more children at risk when the offenders have to start making more *new* images to share that aren't currently known about by the authorities to work round this matching process.
As long as you don't send those pictures to an adolescent's device, you are fine.
The CSAM is a finite list of known abuse images that have been confiscated from known offenders. The signatures for these images used to check images on cloud services (or in Apple's case, on the local device, before they are uploaded), so, unless you somehow manage to recreate one of those images, you are okay.
The content warning is between child and parent, if the child receives (or tries to send) photos including nudity, they will be warned and if they continue to send/view the image, the parent will be informed.
"The content warning is between child and parent,"
The announcement at https://www.apple.com/child-safety/ does indeed mention warning parents but it also says
CSAM detection will help Apple provide valuable information to law enforcement on collections of CSAM in iCloud Photos.
But Apple can change their minds at any time. What if their politics decide that the dad taking photos is child abuse later?
Arguments about what a company/government/person claims they won't do today don't hold water. What matters is what a tool can be used for. Minds change. Politics change. What is a crime, changes. History both shows and teaches us this, and failure to learn those lessons will bring tears.
Read up on how why the Nobel Peace Prize was created.
This genie will not get put back in the bottle, especially when there are powerful governments around the world that will very, very much want it to be let out and will use their economic might to resist efforts to try to put the stopper back in.
Indeed, we're literally now at the "prove you didn't take naked pics of your kids because you're a pervert" stage.
At the very root of this, we need to ignore the big debate and concentrate on the fact that Apple are illegally scanning our images. There is definitely criminal activity here, but it's not the users, it's Apple.
Not if the algo matches them. Apple and others have switched from straight MD5s to more AI based fuzzy matches.
He shouldn't be photographing his kids naked, that CSAM set is marketed as child abuse, but that's the marketing not the reality.
It's an *expansive* set of images, viewed as sexual abuse by the *reviewers* that put them in. The *most* images they can include. The *widest* possible definitions.
If the reviewer is turned on by his naked kids in the bath, then to him thats a child abuse photo, and his kids photos can end up in that database.
Imagine a childless sexless sad git in a dark office, who, for some reason chose a job where he see child porn all day. Do you really want him seeing your kid's shit covered ass on a changing mat, mixed in with the actual child porn he's reviewing? No. So don't take that photo.
As far as I can deduce hash matching is for "known CSAM", and the AI is only for the child protection scheme where parents are informed if their kids are taking nudes—which is surprisingly common, and something I am in favor of informing parents about (as long as the picture doesn't get back to Apple).
Do some digging and sites like Omegle were apparently once rife with those of questionable age using their iThing to record theirThings when strangers asked them to. If there were a way to inform parents about that, we could save a lot of kids from future regrets, and hopefully from future contact with potentially dangerous sexperverts asking to come over when mommy and daddy are on vacation...
A few years ago a (female) friend of mine sent me a pic of her little ones - naked in the bath. I deleted the pic, then wiped and hard-reset my phone. I then deleted all iCloud backups, because you never know. Ultimately I couldn't deal with the stress that it might have survived all of this and be stored 'somewhere'; I ended up trashing the phone (iPhone SE), deleting my iCloud account and creating a new one.
Paranoid possibly, overreaction almost certainly; but it needed to be done for my peace of mind otherwise I'd always have that feeling that it was buried somewhere, ready to be flagged and studied by some grubby investigator, right before my life was ruined forever. And this - THIS - is the kind of stress that intrusive, faceless, unchallengeable surveillance can give people. It's vile.
And yes, I was very angry with the friend that sent me the pic. On your own device, probably ok. On somebody else's? Quite possibly criminal. Why the hell she thought it was ok to send it I will never know, but she learned quickly that it was very very not ok.
This post has been deleted by its author
Care to reference the exact wording? Because as far as my IANAL brain knows, problems come from creation, posession, and transferring to someone else. I vaguely remember a case where a man unknowingly downloaded lots of CP when he visited a shady site, and investigators pulled it off his drive—he was not charged since it was transferred to him without his express consent or knowledge, and he made no attempt to further transfer it to anyone else. Unfortunately I am not getting any hits else I would link it.
So I am a loving father of two kids, and yes I have naked pics of them in the bath, in the pool etc
As will many, probably almost all, parents. It's normal and one could argue that not having such photos would be abnormal.
BUT in the UK it's illegal - and it's a strict liability offence meaning that there is no allowance for "but it's just being a normal parent" or any other similar defence. So technically, being a loving and normal parent could end up with you on the sex offender register, your life turned upside down, your family torn apart, etc, etc. Even the files not being accessible to you, or being files you don't knwo the existence of, isn't technically a defence.
Even if at some point the police of CPS decide not to pursue the case, you'll have had all your computers taken for examination (don't worry, you'll get them back ... sometime ... perhaps a few years later ... and maybe even still working), you'll have lost your job because no employer want to be seen employing a known paedophile, you'll never be able to go anywhere without the neighbours looking at you in a "funny way" because clearly there's no smoke without fire, and the kids will be traumatised by all that's going on. But that's OK, it's all done because "think about the kids".
And no-one can argue that such a situation is absurd - because to argue against any measure brought in to "think about the kids" must mean that you are thinking about the kids in the wrong way. That's how things like this happen - the wedge goes in, and bit by bit it's driven in harder and harder, and people argue "think about the kids" until it's too late to get the wedge out when they realise what a crazy situation they've created.
"That's how things like this happen - the wedge goes in, and bit by bit it's driven in harder and harder, and people argue "think about the kids" until it's too late to get the wedge out when they realise what a crazy situation they've created."
What if they never realize the situation...or worse, see the burning room and simply go, "This is fine"?
"BUT in the UK it's illegal - and it's a strict liability offence meaning that there is no allowance for "but it's just being a normal parent" or any other similar defence."
Not even, "It was planted on me"? Because otherwise, what's to stop someone dead-dropping a picture of a naked cherub (innocent enough, but just "guilty" enough, too) to every member of Parliament or something?
"As will many, probably almost all, parents. It's normal and one could argue that not having such photos would be abnormal."
Based on a sample of how many?
We certainly didn't take any photos of ours like that. I'm not aware of any other parents I know having done so. Maybe it's a generational thing.
"Based on a sample of how many?
We certainly didn't take any photos of ours like that. I'm not aware of any other parents I know having done so. Maybe it's a generational thing."
As I said above: It's a cultural thing. And, obviously, culture can also change with time.
Certainly no-one cared one bit about kids running around naked on the beach back in the 70s. Or the parents takin a few snaps when they were playing. Things might have been different in Britain.
Apple still haven't given any good reason why this scanning need to be done on the device and not on Apples own servers, if it is only concerned with images uploaded to the iCloud and not about looking for stuff on the device itself, why not just scan the photos that are uploaded to icloud when they hit the icloud server?
You are incorrect. They are encrypted after being uploaded with Apple's key. Apple can decrypt them and has done so repeatedly to comply with warrants from law enforcement. Apple has not announced any plans to change this. If they do, which again they have not, it would not justify on-device scanning.
One angle is the abuse of this scanning technology for nefarious ends, but it needs to be stressed that whether you're for or against it - this isn't the government. It's a private company:
A) Usurping policing powers. Private citizens don't get the remit to take it upon themselves to demand access to people's person and effects to search for and report evidence of crimes. Apple is a private corporation, not a government.
B) Hacking or Extortion (depending on your point of view) The option to 'not update' is not an option at all, because security updates are not unbundled from the feature updates. I can't choose to keep current functionality but keep my device secure. My choice is to either accept this awful practice, or render my phone insecure and unfit for purpose.
This is a whole different (and more fundamental) problem than whether an _authorized_ entity like a government should or shouldn't do these things on my phone.
All I'll say is... if no-one stands up for your rights, then you're stuffed.
You may wish to disagree with the 'screeching minority' all you like, but don't conflate their concerns about individual privacy and the flagged issues with implicit agreement of what the 'thing' is attempting to solve.
I'll rather continue to be a 'screeching minority' and be proven wrong, than stand on the side of those just standing by saying 'oh, just suck it up already' until the 'screeching minority' is proven right when it's too late and everyone screaming 'how did this happen!'
Sorry. Not sorry.
The Register 
Biting the hand that feeds IT
Copyright. All rights reserved © 1998–2025
