back to article Government-favoured child safety app warned it could violate the UK's Investigatory Powers Act with message-scanning tech

A company repeatedly endorsed by ministers backing the UK's Online Safety Bill was warned by its lawyers that its technology could breach the Investigatory Powers Act's ban on unlawful interception of communications, The Register can reveal. SafeToNet, a content-scanning startup whose product is aimed at parents and uses AI to …

  1. nichomach
    Big Brother

    For goodness' sake...

    ...don't encourage them to overhaul surveillance laws with the Home Sec currently in full Bond Villain mode!

    https://inews.co.uk/opinion/priti-patel-anti-protest-powers-stuffed-policing-bill-1316830

  2. Our Lord and Savior Rahl

    I am not certain that it tracks fully, as presumably they'd have the consent of the Childs legal guardian and by definition, that would be sufficient.

    I'm fairly sure that would apply right up until the age of majority, which means the only people affected would be at the tail end of sixth form.

    1. iron Silver badge

      If you had read the article, they require the consent of the sender of the message so consent from the parent or guardian of the child recieving the message does not help.

    2. Doctor Syntax Silver badge

      That might apply to outbound messages but TFA says specifically that the sender's permission is required, not the receiver's. On this interpretation it makes it illegal for inbound messages.

      But wait...what are the consequences for ordinary email spam filtering?

      1. Blazde Silver badge

        This is a ridiculous interpretation of the law. Surely if you have permission from the recipient to intercept then you are simply acting as their agent and the law applies as if the intended recipient 'intercepted' it, so to speak. They might need some extra legalese to nail down that relationship, but all kinds of similar arrangements would be unlawful if you couldn't do this.

        ('Aah that is good coffee. Now I'm ready for you to read me my email Jeeves. Do the silly voices again would you. And skip the parts where mother tells me to get a job, good chap.')

        1. Doctor Syntax Silver badge

          It may be ridiculous but we don't actually know if it's invalid until it's been tested in court. Do we know if it has?

          1. Blazde Silver badge

            It's never going to be tested in court if it's ridiculous enough. The offence has effectively been in place since RIPA 2000 came into force and any dumb prosecutions would surely have been reported by El Reg.

            1. Doctor Syntax Silver badge

              I've seen the occasional case that was too ridiculous to end up in court ending up in court. In any case my point still stands: without a ruling there's no way of knowing whether the interpretation is valid.

          2. Sirius Lee

            In this case I don't think your point is valid. If the recipient of emails wishes, the content of all messages can be routinely processed for viruses and spam by a processor. Sometimes that processor is the recipient but most times it is an email service provider such as Google or Microsoft. We don't ask the recipient if its OK for their message to be checked, it just happens.

            If an application checking the content destined for minors cannot be checked by an app like the one mentioned in the article then it cannot be checked for viruses and spam either. The laws referenced in the article and virus/spam checking processes have co-existed for decades without email service providers being hauled before the courts.

            Likewise, the case that such processing might cause defamation seems also spurious. If the assertion that a message might contain offensive material is considered to be defamation presumably the assertion that an email contains a virus is no less offensive but again, no recipients have been prosecuted on this basis. Why not? Because no court would accept it.

            So it seems to me one of the other problems a start-up company has to look out for are over enthusiastic lawyers looking for a gullible company to pay them some cash.

            The assertion of the CEO that he could go to jail is also nonsense. Presumably the app is developed by a limited liability company. If so, the only reason time in prison is possible is if the CEO is fraudulent or there is a case of corporate manslaughter. The former has nothing to do with an app while the second is unlikely to result from the use of an application (or at least it would be even more difficult to prove than when a train crashes killing passengers).

        2. Graham Cobb Silver badge

          IANAL but I believe the intent is that, of course, the receiver can choose to scan, and automatically delete, their messages once they have received them but no one on the path from sender to recipient can look at the message content without permission of the sender until it is "received".

          The answer to this legal conundrum is to employ fully encrypted email. It would then be simple to declare that anyone to whom the receiver has given the key is delegated by the receiver to scan and process their email (within the terms of their agreement, of course). That would permit spam blocking and would also permit the recipient (but not their parent) to authorise someone to block any emails they wanted.

      2. John Brown (no body) Silver badge

        "That might apply to outbound messages but TFA says specifically that the sender's permission is required, not the receiver's. On this interpretation it makes it illegal for inbound messages."

        On the other hand, the sender is sending to a minor and the legal guardian is installing and authorising the checking of what is being sent to that minor. This is why lawyers get paid the big money to argue the finer points of law.

      3. JohnG

        That's nonsensical. This interpretation would imply that the systems and people engaged in filtering incoming and outgoing messages for malware, porn, disclosures of IP, etc., for any organisation would have to first get permission from both senders and recipients - and this is clearly not the case. When I was the custodian of security devices that blocked incoming emails containing malware or phishing, I didn't first seek permission from those who sent them nor from the intended recipients.

    3. SPARKESFRANKIE66

      IANAL, but it's not that simple.

      Section 3 creates the offence of unlawful interception and refers to Section 6 for the definition of "lawful authority". Section 3(2) doesn't apply as a parent cannot control the other party's use of the messaging service.

      Section 6 specifies when a person has lawful authority. Warrants referenced in 6(1)(a) and 6(1)(c) do not apply here, so we fall back to 6(1)(b), which refers to Sections 44–52.

      Of these, 44 refers to interception with consent; 45–48 refer to interception for business or enforcement purposes by the postal services, OFCOM, or businesses; 49–51 refer to interception in institutions such as prisons, immigration detention centres, and psychiatric hospitals; and 52 is in accordance with overseas requests.

      Section 44 is the only one that reasonably applies and says that, except for an authorisation under Part 2 of RIPA where only one party need consent, both parties must consent to the interception.

      In any case, Section 1 of the Computer Misuse Act would remain a sticky wicket if the sender withheld consent, and because the CMA is written so broadly (for better or worse), adding an exception to Section 1 where the legal guardian of a child has consented would cause more problems than it solved, especially if the intercepter for whatever reason turned rogue.

      1. Blazde Silver badge

        (IANAL either) but I don't see recipient-authorised email-scanning passes Section 3(2), parent involved or not.

        I suppose it hinges on whether the relevant 'private telecommunication system' is 'my email address' in which case I surely have the right to control it's operation and any service providers I designate to scan my email, trash it, forward it to me, or automatically broadcast it straight on Twitter are simply operating it with my authority.

        Or whether the 'private/public telecommunication system' is 'the creaking email system in general' or even 'the internet', in which case the sender could somehow have an expectation that what..? only 1990s email technology were being used to transmit the email and no scanning takes places? (Or the switches and SMTP servers are unlawful too and the magic IPA-immune fairies are supposed to transmit it?). That still doesn't make sense, but conceivably it's an interpretation a supremely tech-illiterate judge might reach forcing costly appeals and an over-cautious lawyer might advise about that.

      2. Blazde Silver badge

        Re-reading along side the CPS guidance I struggle to see what the issue is. Section 45 gives authorisation for 'telecommunications services' for 'purposes relating to the provision of services or facilities aimed at preventing or restricting the viewing or publication of the content of communications transmitted'. Section 261 defines 'telecommunications service' very broadly and surely covers things like 3rd party spam-filtering even if they're not the primary provider of the telecommunications system.

        We're not told exactly what feature SafeToNet planned. I can only see a problem if it's something along lines of a 'hack back'? That doesn't sound like the case though so I think they should get new lawyers.

      3. david1024

        Point here...

        Seems a lot like an additional spam filter or the junk email box on my phone. Emails are routinely mangled there. Missing pictures, tracking jpgs deleted, all sorts of mayhem... And what if someone sends me an html5 email and my email client renders it improperly....

        Is that all now illegal too? Bah. Hardly.

        Since the company has a vested interest in seeing this play out in court favorably.... They should press hard for a favorable interpretation --- especially if they have good lawyers.

  3. The Man Who Fell To Earth Silver badge
    FAIL

    Roads

    "The road to tyranny is paved with good intentions."

    1. Pascal Monett Silver badge
      Coat

      Paving's looking mighty fine 'round here.

    2. Avatar of They
      Thumb Up

      Re: Roads

      Who knew the Tories (and their voters) were excellent civil engineers and road builders. Eton should advertise that.

  4. Yet Another Anonymous coward Silver badge

    Solution

    Declare all children terrorists (personal experience of nephews confirms this) then you can spy on them all you want without any rules

    1. tiggity Silver badge

      Re: Solution

      I'm probably declared a terrorist for holding the idea that Palestine has a right to exist & Palestinians should not be murdered at will by an occupying state...

      Given that Hamas effectively control the Gaza strip & UK regard Hamas (political wing, not just military wing) as terrorist groups then I'm sure that's enough excuse for them to define me as a terrorist, as they will be more than happy to conflate supporting the human rights of Palestinians with support for military wing of Hamas.

      .. So, UK govt can readily find an excuse for most people with a scintilla of compassion to be labelled a terrorist if they feel like it.

      1. Al fazed Bronze badge
        WTF?

        Re: Solution

        were doing it to safeguard children

        ?

        ALF

      2. Cederic Silver badge

        Re: Solution

        No, you're not declared a terrorist for that. If you support firing unguided munitions at civilian populations then you'll be declared a terrorist.

        I hope that helps clarify matters.

      3. msobkow Silver badge

        Re: Solution

        Same everywhere in the world. My favorite is how I get called "anti-semitic" for criticizing the Israeli government. Since when are governments going to church or synagogues? They're ALL amoral entities that don't give one whit about peoples' ideas of "right" and "wrong."

  5. Anonymous Coward
    Anonymous Coward

    So, this is an example of RIPA doing the right thing?

  6. Jonathon Green

    “…we were intercepting incoming messages, without the authority of the person that had sent it in the first place.”

    […]

    “And so what seemed a pretty obvious thing to do – why wouldn't you be allowed to do that – you know, it just put the fear of God in me."

    I was going to say that words failed me. But then it turned out that words hadn’t failed me at all as words like “Good.” and phrases like ‘Have you considered applying for a job in the home office?” Immediately came to mind.

    1. Yet Another Anonymous coward Silver badge

      So I sign my 5 year old up for "My Little Pony Online" and click TOS to agree that her messages are censored, but I can't have the system block dick pics until she opens them because it violates the privacy of the person sending the dick pics ?

      1. Jonathon Green

        Well, the first question there would be “Why are you signing your 5 year old up to a service which allows arbitrary, unvetted individuals (and I’d include other 5 year olds in that category…) to send messages to them directly?”

        1. Cederic Silver badge

          Especially that particular service, given the size and nature of the MLP online fandom.

  7. Pascal Monett Silver badge

    Ah, there it is : think of the children

    "Law enforcement bodies such as the National Crime Agency claim that wider adoption of E2EE will stop them from detecting paedophiles preying on children through messaging apps"

    Because of course they do.

    I'd like to know how many paedos have been caught thanks to this indispensable privacy-violating attitude.

    This is the favorite excuse for snooping, but I've never heard an official declaration stating "We have caught X criminals with this technology".

    So, out with it. How many are now behind bars because of your snooping in everyone's lives ?

    1. Anonymous Coward
      Anonymous Coward

      Re: Ah, there it is : think of the children

      The Home Office periodically trots out figures on the number of terrorist attacks thwarted... it's always a low number... and you always assume that most of them are down to stupidity, at least in bringing themselves to the attention of the authorities

    2. msobkow Silver badge

      Re: Ah, there it is : think of the children

      Well, here in Canada there are regularly articles about people being caught and arrested for such crimes because they were caught by the RCMP divisions that monitor for such activity. So at least here in Canada, yes, SOME of the "think of the children" mantra really DOES have an impact in the real world.

  8. CountCadaver Silver badge

    V for Vendetta / 1984 incoming

    The more time goes on the more convinced I am that we are rapidly spiralling towards some hideous hybrid of various dystopias including 1984 and V for Vendetta....

    Though instead of mandating telescreens, they've persuaded the proles to voluntarily buy them through the medium of "home assistants" something that could be useful with proper privacy safeguards but instead have become an electronic spy in the homes of its users, just waiting to unleash a treasure trove of context free information for the authorities to "prove" wrongdoing.....

  9. Anonymous Coward
    Anonymous Coward

    Please, please intercept this communication...and much good may it do you!

    Quote: "...unlawful interception of communications..."

    *

    Some of us don't care! Because we like our personal privacy and our personal security, we encrypt everything before any message goes near a public channel. Take your pick...AES, IDEA, Blowfish.....or my favourite, our own private triple encrypted book cipher (sample below). Snoops are welcome to tell EL Reg readers what the message says in plain text (i.e. you are authorised!!).

    *

    uv2JG5ef4PcVi1mzCh2Ns54L8T0R6VqNihSR2XqNoxyxINgZwRoZOV2paXSNWvcVWjwXiNgfKNiJ

    eXYVW3ulCHujGXE1gDyl4dOHQN2nc5O3ElKNk907KfmrKDQnkDo1U5UFCFIroTURqREFUVUhqD2z

    YZ4Rgdul0JqhiJSNC5yZ6tMluno3ePaRklwx674b07SjIJuZ8ZyfePYxOHqTCPY3a1M9KdWj0TQz

    YBG5glQ3uv8BsTW5sdaLC5mdufIfYdKJa7CBqFSfqTw94t0BcfKvyz4RydKDSXgLMTWtkXUVgXUn

    6bGZ0jOZY1OfgvM1mhCxMfSPgX0rqB8naDK36D4Ze5I7eRW14TglwFg3E9cTMjoVs9AJyFwjOReH

    G5o50ZKviXCJYjiFQ7apUXwdIdwR6Z09UFkLAXghm3yjELQXgNQp63gPSBEDKbAVqNOLOBUJcNwr

    Q5uhwlgD0vWrkrkLuPYZ4f2j0xmn0ZWt0juxU5ipsRwTqL4hevYby5UJmhIVu5WH81YXit8doTSV

    6501gNqvIrCjqRMHKHghiHm5k9YpsFA7

    *

    1. Anonymous Coward
      Anonymous Coward

      Re: Please, please intercept this communication...and much good may it do you!

      Could you contemplate getting over yourself for a second?

  10. tiggity Silver badge

    Any software praised by "senior politicians" wrt "child abuse" was almost bound to be privacy infringing - MPs love their privacy but don't want anyone else to have any and think of the children is always their go to excuse for shafting citizens rights* / privacy.

    UK govt (and Labour have been equally as bad when they had the chance - stares at Blunkett for example) seem to have a target of UK plebs* to have less privacy and rights than East Germans did in the Berlin wall days.

    * rules only to apply to the "little people", just like taxes

    1. Doctor Syntax Silver badge

      The Home Office is particularly effective at house training new Home Secs. Sir Humphrey wasn't in the same league.

  11. Chris G Silver badge

    Every time I hear about governments and cops talking about the detection of paedophiles via online snooping, the thing that is missing is any figures that indicate why they are so concerned.

    Some years back, a member of my family attempted suicide as a young teen, it turned out she had been abused by a neighbour for about four years.

    During the investigation on talking to one of the senior officers, she explained that often abusers are either family members or friends and neighbours of the family, given that, how often are children targeted by paedophiles online and what are the statistics?

    I have a feeling that if authorities and police really wanted to tackle the problem, one of the first things would be via education, after all crime prevention is supposed to be the prime mover for police forces, being able to snoop on everyone at will sounds more like a fishing expedition to me.

    Of course if things have changed and child molesters have moved online, then a major think tank needs to look at practical ways to deal with it without destroying society's basic freedoms.

    1. Anonymous Coward
      Anonymous Coward

      This is the one thing the people in charge never really want to publish

      That 95% of child abuse is from people the child's family already knows

      An ex-friend of mine was done for child abuse, basically doing the abuse for filming by a third party

      The child he was abusing belonged to the people doing the filming and he knew the family.

      And it was done for earning easy money for selling the film.

      Having online snooping would not of prevented that child from being abused.

      And they were only caught because a teacher at the child's school went "where'd you get these bruises?'

      1. Cav

        "This is the one thing the people in charge never really want to publish

        That 95% of child abuse is from people the child's family already knows"

        Not true. That fact is not kept secret and is oft repeated.

  12. FuzzyTheBear
    Stop

    not what it seems

    The real problem is not what it seems. it's about education. it's about being responsible adults that educate the kids. Surprised ?

    If you snoop .. you snoop .. email or listening on the telephone or anything else , snooping is snooping. stop it.

    If you educate your kids , teach them how to say " hey .. dad .. something's not right .. " hey mom .. i got an invite , can you check it for me ?

    and teach him to rely on your judgment when something is odd to them well you done your job.

    You got to build trust between you and your kid.

    Snooping just shows you don't trust the kid.

    Teach , be a parent and all this is useless and you will have given the kids the tools they need to get by in life.

    Really .. snooping in " parental controls " is just for parents who don't do their jobs of educating their kids.

    Period. Discussion closed.

    1. The Basis of everything is...

      Re: not what it seems

      So you expect kids to exercise perfect judgement on every email or message they receive?

      Every few months I have to go through corporate don't be a twit training about spam emails. In an IT company of 1000's. And we still get fskin eejits opening the few phishing mails that get through and clicking on the links. Grown, educated, IT literate adults who should know better. Think about it.

      1. Al fazed Bronze badge
        WTF?

        Re: not what it seems

        There may be something wrong with the way you are attempting to train eejits who probably shouldn't have been employed in that eMail reading/sending role in the first place.

        Do you use a "ready made" training software for the eejits ?

        ALF

  13. Anonymous Coward
    Anonymous Coward

    Which part of the process don't you understand?

    @tiggity

    @Chris_G

    *

    Surely you don't want our political class to soil their hands with ANALYSIS?

    *

    Oh no........they TELL us what's wrong.....they TELL us what they plan (not) to do about it.....

    *

    ......and we plebs just SNAP TO ATTENTION!!

    *

    Which part of the process don't you understand?

  14. Anonymous Coward
    Anonymous Coward

    My device

    I feel strongly that if a message comes onto my device, it is now my property to scan/delete/share as I wish. So if I give permission for an app to scan and delete a message on my device - the rest of the world can f off. I don't "have to read it" if I don't want to - or if I allow an app to scan and delete it from my device, it is no different than if I did it - so what ever.

    1. Chris G Silver badge

      Re: My device

      @AC, the fundamental aspect you are missing is that the app intercepts the message so that anything naughty does not reach the recipient's (child's) device, at least that is how I read it in the article.

  15. Vimes

    '"We were a very young startup then but it worries me that those that don't have the finance to get professional advice will cut corners and innocently/naively breach laws like [the Computer Misuse Act] etc. We see that all the time, especially with international safety tech providers entering the UK market. They often have no idea these laws exist."'

    Really? Seriously?

    I was able to contribute towards stopping a telecoms company from introducing scanning tech provided by an Israeli company. If a mere member of the public with zero legal experience and zero funding can understand enough about the law to do that then there is nothing to stop companies from knowing enough to see the potential pitfalls.

    In my opinion If they don't know about the law by now then in all likelihood it's because they don't want to know. This is especially the case when you consider some of the laws have been around for more than thirty years (even RIPA - now the IPA - has been around for more than two decades). Their existance should really not come as such a shock.

    The government even publishes the damned laws online at legislation.gov.uk for anybody to read for crying out loud. This stuff is really not so difficult to find once you go looking.

    1. msobkow Silver badge

      As a recent launcher of a home business doing consulting, I must say the government certainly doesn't make it easy to find out what is going to be relevant to your needs, so yeah, I can see legal fees to consult about possible issues adding up rather rapidly.

    2. Al fazed Bronze badge
      Facepalm

      No.........

      Maybe they're Americans ?

      ALF

    3. Anonymous Coward
      Anonymous Coward

      The company in this case did the right thing - check with legal experts - and should be commended for it.

      I suspect too many companies from the smallest family firms to medium size enterprises aren't aware of their many and varied obligations under health and safety, employment, and data protection law and therefore don't check whether their activities are compliant.

      That all this legislation and regulation is readily available for directors and managers to read makes it all the more stupid that they don't. (I only started putting the company *registered* address in my email signature after I heard about the Companies (Trading Disclosures) Regulations 2008)

      It's more luck than judgement that stops bad things happening in these companies, and if bad things are happening that the "victim" is equally unaware and so we don't hear about.

  16. John Brown (no body) Silver badge

    Almost every email service currently availble?

    “…we were intercepting incoming messages, without the authority of the person that had sent it in the first place.”

    Surely every email service available which does incoming spam filtering is also legally on shaky ground based on that? Only the recipient has signed the T&Cs allowing that filtering and only the recipient has the power to change the settings. In the case of a child, the parent or other legal guardian is the "legal authority" and is allowed to take measure to protect their charge, which may involve filtering their email or other incoming messages, either on the device or at the server end, including forwarding those message automatically to another filtering service. Just like everyone already does now.

  17. msobkow Silver badge

    As far as I'm concerned, the pervs that are trying to groom children HAVE no rights because they don't even qualify as HUMAN in my books. :(

  18. Al fazed Bronze badge
    Facepalm

    Here we go again "were doing it to safeguard children"

    So what's wrong with using Pegasus if you are a parent who has a level of NO TRUST with their children.

    For fucks sake just take the plug off their PC or take away their mobile phone. It's the cheaper option, it'll work, save you loadsa money and won't break any UK Computer Misuse Whacks.

    ALF

    1. neilfs

      Re: Here we go again "were doing it to safeguard children"

      Until said child faces the following issues:

      Peer pressure

      Isolation

      Bullying

      Even placing screen time limits on a child’s device makes then different to their peers and can lead to the above. You then see unhappy, depressed children.

      The tail wags the dog here. The only want to prevent that so if all parents did the same. Therefore removing the difference.

      Then younger children with older siblings. They’ll push the boundaries at a far earlier stage after seeing what their oldest sibling is doing - is who could be ten years older.

      The ideal nice thought is to remove and restrict. The practicalities and social stigmas that can create cannot be ignored either.

  19. MatthewF

    TLS/SSL Inspection

    I wonder if there are considerations need to be made when using TLS inspection firewall devices? For example a user logging on to their webmail, the content would be inspected by the appliance for malware etc.

    While the user may have consented to this through their organisation's Infosec policies "No expectation of privacy using corp IT" - the sender may not.

    1. msobkow Silver badge

      Re: TLS/SSL Inspection

      Shhh. The law only applies to little startups; the "big boys" get to ignore it because they have TEAMS of lawyers...

  20. Ian Johnston Silver badge

    Google scans all of my email. Why are this lot different?

  21. G R Goslin

    Why

    Why is snailmail not being included in this farrago? Email, is only the electronic application of the ordinary mail system. Whatever can be sent by email, can just as easily be sent by snail mail. An SD card can hold an enormous amont of data, and attached to a piece of card in an envlelope, practically undetectable to sight and touch. It doesn't even need to have to have a source address.

    I only wish that something was done about spam. I'm tired of shovelling the stuff out of my inbox, on the PC, and have given up running an email mail client on my phone, since I've never found a spam filter that works on Android to any real effect.

  22. Anonymous Coward
    Anonymous Coward

    Bye bye spam filters.

    So when Mimecast or google or whatever, intercepts and scans my incoming emails (with out the senders consent) for spam and malicious payloads they are breaking the law? Or is there some exception? Somehow I doubt it.

  23. Jeff 11

    IANAL but I'd expect a test of this law would clarify the illegality of the interception and relay (e.g. for off-device third party analysis) - as opposed to rules-based filtering - of incoming messages on behalf of the recipient. The two are very different. If the latter were illegal then one could argue that every spam filter, SPI firewall, traffic analysis package et al would fall afoul of it.

  24. Ian Johnston Silver badge

    As far as I can see, RIPA only applies to public authorities. Presumably that's why spam filtering and GMail scanning are fine ... which would make parents using this technology to scan their children's communications fine too.

  25. Spanners Silver badge
    Devil

    Government Reforms

    When the government wants to make changes to existing legislation or standards set up by adults, you need to use quotation marks around the word "reforms" as they are trying to redefine what that word means.

    Once upon a time the word meant to improve, update or make better etc. What government "reforms" are intended to do is weaken or make less effective.

  26. RogerT

    Children do have some rights to privacy which their parents cannot take away.

    A Gillick competent child has the right to arrange medical appointments without their parent's knowledge.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2022