back to article Fancy some post-weekend reading? How's this for a potboiler: The source code for UK, Australia's coronavirus contact-tracing apps

The NHSX, a technology group within the UK government's National Health Service, has released the source code for its Android and iOS COVID-19 coronavirus contact-tracing apps in an effort to allay privacy concerns and improve the code. Developers who have examined the blueprints have not been entirely mollified, and have …

  1. john.jones.name
    WTF?

    Australian Gov legal team can not read it seems

    honestly I dont think Australian government legal team have a clue what they are doing...

    They released the source code and although they copied from the opentrace repo (Singapore gov funded) which is under GNU General Public License v3.0

    (Section 2 of GPL says that modified versions you distribute must be licensed to all third parties under the GPL.)

    The Australian gov dept tried to license it under a new license and claim copyright... which is not how this works...

    Incompetent legal dept would be a nice way of putting it...

    I also love that they "archived" the github versions... no issues can be filed...

    In a Australian gov public hearing the dept also refused to acknowledge that AWS could be (and very likely already is on regular basis) forced to hand over data based on United states courts (United States Foreign Intelligence Surveillance Court FISC, also called the FISA Court) the Australian Attorney General office cited that the advice was confidential (again rather petty and silly).

    The only way they can turn this around is to pivot to using the Apple/Google decentralised model and say the app was a placeholder, trying things out... wipe most of the current data in the store and start distributing the list of tested confirmed codes that people can check securely on the phone uploaded by the health Dept after they confirm things with the individual via a medical diagnostic test (which is open to anyone now).

    hell to really right their wrongs the best way would to host the data on secure Australian servers by an Australian company or Gov Dept.

    1. JassMan

      Re: Australian Gov legal team can not read it seems

      There is only one thing more stupid than plagiarism and that is telling everyone your software is based on that by another author, then removing the original license, not crediting the original author, then putting in your own license.

      Actually there is one thing more stupid and that is to set the archive flag so that when you get taken to court, you cannot erase the evidence of theft.

      I expect the coffers of the Singapore Gov, the FSF, and possibly GNU will be swelling nicely.

      The terms of GPL V3 are very simply stated with lots of info about the purpose and use of the license. The Ozzy lawyers are either shysters or the most illiterate people on the planet.

    2. Magani
      Big Brother

      Re: Australian Gov legal team can not read it seems

      ...the best way would to host the data on secure Australian servers by an Australian company or Gov Dept.

      You were doing fine up until the 'or Gov Dept' bit.

    3. Sanguma Bronze badge

      Re: Australian Gov legal team can not read it seems

      Well,in that case, just ignore the Aust-Govt legal detritus and use it as under the GPLv3. I'm sure the Australian Federal government will weep and wail and gnash its teeth, but - just take them to court over it.

    4. bazza Silver badge

      Re: Australian Gov legal team can not read it seems

      The Australian gov dept tried to license it under a new license and claim copyright... which is not how this works...

      Unless they’d been given permission to do so by the copyright holder.

      I’ve no idea if that’s what’s happened, but I dare say that as it originates in a Singapore gov funded effort, and the two governments get on pretty well, we can’t discount the possibility.

      1. john.jones.name
        Holmes

        Re: Australian Gov legal team can not read it seems

        True for a license, but not for copyright... Australian Gov claim Copyright.

        (they could transfer ownership and then license it back to Singapore... I do not see it... they messed up)

  2. Anonymous Coward
    Anonymous Coward

    The first thing we do, ...

    1) Is all the source code there? Is it possible to build the App from this source or is some code missing?

    2) Are there any dependencies on 3rd Party libraries doing nefarious things whose source is not provided?

    3) Is the App built from the provided source an exact binary match of what is being downloaded?

    4) Repeat all of the above when a new version arrives or tries to update itself in place.

    ... and that's just the Client side.

    1. ibmalone Silver badge

      Re: The first thing we do, ...

      0) What is the data use policy and consent being collected? Because the best case of the software audit is finding it upholds that, but you still have to ask what is going to be done with it.

      Right now many people seem to be of the view that they'd be willing to compromise their privacy temporarily to help stop the spread of coronavirus. That's probably a reasonable and responsible choice. It would be quite wrong to take advantage of that altruism by conning them into accepting permanent retention of their data, or quietly continuing to track those who forget to remove the app after the crisis is over, or collecting more data for other purposes than is needed for coronavirus.

    2. Doctor Syntax Silver badge

      Re: The first thing we do, ...

      "Is all the source code there? ... and that's just the Client side."

      That's the problem right there. Leaving aside the matching algorithm what is there that protects the data against access beyond that algorithm's requirements and what is there that provides for deletion on the data subject's request?

      We know the answer to that because we've already been given it; nothing at all.

  3. MonsieurTM

    And the BCS position?

    https://www.bcs.org/more/about-us/press-office/press-releases/new-bcs-study-supports-coronavirus-tracing-app-if-privacy-measures-convince-the-public/

    Basically "the only issue the BCS sees is that the public will have to be duped, brainwashed or forced to install it". I hang my head in shame, as a MBCS, has produced such a white-wash.

    1. Doctor Syntax Silver badge

      Re: And the BCS position?

      I got through most of the paper before going into skim-mode. In fact it doesn't quite seem to say what you think it says. That's just a summary of a large number of statements about how the system should work without pointing out that the govts. offering fails at a number of them. I wonder if it was written, at least in draft, before HMG confirmed what they were going to do. Alternatively it may have been craftily written so that at some future time they can point out that they were actually critical but without saying anything that will enable HMG to beat up on them right now.

      I do like their phrase in support of mass-testing: “you cannot ‘big data’ your way out of a ‘no data’ situation”.

  4. karlkarl Silver badge

    Its a phone app! Of course its meant to be badly written and low quality.

    1. NATTtrash Silver badge
      Meh

      "No worries people, we can apply technology to solve this difficult, seemingly impossible problem."

      Hmmm, where did I hear this before?

      1. Jellied Eel Silver badge

        Sage & onion

        Hmmm, where did I hear this before?

        Perhaps here?

        https://lockdownsceptics.org/code-review-of-fergusons-model/

        The code. It isn’t the code Ferguson ran to produce his famous Report 9. What’s been released on GitHub is a heavily modified derivative of it, after having been upgraded for over a month by a team from Microsoft and others. This revised codebase is split into multiple files for legibility and written in C++, whereas the original program was “a single 15,000 line file that had been worked on for a decade”

        Possibly the most expensive 15,000 lines of code in computing history. Doesn't include warranting 'as-is', and most certainly doesn't warrant consequential losses. It's stochastic behaviour possibly does allow Ferguson/Imperial to claim predictive skill as it produces different results every run.

        But it does share traits with other models, namely the predictions are easily falsified, ie Ferguson's initial mortality claims that helped trigger lockdowns were massively exagerated.

        But as typical with public projects, start with incorrect assumptions, then chase the pork. So if the disease isn't as bad as originally thought, then there's less justification to waste resources and add risks with contact tracing.. And the risks are many.

        1. keithpeter
          Windows

          Re: Sage & onion

          Not my field really, but John Carmack worked on the Imperial College model for a bit before the Microsoft people got involved. The quote below comes from a couple of parts of Carmack's twitter thread about the code...

          "Before the GitHub team started working on the code it was a single 15k line C file that had been worked on for a decade, and some of the functions looked like they were machine translated from Fortran. There are some tropes about academic code that have grains of truth, but it turned out that it fared a lot better going through the gauntlet of code analysis tools I hit it with than a lot of more modern code. There is something to be said for straightforward C code. Bugs were found and fixed, but generally in paths that weren't enabled or hit."

          It is also worth pointing out that the broad ballpark results of the model were similar in general outline to those produced by a team at the London School of Hygiene and Tropical Medicine. They have put a simplified model up as a Web page for educational purposes. You can twiddle around with that and get thoroughly depressed. You need to read the help page and the associated paper to understand how the variables and interventions interact. The point is that these programs are not systems engineering, they are research. We draw some comfort when different research teams using different methodologies, implemented using different tools give broadly similar results.

          Finally, it might be worth checking the last paragraph of the article quoted in the parent thread to this. The - no doubt very talented and effective - software engineer who looked at the code seems to have some interesting ideas about how to manage public health which probably lie quite a few standard deviations from the views of the majority in the UK.

          (I still think that Ferguson should release the original file)

          1. Jellied Eel Silver badge

            Re: Sage & onion

            Not my field really, but John Carmack worked on the Imperial College model for a bit

            We're Doomed! Doomed I tell you! :p

            It is also worth pointing out that the broad ballpark results of the model were similar in general outline to those produced by a team at the London School of Hygiene and Tropical Medicine.

            That could be confirmation bias, or 'consensus' science.. Which is also true of some climate science. Models contain some assumptions, which can be compared to reality via regression testing/reanalysis.. And if they still diverge, then something is wrong with the assumptions. So R0 or CO2 sensitivity.

            The point is that these programs are not systems engineering, they are research. We draw some comfort when different research teams using different methodologies, implemented using different tools give broadly similar results.

            But the problem comes when research meets politics, or economics. So this model, and climate models have both resulted in massive economic consequences. So given the implications, perhaps there *should* be systems engineering, ie expert software engineers taking the requirements from academics and producing the production models. For many academics, software development isn't their core expertise, and that often shows in the results.. Which is also a challenge for businesses, ie the number of 'systems' designed (ok kludged) in excel or VB.

            The - no doubt very talented and effective - software engineer who looked at the code seems to have some interesting ideas about how to manage public health which probably lie quite a few standard deviations from the views of the majority in the UK.

            Yup. There's also some interesting comments in the discussion. I think there's some merit to tapping the insurance industry for best practices in developing actuarial/population models, especially with 'expert' groups like SAGE. One issue with Ferguson and this model is if it's 15yrs old, it may have started out being developed for other outbreaks, in which Ferguson's estimates have been consistently wrong by orders of magnitude.. But by the law of contagion, many outbreaks share similarities, so an open source & crowd sourced approach could result in a better modelling ecosystem. So it wouldn't be wasted effort.

            #include disease.lib

            and have nicely commented sections where academics can parameterise or insert the assumptions they want to test. And having a standardised modelling architecture would also make peer review easier. That's something the WMO took on board after seeing the dire state of some climate modelling efforts.

            1. keithpeter
              Boffin

              Re: Sage & onion

              https://epcced.github.io/ramp/

              Put something together and run it past these people.

              You might find project 5.6 of interest

              (I have some interest in what comes out of project 5.2 based on a walk across town this morning as my daily exercise)

              1. Jellied Eel Silver badge

                Re: Sage & onion

                It all looks pretty interesting. Also thinking it's where the contact app could feed into 5.2 & 3. Analysing footfall is already a thing, but could be interesting to build data based on app contacts. Or sim-contacts and probablitly of infection. Seed a bunch of mobiles with a disease model, see how long it takes to spread.

                Also curious about medical data. Some might be simpler than others, ie test results. Beyond that has security & privacy implications, along with social. So should snatch squads grab potentially infected people and test or quarantine them? Saw a rather dystopian suggestion that California may be considering just that. Reanalysis will also be fun, ie ONS comments about limited data based on death certificates. Then diving deeper, accurately coding patient data to attribute outcomes, so teasing out deaths attributable directly to COVID that may/may not have had some co-morbidity with flu or other illness(es). It's similar to some.. questionable stats around smoking, where past smoking can end up with deaths being misleadingly 'smoking related'.

                As most medical types know, deaths fall into only 2* categories, heart failure or brain death, after all.

                *Ok, 3.. power failure..

          2. Doctor Syntax Silver badge

            Re: Sage & onion

            "Finally, it might be worth checking the last paragraph of the article quoted in the parent thread to this."

            Very interesting. The author seems to think this sort of thing should be left to insurance companies who have proper data scientists and academics should keep out. I wonder who trained those prper data scientists.

            1. keithpeter
              Windows

              Re: Sage & onion

              @Dr Syntax

              I'm a bit worried about the trend towards criticism of epidemologists and public health modellers generally based on micro-analysis of small samples of particular implementations of models. People are playing defence lawyer when they should be looking at the basics (one person infects between 2.4 and 3.6 other people on average when there is no immunity in a (sub)population, there is a 6 to 14 day period when someone is infectious but not showing symptoms, simple differential equation)

              This is applied (messy, political, entangled with desperate economic imperatives) science not systems level software engineering. The basis of science is that independent work by different teams using different methodologies tends (with eddy currents) to converge on something that we can regard as close to truth or reality in some sense. This is not an argument about engineering change orders or which library to use or what code style might be best.

              I suspect that there is a lot of 'flood the zone with shit' going on by various kinds of useful idiot.

              1. Jellied Eel Silver badge

                Re: Sage & onion

                I'm a bit worried about the trend towards criticism of epidemologists and public health modellers generally based on micro-analysis of small samples of particular implementations of models.

                Well, if there are enough models, then there'd be scope for WHO to.. model ensembles on the WMO's work. Which is another interesting modelling challenge. On the one hand, we're told the 'science is settled', on the other, 100+ models producing widely different results. But having slowly standardising an approach, it makes reanalysis simpler. But both are predictive models, which can be 'fact checked' against reality.

                People are playing defence lawyer when they should be looking at the basics (one person infects between 2.4 and 3.6 other people on average when there is no immunity in a (sub)population, there is a 6 to 14 day period when someone is infectious but not showing symptoms, simple differential equation)

                But that isn't what the Ferguson model claimed. The claim was 500k dead, and that's looking unlikely. But then there was also a previous prediction of 200m dead globally from bird, swine & BSE, where the reality was <1,000*. Or one of the odd situations where mortality was exceeded by '57 & '68 flu outbreaks. Or some other bad years, where the BBC & rest of the MSM didn't run screaming headlines and running death counts.

                So R0 is just one parameter that is/was uncertain.. Which model predictions translated into lockdowns to minimise impact if R0 is high... But if mortality rates are low, then lockdowns may have been the wrong approach, and 'herd immunity' more effective. In an actuarial sense, once the panic is over, it'll be possible to look the cost per life. The kind of grim calculations insurance types (and politicians) are expected to do.

                This is applied (messy, political, entangled with desperate economic imperatives) science not systems level software engineering. The basis of science is that independent work by different teams using different methodologies tends (with eddy currents) to converge on something that we can regard as close to truth or reality in some sense. This is not an argument about engineering change orders or which library to use or what code style might be best.

                It probably should be, given many scientists aren't software engineers. That's just a necessary evil.. I mean skill they'd need some familiarity if they want to analyse data.. Which they can frequently get wrong. Politics comes in when politicians decide to act on bad data, so where this is critical, perhaps more formal methodolgy should be used. I mean I was forced to learn Z at Uni, so some scientists should also be expected to suffer!

                Again there's also a risk from consensus science. That's been pretty common in climate science, especially if anyone dares to try and publish something that challenges the consensus. A crucial variable there is CO2 sensitivity, and if you assume that's high, models predict high warming rates. But frequently fail reanalysis, ie comparison against historical data. If CO2 sensitivity is low, so will warming rates... But that's a rather controversial view because that 'science' has become highly politicised.

                Same challenge exists with epidemeological models. Assume high R0 and mortality, it'll predict a lot of deaths. Reality may disagree, but that's where scientists need to try and educate politicians about confidence levels.. And ideally get the ONS to explain statistical confidence tricks that can get used to create false confidence. Especially given the money at stake for treatments and vaccines, like one of the previous trials where the sample size was very small, and the outcome had to be redefined to show any significance at all. Then there's stuff like 'relative risk'. Take 2,000 patients, give 1,000 a placebo, another 1,000 balm from Gilead. Placebo group has 2 deaths, balm group 1. Claim treatment offers a 50% improvement in outcomes rather than a 0.1% difference. But those kinds of stats sold an awful lot of statins..

                * Swine flu in India may have killed many more, depending on how that's defined.

                Also found another interesting article-

                https://architectsforsocialhousing.co.uk/2020/05/12/the-state-of-emergency-as-paradigm-of-government-coronavirus-legislation-implementation-and-enforcement/

        2. Anonymous Coward
          Anonymous Coward

          Re: Sage & onion

          So.... you trust that the authors credentials are legit?

          I'm making no comment on the code, but quoting daily mail and telegraph 'journalist" toby youngs conspiracy website does your argument no favours.

          The site compares the lockdown to government dictatorship, is promoting uv-lights and chloroquine, and is funded by corporate interests. (who only care about their profits, not health)

          ... and that's all on the front page!

          It's basically alt-right America without the guns.

          1. Jellied Eel Silver badge

            Re: Sage & onion

            So.... you trust that the authors credentials are legit?

            As an old cowboy once said, trust but verify..

            I'm making no comment on the code, but quoting daily mail and telegraph 'journalist" toby youngs conspiracy website does your argument no favours.

            Ah, shooting the messenger. I had no idea it was Toby Young's website, and no real idea who they are. But theres a bunch of words in the article and it'd been mentioned on a bunch of other websites. So I figured I'd read it. Then follow some of the links, see twitter comments, read comments discussing the article.

            You on the other hand aren't commenting on the code. One aspect of my argument doesn't really rely on the code anyway. Ferguson predicted 500k deaths, we've had around 10% of that. The model's code quality may explain why it's so far out.. But then that's been normal for Ferguson's epidemic predictions.

            The site compares the lockdown to government dictatorship, is promoting uv-lights and chloroquine, and is funded by corporate interests. (who only care about their profits, not health)

            Oh Noes! Not corporate interests! But in any crisis, they'll be flocking around cash piles. So UV's used to kill bugs. Not sure why you're afraid of that. Mass house arrest is something few dictatorships have managed in the past, yet we're mostly complying with it. Except perhaps those wooly Swedes, who're acting as a bit of a control group.

            As for chloroquine.. Yes, that's a conspiracy. Possibly linked to your 'corporate interests' because drug dealers will make more money selling new patent medicines than old, out of patent drugs. But for science!

            https://www.bbc.co.uk/news/health-52622804

            The trial, led by a team at Oxford University, will compare with the current best available care a number of low-risk treatments recommended by an expert panel advising the chief medical officer for England, including:

            a seven-day course of hydroxychloroquine, usually used for acute malaria or arthritis

            I guess Oxford Uni's been infiltrated by alt-right conspiracy theorists. Meanwhile, over on the left, the Democrat's favorite expert, Alyssa Milano called for Trump to be impeached for suggesting it might work.

            But such is politics.

            1. Anonymous Coward
              Anonymous Coward

              Re: Sage & onion

              It's telling how you assume that because I pointed out the problems with that website I must be a card carrying democrat.

              I have no time for alyssa milanos hypocracy on this, "me too", or any of her previous comments. I have even less time for the democrat party.

              But thanks for that - it explains why the rest of your reply was so skewed.

    2. Quando

      >> Its a phone app! Of course its meant to be badly written and low quality.

      As opposed to all those perfect websites, OS’s, embedded hardware platforms etc.?

  5. Doctor Syntax Silver badge

    The technologists have been asked to build a system in a certain way and it's likely there's no body in the loop saying, 'You don't really need this.'

    That seems particularly generous in this particular case. A less generous view would be that there's someone in the loop saying "We really want this."

    In the wider sense of the govt's response there really seems to be a lack of somebody in the loop asking "How do we do this?" or "Could you clarify that?". The logistics of just about everything have been abysmal and the alleged "rules" of lockdown only get defined when some over-zealous enforcement hits the media.

    In the circumstances it's right that somebody needs to make big top-down decisions but they also should ensure that there are people to deal with the bottom-up details of implementation and that the inevitable questions that raises get answered promptly and consistently.

  6. Chris G Silver badge

    Best option, cheapest option

    Leaving aside the potential for data misuse and privacy loss, possibly and probably beyond the current pandemic, I feel that some governments see an app as a cheap alternative to finding, organising and carrying out extensive testing.

    In Russia for example, everyone is encouraged at the first sign of symptoms or suspicion of them to get a test, that brings both early isolation and treatment helping to reduce the impact.

    So far the percentage of deaths compared to infected seems to be lower than the majority of other countries.

    Testing was shouted about a lot in the beginning but now I see more talking about apps than tests and little done done about finding a genuinely useful test.

    1. doublelayer Silver badge

      Re: Best option, cheapest option

      Warning. Potentially incorrect causal relationship detected.

      "In Russia for example, everyone is encouraged at the first sign of symptoms or suspicion of them to get a test, that brings both early isolation and treatment helping to reduce the impact." leading to "So far the percentage of deaths compared to infected seems to be lower than the majority of other countries."

      This could be caused by several things. The easiest one is that more people are getting tested, meaning the number of people who we know to be infected in Russia is closer to the total than the number we know to be infected elsewhere. If we acknowledge that there is a significant chunk of the population that is now or has contracted the disease but didn't get tested and either showed no symptoms or thought it was a standard cold, we could also have a higher denominator leading to similar death rates.

      There are other methods for arriving at similar statistics. It could be that our lockdowns are being more effective at blocking transmission, meaning fewer people get infected, but those who do are more at risk of dying from it because they are more often elderly people in close proximity. Or that Russia has a test that produces reliable results faster, meaning people are caught earlier in the progression of the disease. Or that Russia has a worse test that produces a lot of false positives, but they are willing to live with that because overactive isolation can't really hurt. Or the thing you said. All are possible. None have been proven.

    2. Charlie Clark Silver badge

      Re: Best option, cheapest option

      You must have been watching Russia Today or something as other reports from Russia suggest that people who ask for the test don't necessarily get it. But the testing regime of most countries (Iceland seems a notable example) seems to be fundamentally flawed: testing resources are limited so priority is given to those with symptoms. This means it's always behind the curve and still only gives you snapshot of a particular moment in time. Research indicates that a large number of those infected develop little or now symptoms so your sample in skewed. So, over time the only really useful testing will be antibody testing, which is currently not reliable enough. Nobody is admitting it but I think policy is moving towards the Swedish model of letting the virus spread through the less vulnerable parts of the population during the good weather so that the potential wave in the autumn (combined with the flu) will be less devastating.

      Fortunately, though pretty infectious, it looks like only around 60% immunity is required (as opposed to > 80% for measles) but this is unlikely to happen organically this year or the next, but in some key areas (such as hospitals) it could be possible, which should limit retransmission to some severely vulnerable groups.

    3. Pascal Monett Silver badge

      Re: I see more talking about apps than tests

      In the political sphere, perhaps, but in the medical sphere, all the doctors want tests. And they're all saying that contact tracing is a requirement, but it needs to be followed up with tests. And they say that the only way we'll deal properly with this pandemic is by testing.

      They have their mouths full of the word, in every way they can say it. For some reason though, the politicians are not hearing the message very well. Well, not many of them. Some countries got their shit together quite quickly, and they're on top of the problem. It's just Europe and the USA who are not putting their money where they should. One group because God knows why, and the USA because their government is a pile of shit right now.

  7. Anonymous Coward
    Anonymous Coward

    Meanwhile Apple and Google seem to be managing to herd almost everyone else into using their solution .... no doubt in a year's time they'll be telling everyone that as they "saved the world" the all this talk about taxes and monopolies is disprectful and must stop (and doubtless the phones will remind us to go out each week for the regular "clap for the Apple and Google coders" sessions - GPS position, bluetooth signature of other phones in range and volume of clapping recorded by phone recorded for future reference)

    1. Oh Matron! Silver badge

      Such a shame you can't downvote more than once... So much wrong with this... I feel sorry for you

  8. c1ue

    Schneier posted a thorough takedown of contact tracing apps

    Scheier on Security for details, but a summary would be:

    1) Apps can't tell if genuine contact was made or not (i.e. a wall between people)

    2) Apps can't work if a person doesn't have a functioning smartphone

    3) Even Singapore was only able to get 20% adoption rate

    In the former case - are you going to quarantine 2 weeks based on this shoddy confidence level?

    In the latter case - the app cannot provide any measure of security regarding exposure.

    In the middle case - if even Singapore can't get more than 1 in 5 residents to install - why would anywhere else in the world, expect better? Except of course China...

    So why bother at all?

    1. Yet Another Anonymous coward Silver badge

      Re: Schneier posted a thorough takedown of contact tracing apps

      >So why bother at all?

      It's quicker and cheaper then doing anything else and let's you pretend you are being all cyber and digital and data and stuff ending in x

      It's everything Dominic Cummings loves

    2. Anonymous Coward
      Anonymous Coward

      Re: Why bother at all?

      Better to be seen to be doing something, anything even if it ends up shite and naff all use in solving the problem of CV-19 spreading.

      No matter, it will make lots of headlines for the 'Red Tops' in the coming weeks. Sadly, I see this as a way of diverting attention away from the total disaster that is happening in our care homes. I'm just thankful that my 98year old Mother's Care Home locked down a week before the Government Lockdown. So far they have had just one case and zero fatalities.

      OTOH there are no textbooks telling Governments how to deal with a Pandemic. There will be many written after this one is over.

      1. Anonymous Coward
        Anonymous Coward

        Re: Why bother at all?

        Sadly there are lots of textbooks, some were written by our own government. Sadly the complacent numpties who are our elected politicians were too busy thinking about pianting Union Jacks on the cliffs at Dover to bother reading them. And even now we seem incapable of actually looking outside our borders, and reading the books other have written and are writing. Debate this morning about transmission risk in school children. Cue scientist saying 'it's too early, not enough data etc. etc' - and no sense that we might ask China, Italy, France or Spain what data they might have. Still their data is probably written in foreign, and we are special.

        1. Aging Hippy

          Re: Why bother at all?

          It's good to learn from your own mistakes

          ...... but it's better to learn from someone else's.

    3. John Brown (no body) Silver badge

      Re: Schneier posted a thorough takedown of contact tracing apps

      "So why bother at all?"

      You are forgetting "if it saves just one life"

      1. ChrisElvidge

        Re: Schneier posted a thorough takedown of contact tracing apps

        Govt: "We have a problem and something must be done"

        AppCoder: "This is something"

        Govt: "OK. Lets do it. Problem solved"

        It is becoming more and more obvious that (sorry Clemenceau) pandemics are far too important to be left to politicians.

  9. This post has been deleted by its author

  10. Crypto Monad

    Apple-Google API

    I can see Apple releasing an updated iOS with this API, and it being quickly picked up by the majority of iPhone users.

    But a Google API? For most Android phone owners, who depend on updates being released by the manufacturer, they'll never see this. Seriously. If the API were released today, and even 10% of Android phones had it installed in the next 3 months, I'd be astonished.

    It seems to me there's little point writing the Android tracing app to use this API, if almost no Android phones will have it.

    1. John Brown (no body) Silver badge

      Re: Apple-Google API

      You've clearly not bothered to read any of the articles about this. Whether it's of any use in another matter, but there's no reason the majority of Android device won't have access to it.

      1. Medieval Research Council

        Re: Apple-Google API

        You've clearly not bothered to read any of the articles about this. Whether it's of any use in another matter, but there's no reason the majority of Android device won't have access to it.

        At the moment there is, it requires Android 8 or newer. The figures I've found for Android version in actual use that have 8 or higher is around 40%. We have to wait for versions that will work on earlier Androids to get decent access figures. I have prepared an old phone to use with the app (data scrubbed, factory reset, new SIM, new email account) but it's version 5.1 -- too old at the moment.

        1. Old Tom

          Re: Apple-Google API

          The API requires Android 6.0 or higher, not 8.

          1. BenM 29

            Re: Apple-Google API

            "The API requires Android 6.0 or higher, not 8."

            but still depends on the manufacturer/carrier releasing an update... the end user actually installing it and the update not being embargoed by the Good old US of A if it is an update for a Huawei phone...

            All of which, together, is highly likely /s

    2. Anonymous Coward
      Anonymous Coward

      Re: Apple-Google API

      I think the android app, unless Google have done so very clever trickery, will be DoA, due to the cheaper manufactures' use of aggressive power saving techniques.... One hopes...

  11. Version 1.0 Silver badge

    Another problem for the solution?

    Contact tracing is an old technique for dealing with disease outbreaks - so now we are being told that technology can solve the problem? Maybe it can but we're going to have to test it first and show that it works before it's any use at all - right now it's just another idea with zero evidence that Bluetooth contact tracing in this form will medically function. I know that it sounds like a good idea and it's got a better chance than drinking bleach, but maybe we'll have a working app that actually provides the functionality that we need in a couple of years - or maybe not. No way to know for a long time.

    1. Charlie Clark Silver badge

      Re: Another problem for the solution?

      Contract tracing requires employing and training people to do the work. This costs money, so even if there are several new centres opened to the relevant media fanfares, you can be sure that they will be funded only as long as anyone is looking (3 to 6 months normally).

  12. Irongut Silver badge

    Having looked over the UK code a few days ago it has all the problems I expected when I heard the government plans. It asks for too many permissions (including all the location permissions), collects too much data (Google Analytics, Firebase and MS AppCenter are used to build a profile of the user!) and is full of simple mistakes any junior dev should spot (no minimum Android version specified).

    There is no way I will ever install this app.

    1. Immenseness
      Big Brother

      Location permission

      And of course in the later versions of Android, *allowing* location permission also *requires* you to turn on and keep turned on, GPS.

  13. IGotOut Silver badge

    Am I missing one huge thing.

    What is the point of this app?

    They say has no location, so that means the government can't target resources to where it may be required.

    If I've been pinged by someone who has said they have had it....then what? Am I supposed to self isolate for 2 weeks because I MAY get it, even if the odds are actually very low (you don't get it just walking past someone)? For me in the country this could mean patient X only pings 100 people in the 2 weeks before symptoms, but in the city, that could be thousands and thousands. Do all those isolate? And if so, do those 5000 alert another 100,000?

    Or is it simply just to gather numbers? If so why not just have an app that reports back "I have it".

    I just see no point at all in it.

    1. Andrew Findlay
      Boffin

      Re: Am I missing one huge thing.

      The UK app requests the first part of your postcode. That gives enough location data for NHS resource planning.

      As for who gets advised to isolate, the algorithm for that is not simply 'have you pinged an infected person'. It will undoubtedly evolve as more experience comes in, but the suggestion is that the initial version will include factors for proximity, time, number of contacts with potentially infected people and so on. It will also have a manual check on any particularly large cascades of notifications, to reduce the risk from unexpected events and from malicious actors. This seems to be the main justification for the UK app being partly centralised rather than completely peer-to-peer, and it does make some sense. This also allows for the possibility of it telling you that you had received a false alarm and can go back to life as normal.

      There are good articles on the NCSC website by Ian Levy:

      https://www.ncsc.gov.uk/blog-post/security-behind-nhs-contact-tracing-app

      and more tech detail here:

      https://www.ncsc.gov.uk/files/NHS-app-security-paper%20V0.1.pdf

      It is worth pointing out that your data only gets sent to the central site if you declare yourself to be infected. You may also show up in uploads from infected people that you have been near.

      1. Doctor Syntax Silver badge

        Re: Am I missing one huge thing.

        " This also allows for the possibility of it telling you that you had received a false alarm and can go back to life as normal."

        False positives are likely to be feature. Being told - maybe - that it was a false alarm might come as a relief at first. After a few false alarms, maybe discovered false by being told or more likely by the non-appearance of symptoms, leading to days of unnecessary isolation, will likely lead to the app being removed or at least disregarded.

        What's needed is the ability to follow up all these alerts with a test with a quick turn-round.

  14. mark l 2 Silver badge

    I am not an Android app developer but I am sceptical that it has to have location permissions granted to use bluetooth.

    Are there any app developers who are able to confirm that it is required? As if not then I will either not install the app or if i did I would disable location permission

    1. really_adf

      I am sceptical that it has to have location permissions granted to use bluetooth.

      https://developer.android.com/guide/topics/connectivity/bluetooth#Permissions

    2. bazza Silver badge

      Try out the Lego Duplo Bluetooth enabled train app. Requires locations permission on Android to be allowed to use Bluetooth. That app goes to significant lengths to explain why it needs the permission, almost apologetically so. And says a that it doesn’t actually use location data whatsoever.

      It’s just another way for Google to try and force Android users to share their every movement with Google. Google’s assumption likely being that Bluetooth is often used in cars, and they want car location data to keep Maps traffic data current.

  15. Alan Sharkey

    So, it relies on everyone having a modern phone. It relies on everyone installing the app. It relies on everyone carrying their phone with them at all times. It relies on some central service to do all the correlation. It relies on people behaving "sensibly" when they get pinged that they may have come near someone who might have the disease.

    Can anyone see any issues here?

    Alan

    1. This post has been deleted by its author

    2. sanmigueelbeer Silver badge

      Can anyone see any issues here

      In some countries, the contact-tracing app is, currently, on a volunteer basis. Manual contact-tracing relies very heavily on the unreliable memory of a person.

      Singapore has shared their contact-tracing app with Japan, South Korea and Taiwan. (I am not sure if China developed their own contact-tracing app or not.)

      While South Korea has made the app mandatory for everyone (including foreign visitors), Taiwan has also relied heavily on this app.

      East Asian countries (Singapore, South Korea, Taiwan, China and Japan) using contact-tracing app are seeing the results and are "flattening the curve". While the rest of the world are still scratching their collective heads and scampering for stable supplies of PPE and reliable testing kits, these East Asian countries are gearing up for the "second wave".

      Facebook, Google, Amazon, Netflix, etc. probably have more personal information about me than this contact tracing app but while Amazon, Facebook, Google, Netflix, et al, are doing this to make money, this contact-tracing app is here to help me save my life, my loved ones and a few others.

      NOTE: In mid-March, a close friend lost both her parents, ten hours apart, to COVID-19 in TWO (2) days after entering a NYC hospital. Until now, nobody knows who gave it to them or when they got infected.

      My previous comments about contact-tracing app vs privacy were viciously down-voted. And maybe I deserved it then. Before down-voting my comment (again), put yourself in the shoes of the people who have died and their family.

      Thank you for taking the time to read this comment and I apologize for unable to make it short.

      1. Anonymous Coward
        Anonymous Coward

        Singapore has shared their contact-tracing app with Japan, South Korea and Taiwan. (I am not sure if China developed their own contact-tracing app or not.)

        You’re not so familiar with the ways of a totalitarian regime such as China? Knowing who you are, where you are, who you’re meeting, what you’re saying and what you’re thinking comes as standard. And if you don’t patriotically cooperate with the authorities in this by carrying a phone on which location services can’t be turned off, you quickly get into a lot of trouble.

        All enabled and supported by tech given away by - Google.

      2. Charlie Clark Silver badge

        using contact-tracing app are seeing the results and are "flattening the curve"

        Except they keep seeing new outbreaks (new one today in South Korea), which are to be expected in any kind of epidemic. It's a fallacy to expect any technological solution to the problem.

        In mid-March, a close friend lost both her parents, ten hours apart, to COVID-19 in TWO (2) days after entering a NYC hospital.

        Every death is one too many, but we're still looking at something comparable to a severe flu season.

        1. Roland6 Silver badge

          >Every death is one too many, but we're still looking at something comparable to a severe flu season.

          That is only because of the measures taken to limit contact and reduce the rate of transmission. Without these measures, we could expect to see a death rate somewhere between x10~x44 higher. From memory, CoViD-19 could kill between 2~5M people in the UK between March and Christmas if we carried on as 'normal'...

          Remember, no one gets admitted to the high dependency wards with flu, only the few who develop pneumonia.

          1. Charlie Clark Silver badge

            That is only because of the measures taken to limit contact and reduce the rate of transmission.

            As we do with flu: annual vaccination campaigns and residual immunity in the population.

            Without these measures, we could expect to see a death rate somewhere between x10~x44 higher.

            Based on what particular empirical evidence? Certainly not the case in Sweden, which has some of the least restrictive measures. At the moment we have some assumptions and some correlations.

      3. CommanderGalaxian
        Mushroom

        "My previous comments about contact-tracing app vs privacy were viciously down-voted. And maybe I deserved it then. Before down-voting my comment (again), put yourself in the shoes of the people who have died and their family."

        Anyone using emotional blackmail deserves to be downvoted for that reason alone.

        (The fact that you even resort to emotional blackmail probably means your arguments are pretty shit too).

    3. werdsmith Silver badge

      So, it relies on everyone

      Not everyone, it needs about 60%.

      1. I am the liquor

        Does it become useless if you have less than that though?

        Say you get an uptake of 20% (meaning 20% of the population have the app installed and actually running properly on their phone when they're out and about). So you've got 20% of currently infectious people. Half of them might never develop symptoms or might not tell you, so maybe you actually find out about 10% of the infectious people. The app could potentially identify 20% of the people they've infected, so 2% of all new infections. With good testing, you might catch a lot of those 2% in time to stop them passing it on, and maybe you could reduce the overall infection rate by 1%. It's not a lot, but it's not nothing. Based on on-line models I've seen, even a 1% reduction in R0 can reduce total fatalities by hundreds.

        The above is very much back-of-an-envelope stuff. Hopefully some boffins somewhere are working out the real numbers.

        1. I am the liquor

          A downvote's not really much of an answer to my question, is it, so I looked into it and found an answer myself. Some boffins have indeed been working out the real numbers:

          https://045.medsci.ox.ac.uk/files/files/report-effective-app-configurations.pdf

          Looking at figure 6 in that report, their various scenarios have cumulative deaths over 140 days being reduced by very roughly 10% when app uptake is 20% of smartphone users.

          80% of smartphone users/56% of the population is what they reckon is required to suppress the outbreak if there's no lockdown and you're relying on the app alone. But it's clear there are benefits even if uptake is much lower.

          1. Charlie Clark Silver badge

            20% is statistically too small to give you any reliable prediction. It will certainly help tracing infection chains but you will be missing so much other data that you're still in the realms of anecdotal evidence when it comes to transmission in the general population.

            1. I am the liquor

              Isn't tracing infection chains the main purpose of a contact tracing app?

              1. Charlie Clark Silver badge

                Maybe a thought experiment will help. A small sample might help you trace contacts in a single classroom in a school but only there. But while you're doing this you will not see the gunmen raging through the rest of the school or that someone is handing out syringes and, as a result, any decisions you take will be based on very partial information.

                More draconian, but probably more useful is keeping records of people in places where they spend some time: restaurants, medical practices, etc.

                1. I am the liquor

                  If you're trying to use data from it to do population epidemiology, or make policy decisions, then sure, a small sample may not be very helpful.

                  But it's a contact tracing app, not a population epidemiology app. The point is to identify specific individuals who might be infected, so you can prevent them passing on the infection. Even if you identify only a small percentage of all infections, that still allows you to reduce R0 by that percentage. And even a small reduction in R0 can make a difference over the course of the outbreak.

      2. CommanderGalaxian
        Boffin

        "Not everyone, it needs about 60%."

        Last time 60% was mentioned was with the government's scientifically illiterate "herd immunity" strategy.

        Just saying.

        1. I am the liquor

          No doubt the number comes from the same basic arithmetic. The proportion of the population you'd need to not be transmission vectors, for whatever reason - either by immunity or because they'd be identified and quarantined via a contact tracing app.

          That probably does give a clue about some of the assumptions behind the 60% figure for the app (actually 56% as per my post above)... 56% is not the proportion of the population that would have to merely install the app; it's the proportion that would have to install it, have it running properly all the time, and perfectly comply with any instructions arising from it.

  16. The Central Scrutinizer

    Well surprise surprise regarding the Australian app. No customer support, the developers not listening to or responding to legitimate concerns from other developrs, etc. Typical tone deaf government response to legitimate concerns.

    And they wonder why the download count has stalled.

    Unfortunately there's no app for fixing stupidity.

    1. Boris the Cockroach Silver badge

      Quote

      Unfortunately there's no app for fixing stupidity.

      But there is a care home sorted for those afflicted by the stupidity virus... its called the civil service, because in my experience, the more you suffer from it, the further up the ladder you go...

  17. Rich 2 Silver badge

    Trust no one

    The problem I have with any of this is that on the face of it, it may be a good idea. But (and it’s a huge but) the tech industry has such a shockingly bad reputation when it comes to privacy and generally being decent and honest that even if someone could show me that the apps themselves are 100% secure, I STILL won’t subscribe to the idea. Because the app security is only half the problem. It’s what happens to the “secure” data afterwards. And the answer too that is nobody knows apart apart from the tech companies and spying governments that will have access to it. And neither of those parties are honest enough (at all!) to tell the truth.

    And anything that uses googlies analytics is, by definition, insecure and privacy-invading. So the app can’t even get past the first hurdle

  18. Sanguma Bronze badge

    "There are at least 60 such contact-tracing app projects underway"

    Any more info? Like, who where what how? And how many are taken "private" from the GPLed OpenTrace source tree?

  19. Anonymous Coward
    Anonymous Coward

    Another piece of software coming under more open scrutiny now is the model used by the bunch at Imperial College. The Telegraph has an article here https://www.telegraph.co.uk/news/2020/05/10/chilling-truth-decision-impose-lockdown-based-crude-mathematical/ It’s paywalled but from the bit you can read there’s suggestions that this is a complete dogs dinner of a piece of software.

    Now given that it’s effectively been the basis of life critical advice, you’d think that the software properly ought to be akin to safety critical with all the attendant design, review, verification, and validation one normally associates with such software. Or, the people with the model would be aware of the likely unreliability of their creation and not give advice based on its output.

    Nope.

    I fear that lockdowns and the displacement of people they’ve caused (eg from hospitals into care homes) will be found to have been ill advised in the circumstances of the virus having been far more widespread than modelled. This seems especially likely given that it now looks like the official index case was preceded by other cases weeks before end of Jan. Maybe “Protect the care homes” would have been a better bet.

    I also fear that there will be little official interest in finding out whether or not what we’ve actually done was optimal. Of course, being generally better prepared would have been of benefit (eg South Korea has got this right), but we mustn’t let that get in the way of finding out whether we made good use of available information.

  20. Mike 137 Silver badge

    Why?

    "... the apps, which are supposed to be pro-privacy, use Google Analytics and the Firebase Analytics framework, configured in a way to allow personalized web advertisements."

    Translation: "Where there's an opportunity, however slim, to monetise, we will. That's our definition of capitalism".

    However, it's possibly unlawful. Quite apart from any "cookie" use, we need a decision to set the precedent - are personalised adverts "electronic communications" within the definition used by the Privacy and Electronic Communications (EC Directive) Regulations? If so, prior consent would be required. But who actually cares? Practically nobody it seems. Business and government currently stomp all over personal privacy and the ICO mostly seems to turn a blind eye.

    1. Anonymous Coward
      Anonymous Coward

      Re: Why?

      Perhaps they’re planning on using Web Analytics to be able to target informational messages at people who the system has determined are blatantly breaking the rules? That’d be a good use of the technology at any rate.

      Pretty much certain that it won’t get used for selling stuff, that’d discredit it thus defeating the very purpose of the app. Still, there’s room for stupidity on their part.

      However, it's possibly unlawful

      I don’t know if you’ve noticed but we’re in the middle of a pandemic. In such situations Acts of Parliament have been passed really rather quickly. There’s a whole law governing the lockdown for instance.

      So, some smart arse “that’s illegal” activist is likely to be given two things; specifically, a healthy dose of short shrift and a new Act of Parliament saying it’s legal (if they haven’t got one already, ink still wet).

      And were use of such an app to become a legal obligation, yes that would be a dramatic though ultimately still a democratic step. This isn’t China where they weld people into their houses and throw away the angle grinder just on someone’s whim.

      1. I am the liquor

        Re: Why?

        While it's true that acts of parliament can be passed quickly, the law governing the lockdown is not an example of it. That's a statutory instrument created by the health secretary under powers granted by the Public Health (Control of Disease) Act 1984.

    2. hoola Bronze badge

      Re: Why?

      This is exactly what I was thinking, why the hell is this being used as a platform to target adverts?

      If this is for the greater good then nobody should be able to benefit from financially and that includes all the scumbag advertisers.

      The one bit I am genuinely puzzled about is the argument that the that Apple/Google offering is using this distributed data.

      For any app to be of use a central database has to be updated:

      A register of people that have been tested with dates

      Location

      Contact details for tracing

      It does not matter how else you approach this, you cannot leave the decision as to whether you have had the virus to the app users. That is not accurate enough. You cannot leave the tracing to the app users for privacy and security reasons. Whether Apple and Google are just acting as an intermediary is and not accessing the data is a bit of a red herring. The arguments that the UK approach means the Government can access the data falls down because the on both systems the NHS has to have all the data. If it does not have the data the app is useless distributed or centralised. The fact the Government can access the data is a separate issue of trust as the source is the NHS, not any intermediary.

      Maybe I have completely misunderstood something here but I just don't see that there is any advantage of one over the other. The fact that both are probably useless is of more concern and are using valuable resources that would be better allocated elsewhere.

      1. doublelayer Silver badge

        Re: Why?

        You are misunderstanding several concepts. We'll start with decentralized data storage. This works, and the government doesn't need access. It works like this:

        Your phone knows some random identifiers it's been screaming into the void. It also knows a bunch of identifiers it has heard from others' phones it has been near. So all you need to do to warn people you were near is to send a list of your identifiers to a public system. If they get that list, they can check the identifiers against their own list of the ones they heard, and if there's a match, they get an alert. Your name doesn't need to be attached when you send out the identifiers, and you certainly don't need their names.

        Ah, but I see you are concerned that people will report unreliably, causing a bunch of false positives. A reasonable concern. So the solution is to give all the information to the government and have them do all the reporting? No, it isn't. A better solution is to give health providers signing keys. If someone tests positive and reports identifiers, their report is signed with a health provider key. You can't report without a key, or at least a report may not be trusted without a key. Keys only go to health providers.

        You also are completely misunderstanding the utility of this app, if it actually has any. You seem to think that it's useful only by sending a bunch of data to the NHS. No, not really. They have information from tests, which they can use. The utility of this app is supposed to be that people know when they have to stay in quarantine. A warning from an app like this is not a positive test. Treating it as one would destroy any dataset. Nor is it a good reason to use limited testing capacity on that person. It would be great if we could get that many tests, but we don't have the capacity now.

        For that reason, the app idea is very limited and may possibly cause more harm than good. However, you seem to only acknowledge the extremes--either the app is worthless or the app provides crucial data to health authorities. It is instead intended to provide information of tentative reliability to the public. Given that, there is little or no benefit in centralized data storage. Simultaneously, there is significant risk in centralized data storage. If we are going to have such an app, it is very important that it be decentralized in order to get sufficient uptake.

        1. Roland6 Silver badge

          Re: Why?

          Yes, the decentralized approach is tempting, however, a big issue is what can we learn from it, to better prepare us for the next time, and there will almost certainly be a next time. Given the number of SARS/Corinavirus outbreaks in the last 20 years, I suspect we'll see another SARS/Coronavirus pandemic within the next 10 years.

          Not saying the UK NHS app is better or actually does the job, just that it might be wise to think of the bigger picture. From what I can see the UK NHS app provides more data to researchers.

  21. Robert Grant Silver badge

    In an analysis on Thursday, Reincubate, a UK-based developer tools software biz, said that the inclusion of the ACCESS_FINE_LOCATION in the Android app is necessary for using Bluetooth. The iOS version, the company says, does not request location permissions.

    This is the case, or at least was. We hit the same thing. Annoying.

  22. lisporbust

    Bluetooth test

    Equipment:

    1 cheap Bluetooth speaker (£20 from Amazon 3 years ago)

    1 budget smartphone (Wileyfox 4 years ago)

    Method:

    Switch speaker on and leave in kitchen.

    Take mobile phone to end of front garden, enable Bluetooth, pair with speaker, select music track, press Play. Place mobile on ground.

    Walk back to house, enter through front door, walk to kitchen in back of house, measuring distance.

    Results:

    Music playing loud and clear on speaker.

    Distance between devices > 10 metres as required for Class 2 Bluetooth (normal for mobile phones).

    Note:

    The devices had a window and thick Victorian walls between them.

    Conclusion:

    Bluetooth registered devices as being 'in proximity' at 5 times the current UK social distancing distance, covering a 25 times (5 squared) larger area than the social distancing standard.

    Example of application:

    Walk down one side of a high street and be recorded as being 'in proximity' to people walking down the other side of the street.

    Possibly the people might be browsing in different shops on the opposite sides of the street (not tested).

    Surprise, surprise:

    (1) politicians, (2) journalists, (3) government science and technology advisers seem to be unaware of the sort of school-age experiment that any of them should be capable of undertaking and which is quite important in this context.

    OK, not that much of a surprise. Many switched from bottom of science class to learning ancient Roman quotes at about age 15.

    As someone wrote about beloved BloJo: 'A man who was educated beyond his intellect'.

  23. This post has been deleted by its author

    1. Fonant

      Re: Privacy Concerns?

      See: Brexit campaigning, targetted adverts on Facebook, Cambridge Analytica, Trump, Cummings, Farage, Aaron Banks, Thiel, Mercer, Palantir, Wigmore, 55 Tufton Street.

      It's subtle but powerful. Find out lots about people and their lives, then carefully tell them what they individually want to hear, and they'll vote for you. Get into power, and you can do largely what you want.

      They're not necessarily targetting individuals, they're targetting the entire population of countries.

    2. EnviableOne Silver badge

      Re: Privacy Concerns?

      the issue is not the app itself, its the size of the data set that will be stored centrally. with a data set that size de-annoymisation is a simple task, and you can track and identify anyone and who they contact.

      that data set will become a huge target and the security on it will probably be sub par.

      on top of that the algorithms and data structures in the central system will be a minefield for privacy and probably violate DPA2018.

  24. sanmigueelbeer Silver badge
    Thumb Up

    How has Vietnam done so well to combat COVID-19?

    How has Vietnam done so well to combat COVID-19?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020