back to article How a Kaggle Grandmaster cheated in $25,000 AI contest with hidden code – and was fired from dream SV job

A Google-backed competition to develop machine-learning software to help abandoned animals find loving homes turned ugly – when it was revealed the winning team cheated. PetFinder.my – a non-profit that lists profiles of dogs, cats, and other small creatures for people to adopt – devised the contest to “save homeless animals …

  1. YetAnotherLocksmith

    Of course, if the AI program was actually intelligent, this is exactly what it would do! It would look at the answer!

  2. ridley

    "Harnessing the power of data"

    Not entirely convinced that the sponsor for this article will want to be associated with it.

    1. JDX Gold badge

      Re: "Harnessing the power of data"

      One corollary of this is the AI can detect which animals are never going to get adopted and they can euthanize them early to save resources...

      ...is an example of when AI can actually be scary, because their models presumably literally do predict which animals will be put down and it only takes a human to decide this is the logical outcome.

      1. Anonymous Coward
        Anonymous Coward

        Re: "Harnessing the power of data"

        Great point. Cold logic says that's probably the best way to help the most animals.

        It's also incredibly wrong.

        1. Loyal Commenter Silver badge

          Re: "Harnessing the power of data"

          Not to sound glib about it, but who's to say there isn't someone making those decisions in animal shelters already? They don't run on infinite amounts of money, after all. One would hope it doesn't happen too often, but animals must get put down for economical reasons, and part of the decision about which animals get that fate would presumably contain at least an element of whether they are likley to get adopted or not.

          Given that most repsonsible shelters are going to be spaying the cats and dogs that pass through their doors (you know, to combat the problem of irresponsible pet ownership which is the cause of their needing to exist in the fisrt place), the decision may well involve determining whether it is worth getting an animal neutered when they come into the shelter.

          Long and short: folks, unless you are planning on breeding your pets, get tose cats and dogs spayed. Cats especially; the songbird population will thank you by continuing to reward you with a dawn chorus.

          1. Loyal Commenter Silver badge

            Re: "Harnessing the power of data"

            ...if you want really scary, consider the sort of parallel decision-making process that goes on in hospitals, especially when the resources fall short of those required to treat all patients.

            1. John Miles

              Re: "Harnessing the power of data"

              There was a Star Trek: Voyager episode, Critical Care, where care was allocated on a "Treatment Coefficient" in a hospital that looks a lot like how AI could go wrong.

            2. fidodogbreath

              Re: "Harnessing the power of data"

              ..if you want really scary, consider the sort of parallel decision-making process that goes on in hospitals, especially when the resources fall short of those required to treat all patients.

              In the US, we use a much simpler decision matrix:

              Does patient have a shit-ton of money?

              YES: Lives; potentially bankrupted by the cure, but that's their problem.

              NO: Too bad, so sad.

            3. BebopWeBop

              Re: "Harnessing the power of data"

              Does anyone remember the first episode of Andy Hamilton’s radio series in which he played Satan? A decision on retaining life support that turned on the retention of the TV in the staff common room. A brilliant series for anyone who has not listened.

              1. Loyal Commenter Silver badge

                Re: "Harnessing the power of data"

                Old Harry's Game

            4. ShadowDragon8685

              Re: "Harnessing the power of data"

              The solution to that isn't death panels or a death algorithm. It's an algorithm that intelligently raises the tax rate or shuffles some dosh from one service to another.

              Remember: the AI crunches the numbers, WE set the priorities on how we act on that data.

              "Mrs. Marjorie Smythe-Snow is in and out of hospital twenty-four times a year. She's 101 years old and can, on her good days, tell you about every war and global crisis she's lived through. The algorithm says she's going to cost a lot more if she keeps on ticking. What do?"

              "Obviously we remand this drain on society to hospice care!"

              "... No. In fact, so no that you're fired, security, get this clown out of here. No, what we do is raise about... punching in some numbers here... One cent in tax for a month should cover it, or we can short the city of Dover's cleaning supply budget."

          2. keith_w

            Re: "Harnessing the power of data"

            When my son wanted a pet, and we decided he could get a rescue cat (didn't want to walk a dog) from the shelter, they provided a subsidy to have the animal neutered. That way, they don't pay for animals that do not get adopted.

          3. PM from Hell
            Flame

            Spaying and neutering dogs

            Your comment made me incandescent with rage. It comes from a 1970's understanding of dog behaviour. Fortunately there has been a lot of research into both the psychology and the physiology of dogs in the past 50 years. My own research into the subject has concentrated on males as I had a male pup. But some points are applicable to dogs of both sexes

            1/ Responsible owners of pedigree animals do not allow random mating's to occur

            owners of male dogs can tell when they scent a bitch in heat and will put them on a lead. Responsible owners of female dogs don't walk them in public parks etc when they are in heat

            2/ Early neutering of males used to be seen as a responsible option. This is no longer the case, dogs should not be neutered until they are fully grown, early neutering will increase the risk of joint issues, hip dysplasia arthritis dues to joint wear and tear etc. in larger breed dogs they are not mature until 2 years old. By this age most dogs will have finished training and so neutering does not bring any benefits

            3/ Neutering was believed to reduce the risk of cancer in dogs, this is now shown not to be true, obviously removing the reproductive organs will prevent that cancer but research now shows it will increase the risk of other cancers

            4/ Neutering of males was thought to provide a docile dog. In reality juvenile dogs often suffer periods of fear, its a development phase, neutering during that phase will 'fix' the dog in that state

            5/ neutering was thought to prevent 'humping' this is a juvenile behaviour which can be trained out. Dogs are either trained to stop hunting or carry on regardless of whether they are neutered or not. Female dogs also hump it is driven as much as dominant behaviour as sexual.

            6/ there is a small but real risk that neutering a dog may result in personality changes. I have read heartbreaking stories of males who have changed completely after castration. Anyone who has owned a dog knows they quickly become a valued friend and family member, you love your dog for who He/ she is, you know their personality and quirks, seeing them radically change is akin to losing them.

            There are male dogs who become distressed when they scent a bitch in heat and may not be trainable to behave appropriately but these are a small minority of males.

            1. chuBb.

              Re: Spaying and neutering dogs

              Yeah no, while some points have some merit, you assume responsible is the default state of pet ownership.

              It isnt, given the stray cat problem every where, or the dumb asses like my neighbour who get a kitten then let it go feral because it scratches carpet, and havnt bought cat food in 3 years as evidenced by our shared bin and recycling boxes, and ridiculous get rich quick puppy farms i feel that the responsible thing to do is to neuter by default, and only allow non neutered animals to legit licensed breeders, not that would ever happen...

              1. J. Cook Silver badge
                Boffin

                Re: Spaying and neutering dogs

                THIS.

                Here in the US, a lot of the major cities have a serious feral cat problem. One solution is to trap, neuter, and return (TNR) the cats to the area were they were captured. It stabilizes the population, reduces propegation of disease, and it works.

            2. Dr_N

              Re: Spaying and neutering dogs

              PM from Hell>> 1/ Responsible owners...

              Your data sample is waaaaay too small.

              Responsible/knowledable dog owners are a rare breed indeed.

              1. Persona Silver badge
                Happy

                Re: Spaying and neutering dogs

                "Responsible/knowledable dog owners are a rare breed indeed." …….

                perhaps the solution is to neuter the irresponsible ones ?

            3. batfink

              Re: Spaying and neutering dogs

              I'm sorry PM, but you can't be incandescent with rage. According to the new regulations, since 2012 you can only be fluorescent.

            4. GruntyMcPugh

              Re: Spaying and neutering dogs

              @PM from Hell: "1/ Responsible owners of pedigree animals do not allow random mating's to occur"

              There are no responsible pedigree owners, because the entire concept of pedigree is a hideous affectation. There are working dogs from good stock, and then there are designer abominations with hereditary conditions. The latter are 'pedigree'.

              Meanwhile, my first two rescue dogs came into being because the owner of the mother deferred neutering, and then said poor mutt was impregnated by the neighbours German Shepherd who destroyed a fence to get to her. Maybe if you keep your 'pedigree' dogs in some kind of prison you can remove all risk, but it's no life for the dogs.

              Now, I'm on my sixth rescue, with a current pack size of four, which is as many as my wife and I can look after. All are neutered, and none suffer from it. The current oldest just had her 14th birthday, and one of the first two pack members was 15+ when he died, which is pretty good innings for a 40Kg dog. See, I have anecdotes too.

            5. Anonymous Coward
              Anonymous Coward

              Re: Spaying and neutering dogs

              1. if you are "incandescent with rage" at anyone spaying and neutering, I know one animal that may need to be neutered to cope with intractable personality problems. Yes, I assume you are male, that is the human group most uncomfortable with the procedure

              2. you are harping on about "responsible pet owners". Would that there were more, but we cannot even get responsible voters, see the current US and British heads of state.

              3. there is "research" that proves the Earth is flat and that vaccines cause autism.

          4. Someone Else Silver badge

            @Loyal Commenter -- Re: "Harnessing the power of data"

            Long and short: folks, unless you are planning on breeding your pets, get tose cats and dogs spayed. Cats especially; the songbird population will thank you by continuing to reward you with a dawn chorus.

            Was ready to upvote you for the first sentence, then downvoted you for that second, provably false (sub-)urban legend second sentence.

            But thanks for playing...

            1. Loyal Commenter Silver badge

              Re: @Loyal Commenter -- "Harnessing the power of data"

              Sorry, are you claiming that cats don't predate birds?

              Yes, there is the argument that "birds die anyway", but that argument only holds true if the ecological niche filled by birds is full (in which case, the stongest outcompete the weak).

              Just because it's hard to demonstrate that there is a causal link between predation of birds by cats and decline in numbers (correlation doesn't equal causation), doesn't meant here isn't a link.

              If you went round randomly shooting people and used the argument "lots of people die anyway", I don't think it would hold up to scrutiny, any more than the "no evidence that cats cause a decline in bird numbers" argument.

            2. Roj Blake Silver badge

              Re: @Loyal Commenter -- "Harnessing the power of data"

              The RSPB did a study a while back that concluded that cats don't affect bird populations much as they tend to only capture prey that would most likely have died soon anyway.

              https://www.rspb.org.uk/birds-and-wildlife/advice/gardening-for-wildlife/animal-deterrents/cats-and-garden-birds/are-cats-causing-bird-declines/

              1. ThatOne Silver badge

                Re: @Loyal Commenter -- "Harnessing the power of data"

                If you were a feral cat, which dinner option would you choose: Ground-dwelling small animals you can actually compete with (mice, rats, squirrels, moles, etc.), or birds with lightning-quick reflexes, able to instantly fly away?

                Yes, yes, the young ones, trapped in their nests. But here too, why bother climbing all the way to a lone, shaky branch of a tree, instead of just going after the litter of mice right here on the ground? Opportunity is indeed the decision-maker, and individual situations might vary a lot, but the generalization of "cats feed on birds" is a silly "Tweety vs. Sylvester" grade stereotype...

      2. BuckeyeB
        Angel

        Re: "Harnessing the power of data"

        And yet, you have weirdos like my wife. When we went to pick up another dog, I found some really nice ones that I liked. She picked out the ugliest, meanest, unhealthiest looking cur that she could find there. It was apparently massively abused, nearly blind and limped a lot. It had cigarette burns on its skin and matted fur. My guess was that it was headed to the chamber that evening. But the little rat of a dog lasted 2 more years in our house. I wanted one that wasn't broken. She wanted one no one else did. Guess who won.

        The dog.

        The AI will never account for those types of decisions.

      3. W.S.Gosset Silver badge

        Re: "and it only takes a human to decide this is the logical outcome."

        There's a rather excellent book by a rather excellent author which precisely skewers this syndrome, via the literary device of an independent robot. "Amusingly" one chapter for which the book nearly got banned, is now the default behaviour in American hospitals. It's just simple logic, after all: if you create a system/structure a certain way, it will tend to end up... at its logical conclusion.

        Tik-Tok by John Sladek.

        Unreserved recommendation, as indeed for all of Sladek's works (Roderick!), for sheer intelligence if nothing else.

        1. W.S.Gosset Silver badge

          Re: "and it only takes a human to decide this is the logical outcome."

          (great Amazon review here, worth reading in full :

          Tik Tok is two ideas in one slim fast read:

          1 - Asimov's laws don't work (and never could).

          2 - People are idiots.

          Sladek launched his career in the SF New Wave of the sixties, and spent most of his time examining Robots and AI -- in fact in the mid-nineties he was a technical writer for a software firm.

          )

  3. fwthinks
    Pirate

    What worries me the most, is that the code was available to view and the cheat code was not quickly identified. I think this is a major problem with software development today, that code is so complex with multiple layers of dependencies, people simply pick up code blocks and just assume it is secure and functioning correctly. Even open source code is not guaranteed safe until thoroughly reviewed.

    1. Old Shoes

      Fairly typical for amazing AI

      Many of the amazing AI advances in recent years just have poorly paid workers in cheap locations pawing through your data.

      Expensify could magically tabulate your receipts. They were just flogging it out to fellow humans on Amazon Mechanical Turk.

      Spinvox was using the power of AI to turn your voicemail messages into texts that could be discretely read, using human workers. (19 years later another startup called "Vxt" is recreating the same idea.)

      https://www.theregister.co.uk/2009/07/29/spinvox_mechanical_turk/

      There's a tiny robot that drives around Berkeley doing deliveries. Kiwibot drivers are sitting in Colombia.

    2. Sgt_Oddball

      It's like almost all code...

      Making it up as the dev goes along.

    3. Olivier2553

      Unless I am very mistaken, the source code developed by competitors is not public (it could not be during the competition and neither is after). Many of our students use kaggle competition and datasets and try to reproduce the results as part of their curriculum, we would not allow that if the algorithm were already known.

      But is is possible that the one who provided the data for the competition have access to the programs at the end.

  4. Warm Braw

    I'm not sure which aspect of this report causes me the most despair.

    Is is that someone genuinely sat down and decided that the way to solve the problem of abandoned pets was to artificially enhance their social media profiles?

    Is it that whenever you make an ordered list of people, some people will be determined to be first, regardless of the nature of the ranking or the methods employed.

    Is it that those same people are being hired on safety-critical projects, like driverless cars, seemingly on the basis of their ranking on an easily-manipulated high score list?

    Is it that I'd never previously heard of Kaggle and am so clearly out of touch with current trends, yet can't really seem to care?

    Just take me behind the barn and shoot me.

    1. KittenHuffer Silver badge

      I would be happy to oblige your final request but pity has stayed my hand ........ pity that I ran out of bullets!

      1. Zippy´s Sausage Factory

        You clearly enjoy dishes cooked in Brylcreem, I would imagine.

        Now I must go as the rush is coming...

        1. KittenHuffer Silver badge

          I read the first edition during my yoof.

          I'm aware that they released a new edition after all the Peter Jackson films, but so far I haven't grabbed a copy to find out if they upgraded it in the last 40 years.

          1. Charlie van Becelaere

            Yes, they did "upgrade" it by adding footnotes explaining many of the cultural references which no longer resonate.

            I found it a bit depressing to realise that most of the "humour" in that book now required footnotes. (At any rate, most of the character names did.)

      2. Neil Barnes Silver badge

        Bonus point for the 'Bored of the Rings' reference.

        1. Anonymous Coward
          Anonymous Coward

          I've given my copy to Oxfam because I detest LotR and hope someone else will be immunised against it.

          ...sorry, what was the subject again?

    2. katrinab Silver badge
      Flame

      The idea that you can model a human brain using boolean logic?

      1. werdsmith Silver badge

        The idea that you can model a human brain using boolean logic?

        That's the hype being given out by the mainstream media. Researchers do not believe that is what it's about. It's inspired by brains though.

      2. Throatwarbler Mangrove Silver badge
        Headmaster

        Well, you either can or you can't . . .

        1. JetSetJim
          Coat

          It's all a bit fuzzy...

        2. katrinab Silver badge
          Megaphone

          You can't.

      3. Sgt_Oddball

        I'd always been told...

        It was abit of a grey area...

    3. Cuddles

      "Is is that someone genuinely sat down and decided that the way to solve the problem of abandoned pets was to artificially enhance their social media profiles?"

      Definitely this one for me. The rest is just a bog standard machine learning problem combined with the usual result of the current trend of gamifying everything and turning into a competition with a top spot. What gets me is not just the idea of lying about pets in order to get them adopted faster, but the idea that there is even a problem here that needs solving. Some animals are less desirable as pets than others. A mangy arthritic mule is not going to get adopted as quickly as a playful fluffy kitten. Short of just claiming the former is actually the latter and hoping no-one notices when they come to pick it up, there's nothing you can do about that. Humans cheating in a competition isn't something anyone should be surprised by. But I'm am very confused why anyone would think the fact that nice pets are preferred to shit ones is something that could be fixed by throwing machine learning at social media optimisation.

      1. werdsmith Silver badge

        There are people in the world who will choose to adopt a less attractive animal out of compassion.

        1. Phil O'Sophical Silver badge
          Coat

          There are people in the world who will choose to adopt a less attractive animal out of compassion.

          True, my wife chose me, for example.

          1. Imhotep

            As I told my unmarried daughter: There is nothing wrong for "settling" - your Mother did.

        2. Anonymous Coward
          Anonymous Coward

          "There are people in the world who will choose to adopt a less attractive animal out of compassion."

          Yup. One of my cats has severely damaged back legs and hips from an accident when he was a stray kitten. He's also an unpopular colour, since black cats are deemed difficult to photograph by the Instagram generation. Then there's my other cat, an emaciated, flea infested, pee soaked mess when he first appeared in my yard. Both are now healthy, happy and bonded to me.

          1. BebopWeBop
            Facepalm

            I have heard the same about black cats. I must say I so consider our black kit to be very attractive - but then I don’t post anything on Instagram....

          2. Intractable Potsherd

            @james_smith: one of the best cats I have had survived a serious RTA, and was taken in by the RSPCA. When I saw him he was on his own in an enclosure - all but blind, a couple of wonky legs, going around in circles, brain damaged and unable to use a litter tray. He'd been available for adoption for several weeks despite being a beautiful ginger and white with huge eyes (due to the blindness. I just had to have him, and was rewarded by sharing my life with the happiest, funniest, most loving person* I have ever met until he died eight years later.

            *Yes, I count some of the cats I've had as people.

          3. Anonymous Coward
            Anonymous Coward

            for my life friend it went the other way - at marriage we happily bonded but now the wife is an emancipated, flea infested, pee soaked mess

      2. Robert Carnegie Silver badge

        It's not clear that "lying about homeless pets" was considered. Presentation evidently is a factor, an appropriate photograph... vaccination was mentioned, so if that's important to prospective owners then it may be worth doing for animals in waiting. (They may or may not get a different injection instead if not adopted.)

        My local newspaper has a "homeless pet of the week" feature; often these are "rescue" animals and they variously need to be in / not in a family, kept indoors / outdoors, with or without other animals, things like that... but another option, perhaps, is to foster-home unowned pets temporarily and make them more sociable for rehoming.

      3. JDX Gold badge

        I miss the part where they said they would lie about the animals.

      4. Mark192

        ”the idea that there is even a problem here that needs solving.”

        Shelters are effectively in competition against other shelters to offload the animals before they get killed. Also, some potential pet owners may look but not adopt.

        Both these things mean improvements to the way a dog is presented are worthwhile for the shelter.

        I'm unsure where you got the lying bit from. The way information is presented can have a big impact.

    4. Moldskred

      If we just learn what makes someone good at music chairs then we can train everybody to be that good and then nobody will have to go without a chair ever again!

      1. ibmalone

        You are assuming a fixed number of places, there's also the possibility people may decide not to adopt a pet (and they might have multiple pets) or to buy one instead (which pushes up demand on breeders).

    5. ibmalone

      Is it that those same people are being hired on safety-critical projects, like driverless cars, seemingly on the basis of their ranking on an easily-manipulated high score list?

      Well, the high-score list shouldn't be that easily manipulated, the idea is that a test set is held back and your algorithm is tried on it blind once at the end of the contest. The mistake here is getting a bit lazy or too trusting and not properly keeping it under wraps. A bit like leaking the test answers on the internet.

      1. Olivier2553

        I fully agree. What is the use of a test if the answers are already public. And not by a small amount as 3500 out of 4000 test data were public.

        This is a completely missed planned competition.

    6. Imhotep

      What bothers me is that this is being described as AI.

      Has the bar fallen that low?

      1. katrinab Silver badge
        Flame

        Yes.

        Anything with if statements is AI now.

        1. chuBb.

          That and written in python or some other scripting language of the week

          Thankfully this isnt 20 years ago otherwise vbscript would be at the forefront of AI (Automated If's?)

        2. Anonymous Coward
          Boffin

          I think that happened when "Web programmers" started to emerge who couldn't handle selection or looping and think that sort of stuff is rather hard and needs advanced intelligence.

        3. Someone Else Silver badge
          Thumb Up

          @ katrinab --

          Anything with if statements is AI now.

          Post.

          Of.

          The.

          Week!

      2. Olivier2553

        It's machine learning, so I guess that technically it falls in the AI basket.

        But I have hard time with some students explaining them that no, linear regression is not AI.

        1. ibmalone

          You sound like a person who would enjoy this XKCD

      3. Anonymous Coward
        Anonymous Coward

        "The goal was to create an algorithm that could predict how quickly a pet would be adopted based on its profile details, from its photo to its breed, sex, size, age, and whether it had been vaccinated or not."

        Yeah, didn't we used to call that Statistics? Didn't need anything "artificial" to it, it was just confidence-based Mathematics. Computers were just there to do the calculations more quickly.

        1. ThatOne Silver badge
          Devil

          > didn't we used to call that Statistics

          Yes, but with intelligence getting rare and precious, it is now called "artificial intelligence". And in a couple decades, when intelligence has fully disappeared, it will be called "magic"...

    7. Anonymous Coward
      Anonymous Coward

      > Is is that someone genuinely sat down and decided that the way to solve the problem of abandoned pets was to artificially enhance their social media profiles?

      Have you seen this recent story about a 12-year old in the US who hand makes bow ties for dogs in animal shelters because it helps them get adopted more quickly?

  5. Aristotles slow and dimwitted horse

    One of life's losers...

    I guess that "a constant struggle of becoming #1 in rating" is a goal or priority for some people, but to quote John Candy in Cool Runnings... "a gold medal is a wonderful thing... but if you’re not enough without it, you’ll never be enough with it.”

    1. Sir Runcible Spoon

      Re: One of life's losers...

      Upvote for the film reference, definitely on my re-watch list.

    2. T-Unit

      Re: One of life's losers...

      That's a very good point. I have some sympathy for Pleskov, I've been in environments where 'winning' was fetishised to the point that cheating, lying, and sabotaging others was practically encouraged.

      That said, this was no such an environment so that feeling was down to him alone. It does sound like he's learned that lesson though, the hard way.

      1. Someone Else Silver badge
        Alert

        @t-Unit -- Re: One of life's losers...

        That's a very good point. I have some sympathy for Pleskov, I've been in environments where 'winning' was fetishised to the point that cheating, lying, and sabotaging others was practically encouraged.

        cf. Donald Trump

      2. IGotOut Silver badge

        Re: One of life's losers...

        "I've been in environments where 'winning' was fetishised to the point that cheating, lying, and sabotaging others was practically encouraged."

        This is a well proven human trait (I'm sure there is likely some name for it). If you reward the individual e.g. with bonuses, then many will go that way. If you reward the whole group involved, then they work together.

        This is way many sales people are utter wankers. Companies reward the individual, whereas if you have say a profit share scheme, you get fewer arrogant self righteous pricks.

    3. c1ue

      Re: One of life's losers...

      I understand what you're saying, but in this case - the gold medal is accompanied by gold.

      Not so much the Kaggle gold, but the extra work and pay that the #1 position yields.

      So it isn't necessarily an ego thing as Pleskov says as it more likely is a financial thing:

      Kaggle Grandmaster, hire me!

  6. Luiz Abdala
    Holmes

    Do not use public information to test AI prediction.

    The dataset used was biased. They used previously public information from a related source. If they had used undisclosed information, it would be clear the winners' code would do poorly. This guy was actually intelligent to predict WHERE the dataset would originate, and use that to cheat his AI.

    Volkswagen used the same principle on their cheating diesels: they knew the test conditions beforehand, and how to detect them.

    What I think is: people testing AI must learn to use undisclosed sets of data, and not to reveal their test methods.

    Like, using Nürburgring to test how fast a car is. Any vehicle can carry a hidden GPS, and unlock a self-destruct mode that allows the car to go faster than anyone else's at the cost of durability, just to win the fastest lap in a vehicle that couldn't possibly hold that performance for long, and still blame "misuse" to avoid warranty issues if one of said vehicles ever broke down.

    In fact, I heard of some manufacturers having speed limiters that would automatically unlock if the cars were near the Circuit of Suzuka, or any other known circuit of a list, as a positive feature, so it is totally doable.

    1. KittenHuffer Silver badge

      Re: Do not use public information to test AI prediction.

      In which case a GPS spoofer is a must!

      https://www.theregister.co.uk/2018/07/16/researchers_hack_gps/

    2. quxinot

      Re: Do not use public information to test AI prediction.

      There's a number of cars that use that technology. I think it first came to my attention in connection with the Nissan GTR.

      The old version of a car knowing a test condition was dyno testing for emissions way back in the bad old days--if the car had a small light under the hood (bonnet) ostensibly to help you see to check the oil or something, it could just as easily be tied into the undoubtedly crude fuel injection system to make the car run cleaner, for example. Cars were emission tested on a dyno to simulate load, and invariably had the hood opened for connecting to the ignition system to measure RPM.

      I think of that line of thought, I need a car that can measure when radar or laser signals are picked up in the vicinity, and slow me down enough to keep me from getting written proof of my car's performance.....

    3. Kernel

      Re: Do not use public information to test AI prediction.

      "Volkswagen used the same principle on their cheating diesels:"

      Why don't we stop with referring this issue back to VW all the time - Opel, Chevrolet/GMC/Buick, Daimler, Fiat Chrysler, Mitsubishi, PSA (Citroen and Peugot) and Renault have all been found to be doing something similar.

      I've little doubt those manufacturers not on the above list are just yet to be caught cheating, as opposed to not actually doing it.

      1. batfink

        Re: Do not use public information to test AI prediction.

        Using VW is just a shorthand. We all know that most, if not all, of the others are doing the same.

        I knew someone at Ford who said that they knew VW were cheating, as they all buy and test each others' new models when they come out. However they could never quite figure out HOW they were cheating.

    4. KBeee

      Re: Do not use public information to test AI prediction.

      In Japan all domestic cars are limited to 180kph. Until 2004 they were also limited to a maximum 280ps (275hp). Except for a couple of exceptions, the speed limit on Japanese motorways is 100kph (62mph), and a maximum of 60kph on other roads unless a lower speed is posted. All Nissan did was to remove the speed limiter if the car was deemed by GPS to be at a place where it is allowed to go over the normal legal speed limit. So not so much cheating to get extra speed, more like just allowing the car to perform as it was intended.

      1. Anonymous Coward
        Anonymous Coward

        Re: Do not use public information to test AI prediction.

        > All Nissan did was to remove the speed limiter if the car was deemed by GPS to be at a place where it is allowed to go over the normal legal speed limit.

        That's how it was originally marketed, but the feature was never implemented. In reality, Japanese owners who wanted to race, had to take their car to a dealer "to be checked" before and after the race. The dealers manually enabled or disabled the limiter.

  7. TeeCee Gold badge
    Devil

    Kaggle

    ...used by more than a million netizens...

    How many more are required to hit the threshold where Google decide to shut it down?

    1. Comments are attributed to your handle
      Coffee/keyboard

      Re: Kaggle

      It needs to become a household name first, then it's ripe for termination.

  8. Steve 149

    Amazing what people will do. Nice detective work.

    It is sad that pets are only adopted by some because of their 'profile'. Apparently black cats aren't as adoptable because they're not as photogenic. One of our rescue cats has only one eye but doesn't miss a trick. Hence I call him a pirate.

    If it hadn't been for those pesky kids.....

    1. Anonymous Coward
      Anonymous Coward

      Just mentioned the black cat / photogenic thing in a comment above. On the plus side the cat hair doesn't show up as easily if you only dress in black like I do.

    2. The_H

      "One of our rescue cats has only one eye but doesn't miss a trick. Hence I call him a pirate."

      Surely you call him Nelson?

      (bonus beer if you get the 80's sci-fi series reference!)

      1. Anonymous Coward
        Joke

        > Surely you call him Nelson?

        As in Nelson Riddle because it's a mystery how he lost his eye?

  9. trevorde Silver badge

    Auto-cheating

    https://techcrunch.com/2018/12/31/this-clever-ai-hid-data-from-its-creators-to-cheat-at-its-appointed-task/

  10. Anonymous Coward
    Anonymous Coward

    Be economic with the truth algorithm :-/

    "These predictions would be used to optimize and tweak future critters' profiles so that they are adopted as soon as possible."

    We need less of this tech and more honesty in the world, and facing facts such as not all pets can be successfully re-homed.

    1. batfink

      Re: Be economic with the truth algorithm :-/

      Excellent. Your new dog may take bites out of you on a regular basis, but his profile will look great!

  11. Charlie Clark Silver badge
    Stop

    Move along, nothing to see here

    Kaggle is a bit of fun, but it should be obvious to anyone with any experience of online competitions that anything where something can be won will attract cheats.

    Companies see Kaggle at its ilk as a cheap way of getting solutions to problems and that is their problem. And any company that values ratings in such competitions is also asking to be screwed.

  12. Anonymous Coward
    Anonymous Coward

    Did anybody else read "Kaggle" and think "Pelvic floor exercise" ?

  13. anothercynic Silver badge

    And then...

    ... All it takes is one person to be curious to find the cheat.

    Reminds me of something I read a decade ago where someone in motorsport won a big truck in a competition for novices. Except he wasn't a novice in his country, just a novice in the country where the competition was held. It all came to light after a member of the media saw the photo and said "that's not so-and-so". The whole thing blew up massively and the truck had to be returned. It is just amazing what people do to win.

    1. Crypto Monad Silver badge

      Re: And then...

      > ... All it takes is one person to be curious to find the cheat.

      Not necessarily.

      ML models are basically black boxes. He could also have cheated by training the model on the additional pictures in the test dataset: this would not have shown any trace in the code.

      1. Olivier2553

        Re: And then...

        He could also have cheated by training the model on the additional pictures in the test dataset: this would not have shown any trace in the code.

        Only he was not supposed to know the adoption status of the pets in the test set. So he could not use it for training.

  14. keith_w

    code review

    What is amazing to me is that there was no code review done prior to the attempt to ready the system for deployment. How could you give someone a reasonably large amount of money, and bragging rights without at least cursorily checking the code?

    1. katrinab Silver badge
      Unhappy

      Re: code review

      It happens all the time.

      IT projects frequently don’t work. That may be a reason for it.

  15. Jamie Jones Silver badge
    Coat

    I was going to have a right old whinge...

    I was going to have a right old whinge about the usual misuse of country TLD's:

    "PetFinder.my - Oh so the 'My petfinder' company is Malaysian, then? *said sarcastically, with a smug obnoxious look*

    Only to later read "Malaysia-based PetFinder.my,"

    Bugger. A chance to be an angry old moaning git, and you took it away..... I must moan about that...

  16. Sorry that handle is already taken. Silver badge

    "If it's too long, changes can be made to make the pet more attractive, if possible."

    A new coat of paint perhaps?

    1. BebopWeBop
      Happy

      Re: "If it's too long, changes can be made to make the pet more attractive, if possible."

      As our pair explained after they had got into a freshly painted room through a open window (left that way to clear the atmosphere)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like