The Register Home Page

back to article Consultant mistakenly deleted a ton of data – but reported it as a bug

WHO, ME? Is showing up for work every Monday a mistake? While you ponder that question, dive into a new installment of Who, Me? – The Register's weekly column that shares readers' stories of escaping their errors. This week, meet a reader we'll Regomize as "Evan," who wrote to us from the side of a pool while his kids had a …

  1. Korev Silver badge
    Coat

    So it turned out OK Evantually

    1. CA Dave
      Facepalm

      getout.gif

  2. Michael H.F. Wilkinson Silver badge

    Neat escape

    One might call Evan's approach a plan so cunning if you gave it a tail you would call it a weasel. Reporting the data loss as a bug is of course quite plausible, given the complexity of the kind of software stacks found in the wild these days, and especially with SaaS in the mix (which allowed Evan's client to pass the blame to the provider of said SaaS, most likely).

    definitely a case of all's well that ends well.

  3. Anonymous Coward
    Anonymous Coward

    Agent P

    Definitely not my fault. I had Copilot and Grok cross-checking each other's work verifying Claude as it deleted the files.

    1. UCAP Silver badge

      Re: Agent P

      Ah, with that amount of AI doing the work, absolutely nothing will be able to go wrong.

      Hello, didn't I have a file of critical information in this folder?

    2. Roland6 Silver badge

      Re: Agent P

      The problem isn’t so much as to whether the files have or haven’t been deleted, but that the files being listed for deletion shouldn’t have been on the list. Can’t see how verifying that Claude actually has deleted the files it claims it has deleted is going to help here.

  4. Bebu sa Ware Silver badge
    Devil

    No good deed…

    goes unpunished, in this world — only rewarded in the next.

    No ill deed rewarded in this world — in the next……

    Ah, a consultant, you were already booked in.

    1. Mast1

      Re: No good deed…

      Maybe he thought he was Evan Almighty......

  5. khawar

    Guess it won't be for long. Their system admins were not good enough to figure out the possibilities and their likely hood.

    If this incident actually happened, then they shall figure it out soon and get Evan into trouble.

    So much for bragging about it.

  6. David Harper 1

    Not a lot of sympathy for "Evan"

    This is why consultants have such a poor reputation among IT professionals. They are overpaid, and they leave it to the rest of us poor schmucks to clean up their sh!t.

    1. Pascal Monett Silver badge
      Stop

      Re: Not a lot of sympathy for "Evan"

      If you had the proper tools and competence to do your job, he wouldn't have gotten away with it.

      Don't blame consultants for the fact that your IT is a house of cards build on a sand beach by the sea.

      And that is all because Manglement wants those pretty Excel charts as weekly reports.

      1. KittenHuffer Silver badge

        Re: Not a lot of sympathy for "Evan"

        Are you saying that the issues are caused by all the C shells lying around?!?

        1. The Oncoming Scorn Silver badge

          Re: Not a lot of sympathy for "Evan"

          "Are you saying that the issues are caused by all the C shells lying around?!?"

          "You have that vacant look in your eyes that says, 'Hold my head to your ear, you will hear the sea!'"

          Londo had some wonderful put downs.

      2. steviebuk Silver badge

        Re: Not a lot of sympathy for "Evan"

        Not really. We dislike consultants like the one that visited us once. Asked each IT member what they thought of the systems etc. Then palmed all those results off as his own work and discovery. They believed the consultant and ignore the local engineers. You can argue that's not the consultants fault but the consultant, having already been paid could have said "This is what "we've" found. You should have just asked your engineers as they provided me with all these details".

        1. that one in the corner Silver badge

          Re: Not a lot of sympathy for "Evan"

          Hmm, took all your inputs, mixed them all together and passed off the results as his own work, without providing any attribution.

          You sure that wasn't an AI walking around in a skin suit?

        2. Caver_Dave Silver badge

          Re: Not a lot of sympathy for "Evan"

          There is the old truism - "Consultants never tell you anything you didn't already know!"

        3. Doctor Syntax Silver badge

          Re: Not a lot of sympathy for "Evan"

          The consultant told them what they could have found out by asking their own staff, true. But the staff are paid far less than the consultant so what they'd have told them would have been seen as less valuable. If you have senior manglement who only evaluate sources of information based on what they pay for them this is the only way they get the information they need. Don't blame the consultant, he's only doing his job. Blame the manglement who aren't doing theirs.

          1. find users who cut cat tail

            Re: Not a lot of sympathy for "Evan"

            Funny that paying the staff more is never considered as a solution.

          2. steviebuk Silver badge

            Re: Not a lot of sympathy for "Evan"

            No, I still blame the consultant. Nothing is stopping the consultant saying "This is what I've found blah, blah, blah".... oh that's great, amazing. The consultant should add "But I got all this info from your own staff, you could have just asked them."

        4. Roland6 Silver badge

          Re: Not a lot of sympathy for "Evan"

          I think the problem is your expectation…

          Perhaps you need to better engage with the consultants to get more of your ideas into the consultants report. A good consultant should be able to wrangle it to get you the budget and follow on work for themselves, permitting the relationship to be deepened…

          1. steviebuk Silver badge

            Re: Not a lot of sympathy for "Evan"

            The point is, the consultant took ALL our work and made out it was his own work and own findings. That was the issue. Maybe I've just only met dick consultants.

            1. Roland6 Silver badge

              Re: Not a lot of sympathy for "Evan"

              >Maybe I've just only met dick consultants.

              Quite possibly.

              Something I learnt very early on from a highly successful saleman, was ensuring those client staff you worked with got recognised and rewarded was good business, as not only did it help future consltancy work but also successful delivery of projects the client decided you should deliver, as staff saw you as being on their side, albeit at times the resented what you were doing and the elevated level of reward you got. Yes we probably didn't get such large slices of projects (managing a client team doesn't generate as much revenue as running your own team, however, it can be more profitable and can permit you to take on more projects).

        5. djack

          Re: Not a lot of sympathy for "Evan"

          When I do consultancy, I see this as a failure of management. It annoys me no end when management will take note of something their staff have said time and time again only when I say it in a report. The staff know their own system far better than I ever could in my brief time looking at it.

          I much prefer giving new insights into situations or being able to identify information that the existing teams were unaware of, but I do often explicitly ask if there is anything slightly outside of my scope that I could 'notice' and mention in the hope of getting the staff the changes they need to do their job properly.

    2. Andy Taylor
      Coat

      Consultants = Rhinos

      Our nickname for consultants was Rhinos:

      Short sighted

      Thick skinned

      Charge a lot

      1. The Oncoming Scorn Silver badge
        IT Angle

        Re: Consultants = Rhinos

        My father was charged at by a rhino* in a Land Rover, while he was in Kenya during the Mau Mau uprising.

        * My father was in the vehicle not the rhino**.

        ** It might have been a hippo

      2. Sp1z

        Re: Consultants = Rhinos

        You can get a more accurate name if you remove o n s l t a

        1. Adrian Harvey

          Re: Consultants = Rhinos

          Dogbert says consult is a portmanteau of con and insult.

  7. Another Anonymous South African Coward

    What are the chances of the client picking this up, and realized who the real culprit was after all?

    1. Bill Gray Silver badge

      Note to self : should I ever submit a story to this august news organ, modify a detail or two to make it sort of, but not quite, point back to me.

      (I would guess that most submitters do exactly that. Actually, were I the vulture in charge of this column, I'd suggest to the submitters that they make such alterations.)

  8. mihares

    Always type pwd before starting a test.

    Awww that reminds me of that time I tested a "data maintenance" script, freshly written, directly on the company NAS and not in the test directory I set up a couple of hours before.

    Luckily that test directory could be used as a backup before anyone noticed that I nuked a whole research project...

  9. H in The Hague

    Move instead of delete

    I've always found it much safer to not delete files which are no longer required, but to move them to a separate directory (e.g. Oldstuff). Once I've done that, I might take a look at Oldstuff and decide to actually nuke the files. But then it's v clear what I'm deleting and there is less chance of zapping stuff I still need.

    1. Anonymous Coward
      Anonymous Coward

      Re: Move instead of delete

      I agree. Which is why my hard drives are festooned with quite the ramshackle collection of old data which may or not be important any more ...

      1. PB90210 Silver badge

        Re: Move instead of delete

        Each hardware/os migration meant a new 'old-data' collection of folders, including the previous collection, and the ones before...

        I did eventually dedupe and remove a few layers of crud

    2. David Hicklin Silver badge

      Re: Move instead of delete

      So now you have folders called OldStuff, very OldStuff, really oldstuff etc

      1. Roland6 Silver badge

        Re: Move instead of delete

        So move it off line and call it an “archive”…

        I have several disks (USB HDD, thumb drives, CD’s) like this.

        Last year I rediscovered some parallel port connect HDD’s in the garage, which thus date back to the 1990s. Given I have not had need to access them in all these years, it’s probably okay to sledgehammer them and drop the “bent” drives in the secure disposal bag for shredding.

  10. ComputerSays_noAbsolutelyNo Silver badge
    Coat

    Making the customer believe your fault is theirs

    - putting the con into contractor

    1. MiguelC Silver badge
      Joke

      Add insult to injury

      I you add insulting the client to the mix, you get the whole con-sultant ethos!

  11. Prst. V.Jeltz Silver badge

    happened in the middle of a project, meaning Evan's action was profoundly unwelcome

    After a week of back-and-forth with support, Evan got good news. His client's support team was able to restore the data from a backup

    Oh good! so no time wasted then

  12. kacpr

    Joke on him - the company will use the videos to train an AI model to automate his job

  13. billdehaan Silver badge

    It's not just consultants, in house employees do it to

    In the 1980s, I contracted at a major defence contractor. It was originally a VAX based shop, but the system manager (known as the "system mangler" within the company) was so shockingly, jaw-droppingly, breathtakingly incompetent that engineering teams were buying PCs in self defence in order to get work done.

    This was in the days when 80286s were the most common machines. 80386s were just coming out, but quantities were limited. When developers would rather use DOS-based 80286 machines rather than a microVax running VMS to do graphics development, because the PC development was faster, something's wrong.

    The system mangler was legendary. Stories about him were numerous, and ubiquitous. Everyone had a tale about how they'd come to work one day to find that their home directory on the Vax had been erased by accident by him, and there weren't any backups, so all the work they'd done for the past week/month/half a year was gone, and you'd just have to redo it, sorry.

    Not everyone could get a PC, though. Management fought tooth and nail to stop them from them from "infesting" the company, at the SM's insistence. They were uncontrolled, and he wasn't able to back them up. That was the idea, of course.

    While management was demanding that PC developers upload work to the Vax so it could be backed up, the reality was that Vax developers were asking PC developers to download their source to a PC so it was safe from the SM.

    And then, one day, the company discovered Unix. Specifically Apollo workstations. It wasn't actually Unix, but it was close. It ran sh and csh shells. We had a few, and a few Unix capable devs working on it. They set up a generic account with a public download directory so anyone could remote in via kermit and download files to their PC.

    The SM was livid about this, and demanded administration rights. The team resisted, but the management ordered it, so he was given it. It took him less than two days to discover "rm -rf /", running as system root, to completely destroy the entire project history. They had to order master tapes from the vendor to reinstall the OS.

    We thought for sure that SM would finally be put on a leash, but sadly, no. Management issued a memo a week later lamenting the "fragility of the Apollo system", and mandated subsequent work be done on the "underutilized" Vax.

    I saw several admins like that during my career, although thankfully none were as bad as the OG system mangler. But if anyone believes consultants are the only ones who pull these stunts, they are absolutely not.

    1. Anonymous Coward
      Anonymous Coward

      Re: It's not just consultants, in house employees do it to

      Oh god ... i know such people .. the screw up, they screw up again, they screw up so blatantly even higher management should notice

      and the next thing you know they're doing a half hour presentation on how this is all IT's fault for reasons ....

      or some other departments ... or ... but never theirs

      And higher management telling us we need to be more supportive of him ...

      And obviously he (it's always a he ) works from home and never comes to the office ..

      1. billdehaan Silver badge

        Re: It's not just consultants, in house employees do it to

        There's a phrase for it, failing upwards.

        I've seen numerous examples of it over the years. The longer it goes on, the more invincible the person feels, and the more obnoxious they become.

        Some keep it up forever. Others trip up when a new manager comes in from a different division or company where such behaviour was unheard of, and demands a justification. When the usual deflections fail to impress the new manager, there's often a re-org, and the offending employee is either shuffled off to the side where he can't do any more damage, or put on a tight leash with realistic expectations which he or she inevitably fails to meet.

        Often, the offender will decide "on their own" to leave for greener pastures as soon as a code audit or the like is announced, because they want to get out while they can still get a recommendation.

        Coders sometimes do this, but to do real damage, you need to be an architect or an IT administrator. A bad coder can mess up his project, but an architect or IT admin can screw up the entire company.

        1. Roland6 Silver badge

          Re: It's not just consultants, in house employees do it to

          >” but to do real damage, you need to be an architect or an IT administrator.”

          There was one organisation and my project management who disliked my solution to the migration of the paper-based field engineering system to a fully online system accessible from the field (a sparsely populated area with valleys and mountains). So after having got the board to approve the budget for my solution, I was exited the project. Some 18 months later I bumped into the project manager - they had spent the last 18 months going through several architects going round in circles burning money and had finally accepted my solution was the only viable way forward. He at least had the good grace to admit he was wrong in dismissing my, less than elegant but highly practical approach to the problem, unfortunately it was too late to make any difference to the poor project feedback he had previously given me and which contributed to my exiting the company.

          Moral of the story: much of the damage is inflicted by others, after the architect has left the building…

          1. billdehaan Silver badge

            Re: It's not just consultants, in house employees do it to

            In the early 2000s, at the height of the outsourcing craze, it was common for executives to outsource huge swaths of the engineering departments to India, fire the local engineers, and save the company a ton of money on salaries. They would pocket a huge bonus, and the more of the company they gutted, the bigger the bonus. Then they'd leave for greener pastures at other companies.

            It usually took between 18 and 36 months before the company realized it had given away its' core competence. Sometimes it was to incompetents who didn't understand the business, but sometimes, it was worse, as all the work went to competent engineers who'd soak up all the knowledge, then cross the street and work for, or start up, a competitor using all the engineering knowledge that they'd been given, and then compete against their former employers. And because the original company had hollowed out its' own engineers, there was no way for them to compete.

            It became so common that many of companies instituted clawback clauses in their bonuses, making the executives legally pay back their bonuses if the their cost cutting damaged the company, even if they'd left to greener pastures. Oddly enough, there was a significant slowdown in outsourcing after that.

  14. goblinski Silver badge

    Getting out the popcorn for the next "Who, Him?!?", about some other Vulture reader who once lost their job after their SAAS script was blamed for nuking plenty of data out of nowhere.

  15. This post has been deleted by its author

  16. segfault188

    Evan therefore escaped blame and carried on consulting – and is clearly doing well enough to pay for multiple kids to have swimming lessons!

    So things are going swimmingly for them.

  17. Doctor Syntax Silver badge

    "I stepped through every line, I checked all values, and I could see everything was right."

    If this is, indeed the case, and there were no wildcards in thee filename then it may well have been a bug. Never overlook that possibility.

    1. Brewster's Angle Grinder Silver badge

      Yeah, I was at a loss to see how deleting one file could lead to such carnage.

      1. TSM

        Not familiar with the tech involved, but maybe API call doesn't do what you think it does? I've seen plenty of APIs like that in my time for sure.

  18. FeRDNYC Bronze badge

    Works with hardware, too

    Over 30 years ago, I was hand-soldering some modification to a piece of hardware in my computer (which was the fashion at the time) when my hand slipped and I completely lost my grip on the soldering iron. Fortunately, the momentarily-ballistic iron's flight path was arrested before it endangered anyone or set anything on fire. Much less fortunately, the thing that caught it (by the tip) was my recently-purchased, insanely expensive, had-to-save-for-a-year-to-afford-it SCSI controller card (something like $400-$500 in late-1980s money), which now sported a very large scorch mark on its surface.

    It seems obvious I wouldn't be telling this story if the punchline was, "Miraculously, it still worked!" No, the card was dead, completely non-functional following its high-temperature reconfiguration.

    Completely at a loss for what to do, I sent the card back to the manufacturer for warranty service with a generic, play-dumb "card no workee" type bug report.

    Not ONLY did they honor the card warranty, not ONLY were they incredibly apologetic and chagrined about the supposed failure of their hardware, not ONLY did they send me a brand new replacement card, but the REASON they replaced my card with a new one is that they wanted to keep my original card for fault analysis. Along with the new card I received a complete service report on the old one, which detailed my case being escalated up level after level within the company's hardware support and design teams, as employee after employee was pulled into the investigation into what could possibly have caused my card to expire so spectacularly.

    Naturally, I felt more than a little guilty — not only did I basically scam a free card out of them, but I probably wasted weeks of their collective time chasing down a fault that never existed.

    I'm the first to say the whole thing makes me a pretty terrible person. If anything good came out of it, it's that ever since I've been very careful around equipment. To the point where I'm still one of those jerks who will pick up and move someone else's beverage, if they set it down somewhere I deem a little too close to a machine, work surface, travel path, etc.

    1. Doctor Syntax Silver badge

      Re: Works with hardware, too

      Ah, the good old days of SCSI when one of those HP (whatever happened to them?) DAT drives could back up your entire PC to a 2Gb tape.

      1. Roland6 Silver badge

        Re: Works with hardware, too

        > “the good old days of SCSI… DAT drives…2Gb tape”?

        Obviously worked for richer organisations, I just remember using QIC-150 tapes (150MB) which were, at the time, a vast improvement on QIC-24 and floppy disks…

        Although as you allude to, DAT was a big step forward in tape technology.

    2. Bill Gray Silver badge

      Re: Works with hardware, too

      Circa 1990, in my pimply-faced youth, I inserted an 80387 math coprocessor chip in a '386 box at $WORK. Turned the machine on, and didn't get a speed-up in floating-point math code.

      I then learned that

      (a) the '387 wasn't oriented the right way;

      (b) if you looked pretty carefully, you could see some sort of marking that told you which way it was supposed to go (though there was nothing preventing you from putting it in the wrong way);

      (c) putting it back in the right way wouldn't help; the coprocessor had already gone to Silicon Heaven.

      Like you, that was enough of an error to make me much more careful around equipment.

  19. xyz123 Silver badge

    I'm waiting for someone to fess up they accidentally deleted the plan for a perfectly fair and easy to understand HMRC tax system with no loopholes for the rich......

    1. Roland6 Silver badge

      I seem to remember a certain D. Trump a few years back, suggesting an old fashioned styled arm twist - as used many times by monarchs to replenish their treasury, of the US super rich could wipe out the US national debt. Perhaps we could formalise this into the tax code, rewarding such generosity with bronze, silver, gold, platinum etc. baubles (ie. Bank card sized bits of coloured plastic) and associated benefits (eg. Visit to Buckingham Palace, audience with the monarch, afternoon tea with the monarch etc.).

  20. Anonymous Coward
    Anonymous Coward

    Does anyone else detest the red writing for the top headlines?

    for one thing, can't see what I've read!

  21. Stevie Silver badge

    Bah!

    <MODE=INFURIATED_JIM_HACKER> SMOKE, HUMPHREY! LOUD EXPLOSIONS! I can't take this to the PM! I'll be a laughingstock!

  22. Herby

    Scripts that remove files...

    In the scripts that I attempt, my first pass is to put "echo" in front of the "rm" command so I can see EXACTLY what is going to happen. This has been quite instructive for both what it does and what it doesn't do. I learned the "hard way" on my own data.

    1. Robert Carnegie Silver badge
      Facepalm

      Re: Scripts that remove files...

      Like for instance, rm / tmp

      (Don't!)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon