back to article Largest advertising company in the world still wincing after NotPetya punch

The huge cyber attack that swept from Ukraine last week is still affecting companies, and several have been hit pretty hard, including the world's largest advertising business, UK-based WPP. The malware attack, dubbed NotPetya because it masquerades as the Petya ransomware, affected several multinationals running Microsoft …

  1. Anonymous Coward
    Anonymous Coward

    Is it just me...

    ..rejoicing in the fact that an ad-spewer has been really badly hit by this? Serve them right.

    1. Anonymous Coward
      Anonymous Coward

      Re: Is it just me...

      It is the perfect storm in the cesspool

      1. Ad-slinger

      2. Outsourcing to IBM

      3. TUPE-ing staff just so that they are terminated by someone else.

      Well deserved on all counts. Bravo. Bis. Encore. Repeat please.

      1. Anonymous Coward
        Anonymous Coward

        Re: Is it just me...

        Well, all the companies I know that were affected was actually via internal IT rather outsource. There is the common practice in many companies that domain admins use the same prooveleged user in their day to day laptop, almost all cases I have assisted or I have knowledge about this was the case. And there is a very simple reason, service providers use VPNs and jump hosts to admin their customers, those users won't be found in any laptop exposed, so I honestly doubt that outsourcing is the reason here. Patching with internal IT isn't any better and any admin ready will not agree with me.

        Merk/msd for example they have huge internal it department, with cyber security specialists and so on, as far as I know they still have several offices stoped!

        Plain simple reality is that companies want to hire but there aren't enough IT guys, and more than half really have little clue what they are doing and the most vulnerable ones are the local it guys in the office that image your laptop, they have domain admin or computer admin credentials, because they need to image the computers in local offices but they aren't necessarily very skilled.

        So isn't that companies don't want to do better, is more because people have little clue how to do better and there are too many specialists around that have little clue themselves.

        And isn't about the money, creating a second user different from the laptop using a jump host to admin ad costs no money! I can bet that in any company above 1000 employees if you go and do an audit on domain admins... you would be surprised with the results.

        So keep it simple, segregation by design, don't make it too easy on admin and don't be lazy, know what the company does, define secure locations and harden end point protection. If this is done and doesn't cost money even if you get affected is probably going to be contained to specific group of users.

        1. Anonymous Coward
          Anonymous Coward

          Re: Is it just me...

          You would lose that bet then, depending on how you want to interpret it. 7000+ employees, subsidiary has 300 employees that this applies to. No end user is local admin, the local admin account on each desktop/laptop/server is random, changed every 30 days. 0 domain admin user accounts, IT admin accounts are per security zone with separate forests, some requiring to be physically present in that zone, no remote access. When using remote access for the zones that can be, VPN with username, password and smartcard (with pin), to a terminal server for remote admin, with the admin account being only valid for said zone. User permissions set using RBAC methods, well what you can achieve anyway with AD. All access via VPN and RDP monitored, all file access for sensitive files monitored, all systems snapshoted hourly and backed up.

          In higher security zones applocker (I know not 100%, but will stop most) enabled, with white listed apps and locations. Will probably end up enabling that for all servers and admin points in all zones. Applocker is set to log application execution, all system logs centrally stored.

          All systems scanned on a monthly basis for missing patches and known vulnerabilities, pen tests done quarterly.

          Could go on with all the extra bits, but you get the idea.

        2. TheVogon Silver badge

          Re: Is it just me...

          "There is the common practice in many companies that domain admins use the same prooveleged user in their day to day laptop"

          That's certainly not common practice in companies large enough to have an "IT department". Most companies make it such that you cant easily work like that. No email, no profile, etc. on admin accounts. So that you must use a separate user account only for admin type operations.

        3. Halfmad Silver badge

          Re: Is it just me...

          Those local IT staff don't need to understand it, they need to have a process in place and training to say "look guys, you use your admin account when you need to do admin work, at all other times you use a standard account". Nice and simple, also don't give access to network shares etc from the admin accounts to stop them wanting to use their personal shares etc. Typically these accounts require local admin rights and very little rights on the network.

          Hell I can monitor admin account usage due to the way our transparent proxy picks up anyone launching a browser or windows looking for updates.

        4. Anonymous Coward
          Anonymous Coward

          Re: Is it just me...

          WPP insider here; I've also written a lengthy post exposing WPP:

          Actually, the local IT staff are fairly knowledgeable, and I trust my local colleagues more than my own decisions sometimes, and this is coming from someone with 18 years of experience. The twits on the outsourced end that have no desire to "do the needful" is what concerns me, and as they are the sole personnel that manages the computers (except when something enters high priority and malfunctions). For some divisions of WPP, they still use IPSEC-based VPNs, and there are some servers that have not received updates in over 2 years prior to the attack, or basically how they've been greeted with a red screen that reads "Ooops, your important files are encrypted."

  2. fnusnu

    It us just you

    WPP is a huge, British success story

    1. Anonymous Coward
      Anonymous Coward

      Re: It us just you

      So successful in fact that they don't pay their fair share of tax.

      https://www.theguardian.com/business/2009/feb/04/tax-gap-series

    2. Anonymous Coward
      Anonymous Coward

      Re: It us just you

      WPP is a huge, British success story

      That is definitely arguable. Would you regard a company headquartered in the Channel Islands as really "British"? There's Kazakh mining corporations listed on LSE, so WPP's London listing means nothing. And the Channel islands are certainly not part of the United Kingdom, and undoubtedly chosen for reasons of tax avoidance.

      It has however been a huge personal success story, with Martin Sorrell gettng awarded shares worth £42m. But my dear fnusnu, I'm sure he's worth every penny of that.

      1. Anonymous Coward
        Anonymous Coward

        Re: It us just you

        That is an odd thing to say.

        1) WPP has an 'offshore' registered office, as if registering yourself in the Isle of Man, Jersey or Sark makes you any less British?

        2) WPP is certainly up there with being one of the largest employers in the UK. Obviously not all of the +205k staff are British but a good number are.

        Speaking as a WPP staff member we were told about the incident several hours after the original incident occurred. Note that there is no such thing as "WPP VPN Network" which spans every company, Most companies operate their own closed private networks and allow other Joint Ventures with WPP member companies access to that VPN, thus a Ogilvy & Mather and JWT project for British Chocolate Company might be a specific BCCOMJWT company with its own office, seconded staff and IT unit, plus access to both OM and JWT VPNs. This is possibly what occurred in Ukraine, but I do not know..

        All staff were instructed to shut down computers and effectively go home. As the nominated unlucky last out of the door, I got to go through an entire floor of a big office near Strand and double check that every Lenovo or Dell laptop had actually been powered off and every iMac had its power cable pulled out, just in case. Next day all Mac users were told that it was safe for them to work but all Windows users could either have access to a spare Mac, if possible, or wait and see. By the end of the week I think that everyone groupwide was back up and running.

        I do not think that anyone at WPP or a WPP company, apart from a couple of staff in a Ukrainian office, actually had a computer affected by the attack (well, apart from losing several days work). I do however think it was a bit of a relief that a certain company's staff put their hands up almost immediately and confessed that their 'patching' had not taken place.

        On reflection: lack of administration access granting is a major issue - both on Macs and on Windows which I gather is now being dealt with. Better install profiling for both Mac and Windows is certainly called for.

        1. Anonymous Coward
          Anonymous Coward

          Re: It us just you

          Downvoted for working at an ad agency.

        2. Doctor Syntax Silver badge

          Re: It us just you

          Obviously not all of the +205k staff are British but a good number are.

          Some of us have problems associating "good" with members of the pestering industry. That's part of the reason that Ledswinger's post got so many upvotes and fnusnu's got so many downvotes.

    3. Voland's right hand Silver badge

      Re: It us just you

      WPP is a huge, British success story

      You forgot the gigantic "Sarcasm" tags on that. Or at the very least "Joke Ahead" sign.

  3. Anonymous Coward
    Anonymous Coward

    Until cost of shit happening is greater than cost to prevent shit happening nothing will change.

    Sadly this runs all through society from IT to tower blocks and beyond.

    1. Destroy All Monsters Silver badge

      We need to merge boardroom meetings, malware and Grenfell tower conflagrations. But how?

  4. Uberseehandel

    Possible path for malware to attack sites without a "Ukrainian connection".

    I know of consultants specialising in ERP configuration and implementation that rely on teams of developers living in the Ukraine. With so many flavours of Open Source ERP being promoted, from a very small original code base, this could be an important vector.

    1. Uberseehandel

      Re: Possible path for malware to attack sites without a "Ukrainian connection".

      Its the "thumb down" vote that intrigues me ;-[]

      1. Anonymous Coward
        Facepalm

        Re: Possible path for malware to attack sites without a "Ukrainian connection".

        "Its the "thumb down" vote that intrigues me ;-[]"

        I think it's to do with trying to deflect blame for notpetya onto 'Open Source ERP' instead of a malicious .DLL being loaded as part of an update to M.E.Doc, an accounting software package used in the Ukraine, a close sourced commercial product. Access to the M.E.Doc servers was provided through compromised credentials

  5. Aladdin Sane Silver badge

    I thought Google was the world's largest advertising business?

    1. Pascal Monett Silver badge
      Coat

      I'm guessing that "world's largest" actually means "real-life world's largest", which would exclude the Internet and thus avoid shaming themselves.

      But what do I know.

      1. Ken Hagan Gold badge

        In what sense is any advertising more real-world than any other? The product is a message, not an object, and the preferred media of transmission are surely just whatever reaches the target audience.

        1. MonkeyCee Silver badge

          What is advertising

          " In what sense is any advertising more real-world than any other? "

          Like a lot of things, it's to do with how the industry itself self identifies. Ebay isn't a retailer itself, but acts as a middleman between retailers and customers. So would eBay be considered the third* largest retailer in the world, or not a retailer at all?

          The biggest/most profitable/most successful "jeweler" in the UK doesn't sell any traditional gemstones. They sell pretty looking stones, that are well cut, and are very popular with their customers. But becasue they don't sell the *right* shiny stones** they get snubbed by the rest of the industry and are often derided as selling costume rather than real jewelry.

          * or whatever position it would be, presuming Amazon and Walmart conglomorates at least are bigger.

          ** gemstones and gold only have value by merit of being pretty and traditional. That a natural diamond is worth more than a synthetic diamond which is worth more than cubic zirconium (you've got to be fairly expert to tell the difference between these even with a loupe) has no rational basis.

    2. returnofthemus

      I thought Google was the world's largest advertising business?

      LOL!

      But no, Google are the World's largest Spyware Agency, there's a big difference ;-)

  6. adam payne Silver badge

    'IBM declined to comment.'

    Nothing to see here move along.

    1. Anonymous Coward
      Anonymous Coward

      "cloud deal"

      Have you noticed that IBM is calling all of these old school outsourcing deals "cloud" now? It is being delivered "as a service." By that definition, Accenture is a huge cloud company. Just anything delivered as a service is "cloud."

      1. returnofthemus

        Accenture is a huge cloud company

        LOL!

        Funnily enough a couple of years ago this is exactly what the Accenture CTO Paul Daugherty declared.

        The fundamental difference being that IBM do operate a large scale public global cloud comprising over 50 data centre's, with a focus on Hybrid cloud integration, but you highlight well just how generic the term 'Cloud' has become.

        However, if you think that it's easy for large enterprises to pick up everything they've been doing inhouse for many decades and bung them onto someone else's infrastructure easily, then you're mistaken, because if that were the case companies like Accenture wouldn't exist.

        PS Cloud is a journey, not a destination ;-)

  7. wyatt

    Interesting quote:

    "Organisations should never outsource responsibility for security".

    The responsibility will always remain with the organisations board, how they implement this is their responsibility is it not? I can ask who I want to do what I would like, but if I don't detail what needs doing and also check that is being done then I'm liable for their failures.

    1. Korev Silver badge

      But if/when things go tits up and the inevitable blamestorming happens, the board can point the finger at $OUTSOURCER and deny responsibility...

      1. Doctor Syntax Silver badge

        "the board can point the finger at $OUTSOURCER and deny responsibility."

        And when it comes to court the court under GDPR will point the finger right back where it belongs. Or if it's something that affects financial performance the market will also hold the company responsible and amend its share price accordingly. You can't outsource responsibility.

  8. Anonymous Coward
    Anonymous Coward

    IBM...

    RB - outsourced to IBM

    WPP - outsourced to IBM

    BNP - outsourced to IBM

    Hmmm.

    1. ecofeco Silver badge

      Re: IBM...

      Mere coincidence. /s

      1. Anonymous Coward
        Anonymous Coward

        Re: IBM...

        Mere coincidence

        Are you sure you spelled that right. Shouldn't it be "Merde Coincidence" as a shortened form of the classic law of nature: "Like dissolves in alike".

    2. John Brown (no body) Silver badge

      Re: IBM...

      No one ever got fired for outsourcing to IBM.

      But maybe they should be.

  9. Your alien overlord - fear me

    WPP probably missed cut-off dates for adverts to be pushed (to magazines, newspapers etc) so their clients have lost out. Probably will look elsewhere for their next advertising partner.

    Their big corporate clients will have clauses like IT security/best practices as part of any contract. Again, time to look elsewhere.

    So WPP, to save a few pounds in the short term, will probably never recover financially from losing many lucrative corporate clients. This should be shown as a example of outsourcing to other businesses who think it's a good idea to get rid of their inhouse IT staff.

    1. Anonymous South African Coward Silver badge

      And they will not learn, still the outsourcing trend will continue.

      Outsourcer = lowest common denominator

    2. Anonymous Coward
      Anonymous Coward

      RE: Impact

      There are a few risks for WPP from this.

      First of all there is the ability to deliver work. In the short term, I suspect they will cope bu people going the extra mile etc.

      The bigger risk is on the billing/payments side - they probably have similar practices to many other companies (pay late/bill early) that results in very tight deadlines for paying bills if things go wrong. And if suppliers don't get paid, they might not supply services/goods until the issue is resolved which hurts you for a month or two... On the billing side, if you aren't billing, the money isn't coming in which slows things down and delays spending on work for customers.

      If it takes a month to resolve (i.e. read reports of the NHS still being affected in very minor ways), then it might take 2-3 months until business is back to usual.

  10. tedleaf

    " in line with good practice" !!

    Even loody half arsed practice says you don't get affected in the firstt place !!

    You would hope for a better standard of bs from world propaganda pusher,but then they themselves always were the only idiots to believe anything they say..

    Pity it didn't take out every single system they have,the world's bs levels would drop a lot for a while..

    1. Anonymous Coward
      Anonymous Coward

      Re: Good practice

      As a former WPP employee, I know parts of the group had pretty robust patching and AV policies and while local admin access was common, it was mitigated by restrictive inbound firewall policies on devices which was managed via group policy (Windows) and Casper (OSX) in response to previous virus outbreaks and over the last 5+ years the good practices had withstood the daily tests of infected devices being connected to the network. i.e. it was well tested

      When IBM came along, the patch and security management was handed over to them. Whether IBM did anything with that handover information is uncertain.

      1. Down not across

        Re: Good practice

        When IBM came along, the patch and security management was handed over to them. Whether IBM did anything with that handover information is uncertain.

        I think the answer to that is pretty obvious.

      2. Alistair Silver badge
        Windows

        Re: Good practice

        "When IBM came along, the patch and security management was handed over to them. Whether IBM did anything with that handover information is uncertain."

        I can tell you what happened here. The SM on the account said clearly, "This is IBM, (chuckle), you have a patching policy, how quaint, We'll hand this to the EUS SEC team and they'll set up a *real* patching system for us."

        Aaaaaaaaaaanddd.... someone came back with "Yes, we can do that for an additional 2.25MGBP/year on the contract, please sign here".

        And there were negotiations...... which is probably where they are now.

        This is *outsourcing* --- sell shit to the victim at the lowest possible price, sucker them in for as many years as possible and then start bolting on costs once the ink starts to dry.

        1. Doctor Syntax Silver badge

          Re: Good practice

          "sell shit to the victim at the lowest possible price"

          Which, given the victim in this case, explains why the commentariat regards this with more than a touch of schadenfreude.

    2. Doctor Syntax Silver badge

      "but then they themselves always were the only idiots to believe anything they say"

      No, their clients believe what they (the advertising industry) say.

  11. IanRS

    Local admin rights

    "There are many accounting applications that require local admin for applications to run."

    Why on earth does an accounting application require local admin rights other than perhaps for installations?

    1. Anonymous Coward
      Anonymous Coward

      Re: Local admin rights

      The software is written and/or developed under admin, then for whatever demented reason it's simply promoted to prod release without proper QA, it's simply easier to just say "Install/run with admin" rather than do it properly. I'm not blaming devs, they're often under pressure to deliver but we all share the blame on this sort of bad practice. Ops people shouldn't ever give admin rights without full auditing, or at the very least a time limited set of admin rights, devs should only ask and use admin rights when there is absolutely no other choice and it's been discussed with the admin teams why it's needed. We can all play our part in stopping the stupid practice of just being lazy and sticking everything under control of an admin account.

      Finally if done right, a user and their system is not left at the mercy of the scum who will happily screw your PC over and leave you with nothing.

      1. Anonymous Coward
        Anonymous Coward

        @AC - Re: Local admin rights

        No, let's blame devs, they're causing all this. Next in line is the idiot who decides/approves purchasing and deployment of this crap.

      2. Anonymous Coward
        Anonymous Coward

        Re: Local admin rights

        With this type of software, if for some reason really has to be used, I profile it and find out exactly why it needs to run as admin, what retarded crap its doing and then try and work around it so that it no longer needs to be run as admin. But then there are those retarded applications that look at the account name, if it doesn't = administrator don't run as you are not admin, even if your account is admin.

        I hate expensive specialised software that appears to have been written by a 15 year old in their for a school project the day before it needed to be handed in.

      3. Ken Hagan Gold badge

        Re: Local admin rights

        "I'm not blaming devs, they're often under pressure to deliver but we all share the blame on this sort of bad practice."

        I've been developing for NT since version 3.1 and I can place hand on heart and swear that I have never shipped a product that required admin except for configuration that actually does require admin. I won't swear that the configuration was always confined to a separate process, but I will swear that it had a graceful fallback when run as a normal user.

        I most definitely do blame the devs. A gratuitous requirement for admin rights is sufficient grounds for sending the product back and asking for a refund, if for no other reason than it indicates that the developers haven't a fucking clue and ghod alone knows what else is broken under the hood.

        (So actually, yeah, like the other guy I blame the devs first but then also the procurement people who tolerate this shit.)

    2. Erik4872

      Re: Local admin rights

      Believe it or not this was the norm for internal applications ages ago...and those applications exist today. Not everyone is an Agile DevOps phone-based shop; there are plenty of businesses with millions of dollars running through stuff like poorly coded Access databases or VB front end GUIs.

    3. sanmigueelbeer Silver badge

      Re: Local admin rights

      Why on earth does an accounting application require local admin rights other than perhaps for installations

      I think you know the answer to this. It isn't just accounting proggies. A lot of poorly written codes run the risk of not working when OS gets patched. Getting those poorly written codes updated or re-written is a game of Russian Roulette with all six chambers loaded.

      Remember what happened to WannaCrypt?

      1. Wensleydale Cheese

        Re: Local admin rights

        "Why on earth does an accounting application require local admin rights other than perhaps for installations"

        Far too often it's because they haven't paid attention to file ownership and permissions. It goes something like this:

        Step 1: Install one or more data files under admin ownership

        Step 2: Oh Noes, the app won't run, let's run it under admin instead

        Here, the installation program or script is usually to blame, by omitting Step 2a, setting those files to have the correct (non-admin) ownership.

        ACLs offer powerful functionality in this area but are all too often ignored.

    4. Destroy All Monsters Silver badge
      Trollface

      Re: Local admin rights

      Why on earth does an accounting application require local admin rights other than perhaps for installations?

      Preemptive tax auditing & collection?

    5. Doctor Syntax Silver badge

      Re: Local admin rights

      "Why on earth does an accounting application require local admin rights other than perhaps for installations?"

      Because it's Windows and that sort of thing happens there.

  12. Led boot
    FAIL

    Worlds largest advertising company?!?

    ... bigger than Google?

    1. Pompous Git Silver badge

      Re: Worlds largest advertising company?!?

      WPP is the largest advertising company in the world measured by billings and revenue.

    2. Anonymous Coward
      Anonymous Coward

      Re: Worlds largest advertising company?!?

      ... bigger than Facebook?

      1. William 3 Bronze badge

        Re: Worlds largest advertising company?!?

        Yes.

        Facebook, Google et al, are all INTERNET based advertising.

        There is more to the world than INTERNET advertising.

        1. Pompous Git Silver badge

          Re: Worlds largest advertising company?!?

          "There is more to the world than INTERNET advertising."
          There's also more to the world than newspapers, magazines, television, movies, billboards etc. Ad agencies are also known as creative agencies; that is they create the advertising content and arrange for its placement. Some commentards obviously can't tell the difference between content creation and the medium by which it's delivered.

          1. Doctor Syntax Silver badge

            Re: Worlds largest advertising company?!?

            "Ad agencies are also known as creative agencies"

            To themselves and their clients. To the rest of us they're known as pests.

            1. Pompous Git Silver badge

              Re: Worlds largest advertising company?!?

              "To themselves and their clients. To the rest of us [ad agencies are] known as pests."
              Presumably then you would prefer to pay ever so much more for things which would certainly be the case without advertising.

              I have very fond memories of the 1960s and 70s poring over the adverts in Electronics Australia and Electronics Today while deciding what to build next: graphic equaliser, compander, amplifier, loudspeakers... Along with ever so many other electronics enthusiasts in Oz, I used to pay for the Dick Smith Electronics Catalogue that was issued once a year.

              Then there's TV advertising that funds the TV shows that ever so many people live for. Don't like the ads? Then don't watch TV. Simples really.

              Perhaps I'm somewhat biased by having worked for an award-winning magazine that failed to return sufficient income for the investors, so it was folded.

              The DTP revolution led to every man and his dog believing they could do just as well as the professional "pests". My favourite advertisement of all time was one such. It appeared in one of those free airline magazines you get when travelling. It was a gorgeous full-colour picture occupying a quarter page (so quite expensive to place). For the life of me I can't remember what it was advertising, but that's irrelevant in this instance. There was no business name, address, or telephone number... And of course it saved the advertiser heaps by being produced in-house by the advertiser :-)

              Then we can contrast this with advertising in the good old People's Paradise:

              No competitive advertising allowed

              Worked to fulfill the government's economic plan by redirecting demand.

              Propaganda, propaganda, propaganda...

              Public service announcements (aka propaganda)

              Promote use of unacceptably large inventories (stuff nobody wanted to buy)

              Sell obsolete goods (see previous)

              Stores were not allowed to advertise their locations

              Citizens were not allowed to place classified ads

              1. CrazyOldCatMan Silver badge

                Re: Worlds largest advertising company?!?

                Then there's TV advertising that funds the TV shows that ever so many people live for. Don't like the ads? Then don't watch TV. Simples really.

                Except in enlightened countries that can fund radio and TV without the irritation of adverts. Don't like ads? Watch a channel funded by public tax.

        2. The obvious

          Re: There's more to the world than INTERNET advertising.

          ...only until google decide otherwise.

          Give it a couple of years and google maps will be prioritising routes past google-owned hoardings, if they aren't already.

  13. a_yank_lurker Silver badge

    Dumbsourcing

    IT is now a business critical area and outsourcing (aka dumbsourcing) it is a recipe for a well deserved disaster for medium to large organization. Plus, do not dumbsource to someone as incompetent as Itsy Bitsy Morons, not that any of other outfits are terribly competent.

  14. Anonymous Coward
    Anonymous Coward

    Let's make this clear, outsourcing is to blame.

    If internal IT team screws the pooch, it's their pay (or job) that is affected. If it's the outsourcing company, well it's just a customer. Upper management of the two will meet on a golf court and business will go on as before: "IBM has been working alongside our staff and IBMers have been invaluable in working tirelessly to help WPP resolve this issue."

  15. John Smith 19 Gold badge
    WTF?

    3 years to install a "central patch management" system?

    Sounds like something the NHS could have used as well.

    Then again IBM still haven't managed to do so in 3 years (and you can bet they were getting big $ from WPP) so perhaps the NHS wouldn't have been any better off.

    BTW We know the NHS had all that legacy software that simply won't run on anything but WinXP and the most up to date they could get that to run in was Win 7.

    But what's WPP's excuse for this? Those local admin rights sound like trouble, but "Windows has an option to store encrypted passwords" WTF? You have to ask it to?

    The people who need this won't know they need it (because they are too busy clicking on every file link they receive from random email addresses between downloading games from third party app stores) and will have no clue how to enable it.

    Lots of fail to spread around here. Worst case the company goes down the pan. A lot of basically innocent peoples livelihoods could just have ended.

    1. Ken Hagan Gold badge

      Re: 3 years to install a "central patch management" system?

      " Those local admin rights sound like trouble, but "Windows has an option to store encrypted passwords" WTF? You have to ask it to? "

      I don't know what that's all about. Windows (NT flavour, obviously, since DOS doesn't count) has always stored passwords just as securely as any other mainstream OS. Best practice in this area was established about half a century ago and isn't actually difficult.

      Of course, there's nothing stopping some clueless twat of a programmer from storing a password in plaintext in an INI file on a network share open to the universe, but you could do that on any OS.

  16. Erik4872

    "Not our core competency"

    I understand how an MBA might think outsourcing all IT functions is a good idea. They outsource the cafeteria, garbage collection, etc. and never have any trouble. However, I guarantee you anyone making the decision to outsource has no idea how the magic box on their desk or in their lap gets software delivered, patches applied, etc.

    This is where it comes back to bite companies. IBM and the like are trying to get away with doing the least possible work for the lowest internal cost to maximize revenue. They have zero interest beyond the SLA whether your organization is running. This leads to the added problem of long-term outsourcing...eventually no one in-house knows anything about the IT environment and the company is powerless to take back control.

    The company I work for, which is usually on the trailing edge of trends, is finally starting to realize that offshoring all new software development to third parties was a bad idea...precisely because someone woke up one day and realized they were permanently tied to the offshore firms because anyone internal who knew anything about the guts of our software had been fired. If this is happening here, I'm hoping it's the continuation of a trend elsewhere.

  17. hellwig

    How and Why?

    Most, if not all, confirmed cases stemmed from a malicious update to MeDoc, Ukraine's most popular accounting software.

    How did a malicious update make it into the official stream, and why did anyone install it?

    Leaving aside why you should digitally sign these things, what prompted anyone to install this? Was it set to auto-update? And what IT organizations in their right mind leave the door open to automatically pull shit off the internet, ESPECIALLY if it's NOT signed?

    How often does this software get updated? Wouldn't the developers of the software say "Hey, we didn't push an update yesterday"? Why would this software need automatic updates?

    Remember when releasing quality stable software was a successful business model? You updated once a year, if that? In general, updates were not for fixes but new features? Remember?

    Instead, business push more and more frequent updating like it's a good thing. A good thing would not be to release shitty software in the first place.

    Seriously, how much changes in accounting? I'm sure the government only releases updates once a year, if that. I'm sure some companies are still running DOS machines because that's what their accounting software runs on. Peachtree was a very usable product from what I understood.

    1. Pompous Git Silver badge

      Re: How and Why?

      "How often does this software get updated?"
      Accounting software gets updated as frequently as the taxation system changes, so at minimum once a year. Since paying the correct amount of taxes is mandatory, updates to the software that manages such is also mandatory. It's a bugger innit?

      1. Doctor Syntax Silver badge

        Re: How and Why?

        "Accounting software gets updated as frequently as the taxation system changes, so at minimum once a year."

        At a minimum indeed. Because if it's supplying companies trading in multiple countries there may be a whole raft of taxk changes happening at different times of the year.

        But this is best done by keeping the executables as stable as possible and pushing the changes to tax rates as data, preferably human readable text data.

        1. Pompous Git Silver badge

          Re: How and Why?

          "But this is best done by keeping the executables as stable as possible and pushing the changes to tax rates as data, preferably human readable text data."
          Only possible if you can anticipate what lunatic changes are going to be made to the taxation system down the track. Here in Oz for example, this month we have a change in the PAYE system that is date dependent and employee number dependent...

    2. CrazyOldCatMan Silver badge

      Re: How and Why?

      Leaving aside why you should digitally sign these things

      Given that the signs are that the attackers had access to the source code (which almost certainly includes the signing certificate) it wouldn't have helped.

      it probably doesn't help that (with the devs of my knowledge anyway) that their idea of security is locking their desk drawer at night.

  18. Anonymous Coward
    Anonymous Coward

    It is amazing that anyone still has outsourcing in place. Why wouldn't you just move it to a public cloud, which would be better on all counts than an outsourcer, and then hire a smaller in house staff with a high skill set to manage the cloud. If the goal is cost reduction, that would be much less costly.

    1. ecofeco Silver badge

      A good idea, hypothetically, but you are still at the mercy of a 3rd party.

      1. Anonymous Coward
        Anonymous Coward

        " you are still at the mercy of a 3rd party"

        Well you are always at the mercy of a third party. If you still run IBM mainframe, IBM could have some software bug in z/OS which you are at their mercy to fix. Even if you are 100% on prem, you are at the mercy of your suppliers.... Cloud would just be much less likely to get messed up than some third party outsourcing arrangement.

    2. patrickstar

      What does public clown have to do with outsourcing?

      There are lots of places which even have their own physical datacenter but outsource large parts of operating the things in it.

      If anything, moving to your typical public clown (like AWS) with their "lots of disposable unreliable servers" model would require a typical shop to start outsourcing, since their in-house team won't be able to deal with the huge increase in complexity that follows from that model. Plus you might actually have to toss out and replace a lot of existing, well-functioning, software.

      For the "clown services" where individual servers are actually reliable, at most you've gained capacity scaling and not having to maintain the physical boxes. The latter (and to some extent the former) which you can gain in ways that don't involve using someone else's shared infrastructure with all the issues that arise from this.

      In any case, this in no way saves you from needing someone (outsourced or not) to keep the things running on the servers.

      1. Anonymous Coward
        Anonymous Coward

        " typical public clown (like AWS) with their "lots of disposable unreliable servers" model would require a typical shop to start outsourcing, since their in-house team won't be able to deal with the huge increase in complexity that follows from that model. "

        I am not an AWS enthusiast, but you don't need to manage the actual cluster yourself. AWS does that for you. The way to go with cloud is to use a new serverless model, Google Cloud style, where you really don't need to manage anything at the infrastructure level. It is all automated, there isn't even a concept of a VM, storage volume, etc. Just a service, microservice running which is paid for on data throughput... just like you don't need to manage the infrastructure for some third party API you are using. It is just there and it just works.... Also much more economical as if you are not using a service, there is no idling cost.

        1. patrickstar

          Then you at the very least need people who can rewrite/redesign all your existing stuff to fit that model and then maintain the result. Chances are that's not your existing developers / IT dept and/or that you'd need significantly more hands to actually do this with all relevant existing software in a reasonable timeframe.

          So, point still stands.

  19. William 3 Bronze badge

    I'm setting up a board of directors outsourcing company.

    And I'm going to take it to the shareholders meetings of every company.

    Lets' see how those fuckers feel when it's their jobs being outsource.

    Just say to the shareholders, shit dudes, we can give you 20% more money this year in dividends when we outsource those lazy overpaid fuckers.

    1. Ken Hagan Gold badge

      Re: I'm setting up a board of directors outsourcing company.

      I'm assuming you aren't actually going to try this but...

      There's no shortage of smart-enough people in poor-enough countries, so it would probably work. It is also in line with the bizarre-but-conventional wisdom that you can manage something that you have no experience of because management is a skill in its own right.

      All you need now is to find some pissed-off-enough shareholders to volunteer to be your first customers.

      1. John Smith 19 Gold badge
        Unhappy

        "in line with the bizarre-but-conventional wisdom..manage something that you have no experience of "

        When you put it down like that in Red and Yellow it really does look absolutely bats**t crazy.

        But then again the British Civil Service runs on similar principles.

        Which does explain quite a lot.

  20. EyeBeeM
    FAIL

    Former WPP Staff

    I worked at a WPP company during the IBM transition and I am surprised it took so long for something like this to happen.

    The entire process was so badly run, staff who transferred from WPP were demoralised as they knew redundancy were on the way. IBM management didn’t have a clue about how an advertising company works. Their support model might work at a bank but a creative agency is a different kettle of fish.

    All IBM wanted was knowledge transfer, once they had this they got rid of staff. During my 6 years, prior to IBM we never had an incident like this.

    WPP lost some fantastic talented engineers and I feel sorry for the staff who have to go through a call centre in India before waiting for day to have a simple IT issue resolved.

    Outsourcing never provide a better quality of service, its decided by accountant who only see the bottom line. I wonder how much money was lost for all the down time, clearly these figures are never factored into these so-called cost saving exercises.

    1. CrazyOldCatMan Silver badge

      Re: Former WPP Staff

      I wonder how much money was lost for all the down time, clearly these figures are never factored into these so-called cost saving exercises.

      Of course not - we have an SLA!

      Of course, said MBA-muppets have never actually read the SLA, let alone understood what the terms mean. Or that the penalties are harder to get at than an England World Cup team..

    2. returnofthemus

      Re: Former WPP Staff

      "IBM management didn’t have a clue about how an advertising company works".

      LOL!

      IBM have been working with the Advertising industry and WPP particularly for decades, in fact IBM were once the principal supplier to Mediaocean (formerly Donovan Data Systems) who were a leading supplier of software to Ad agencies.

      Furthermore, IBM have built one of the world's largest digital agencies

      Having worked with a large proportion of WPP companies, I'd blame the operating model, far too many companies in the group have a fairly liberal autonomy and that clearly needs to be brought under a centralised control, running outdated Windows PC's also doesn't help.

  21. Stevie Silver badge

    Bah!

    Correlation with BYOD policies?

    1. Anonymous Coward
      Anonymous Coward

      Re: Bah!

      No correlation with BYOD policies.

      WPP has allowed (expected) contractors to provide their own laptops for 10+ years in many countries and the systems they had would contain issues to an office and usually just the affected machine.

      This time EVERYTHING's infected - BYOD doesn't give you access to Active Directory domain controllers...

  22. Herby

    Why why, why...

    Doesn't this type of problem attack the command/control of the spam networks. I get multi-megabytes of junk about Glucophage (or whatever it is called) all day and all night. Please have an attack vector that looks for spam generators and KILLS them.

    I might even contribute, but in the end it is probably illegal.

    1. Anonymous Coward
      Anonymous Coward

      Re: Why why, why...

      Most under WPP's employ are those designing the advertisements, providing PR, and even website design for several clients. Sadly, they've no control of where the ads go.

      -WPP insider

  23. Anonymous Coward
    Anonymous Coward

    WPP always make great IT decisions. LotusNotes for example. I wonder who supplied that to them?

  24. D Moss Esq

    I got it from Agnes

    Some companies, such as Maersk, did direct business with Ukraine, which would explain how the malware got on its system, the F-Secure man added. "However, one victim we spoke to had no ties to the Ukraine at all, so it is a mystery as to how they got infected. Its spread via VPN is one possibility."

    https://www.youtube.com/watch?v=7jKiunSRzAI

  25. EyeBeeM

    One word "Redundancies"

    So obvious this was going to happen, When IBM won the contract they guaranteed staff their jobs for 12 months. Once this was up they reduced the staff who transferred from WPP by 90%. Out of the remaining 10% left most left after seeing the direction (Down) IBM was taking WPP. IBM and WPP lost experienced knowledgable staff and even with knowledge transfer IBM are absolutely clueless.

    This is only the tip of the iceberg and I am glad its being exposed. The IT service at WPP is shocking since IBM took over. Any member of staff, excluding directions who have to tow the party line will tell you the same.

    Simply calls such as password resets which used to take minutes now takes hours with IBM's call centre. A new user setup needs at least a weeks notice, this doesn't work in advertising when staff are brought on last minute to deal with new pitches/accounts.

    Martin Sorrell couldn't care less, he has 2 full time WPP IT support staff, so never needs to contact IBM, unlike the rest of his staff.

    IBM are traded on their name and past glory. The IBM of today is nothing like what is used to be, run by accounts and suits. Don't even get me started on Watson... Does anyone know what it actually does!

  26. Bob Hoskins

    Class action lawsuit against the NSA?

    Anyone?

    1. Destroy All Monsters Silver badge

      Re: Class action lawsuit against the NSA?

      My Chinese Cookie says "unlikely"

  27. W. Anderson

    Address the real software vulnerability

    There should be no sympathy for companies, organizations, academia, governments or any other entity that continues to use Microsoft Windows Operating System (OS) Software that has been confirmed as the attack point for NotPetya, Petya, WannaCry and every other Ransomeware and attack Vector in last several years, including those against HomeDepot, Target, TJX and dozens of other victims.

    Viruses and Ransomeware do not affect Apple MacOS, Linux or ChromeOS computing endpoints, so the continued use of Windows is a stupid and non-sensical decision in 2017, particularly for Business and governments.

    Richard Clark, Cabinet level appointee as Cyber Czar to President George Bush administration, who was further retained by President Obama for a while, declared in his retirement speech in 2010 that one of the greatest threats to USA going forward was Cyber Terrorism, and Microsoft OS and other software were major contributors to that vulnerability.

    1. dew3

      Re: Address the real software vulnerability

      Richard Clark... was not cabinet-level appointee (he was one of several dozen "special advisors") and left government in 2003, making it rather hard for Obama to retain him - and no, Obama did not rehire him later. Presuming he said that (I cannot find the quote anywhere), and without defending Microsoft or its virus-magnet offspring Windows, Clark was a professional manager with approximately zero technical/IT chops; I always took anything he said on any technical subject with a quite large grain of salt.

    2. patrickstar

      Re: Address the real software vulnerability

      Forward 15 years after a lot of places followed your advice and switched to MacOS:

      "There should be no sympathy for companies, organizations, academia, governments or any other entity that continues to use Apple Mac Operating System (OS) Software that has been confirmed as the attack point for every Ransomeware and attack Vector in last several years.

      Viruses and Ransomeware do not affect Windows, Linux or ChromeOS computing endpoints, so the continued use of Windows is a stupid and non-sensical decision in 2017, particularly for Business and governments."

      It's a matter of market share and thus attacker's desire to target it, not some inherent security deficiency that the other systems lack. Especially in the case of the current outbreak (NotPetya) blaming Windows is a sure sign of idiocy since it largely spread via stupid admin practices and not some inherent Windows flaw.

      In fact, there has been ransomware for both Linux and MacOS. They just aren't nearly as wide-spread because both systems have a really low share of desktop users, especially in the kind of places that tend to give large headlines when hit by ransomware.

      As for ChromeOS, if you want a really locked-down desktop environment with no chance of running applications introduced from outside, you can do that with Windows (or any other of those systems) instead of signing over your soul (and corporate secrets) to Google. If anything, THAT should be considered a non-sensical decision, particularly for business and government and anyone else that actually does real work on their computers.

      PS. The large TJX credit card compromise a number of years ago mostly involved open WLANs and SQL injection... not exactly Windows specific attack vectors there either.

    3. Anonymous Coward
      Anonymous Coward

      Re: Address the real software vulnerability

      WPP insider here: There were a large portion of Mac users within the company, but there were certain software that are only designed for Windows, including MediaOcean Daisy that looks like it has barely changed since the 1990's.

  28. Anonymous Coward
    Anonymous Coward

    WPP Insider

    Good day: I am another WPP insider. I wish to remain anonymous, but will happily address any matters.

    For the record, I dislike all forms of advertisements; I am only staffed with WPP due to personal constraints due to lack of employment. However, I enjoy my tenure in a manner reminiscent of Stockholm Syndrome, but I am still seeking better employ.

    The insider’s claims are mostly correct, but I feel that certain aspects can be expanded upon:

    Most tasks were expected to be handled by a team in India internally known as “Worldwide IT” or “WWIT”, which is not a matter of race and ethnicity, but a coincidental lack of core competency among that group, proving that IBM have not provided proper scouting for people employed within that region. Most users greet their support with extreme chagrin, and all retained IT share a disdain for their lack of proper service, especially as myself and colleagues often bear the wrath of their incompetence, further worsened by most policies that are indicative of a desire to reduce costs at the expense of all that are involved, rather than to improve service… When a password reset entails half an hour minimum on the phone, or a matter that takes 30 minutes for resolving is stuck in their queue for 7 days, I am appalled that they have not sacked any of them; “lack of technical support” is an understatement.

    6 months? Try 2 bloody years! I have encountered a server that was utilising Windows Server 2008 that has pending updates going back to year 2015! Even worse, I’ve brought the matter to management, and that was ignored with a general preference to only perform the bare minimum, and further casual conversations with that manager indicates a general unwillingness to perform his duties due to a lack of enthusiasm for his position and low pay (which I myself am paid half the national average for an IT Manager)… Even though I’ve downloaded updates, that blade still got infected.

    Do you want to know what happened during WannaCry, at least for the sectors I were responsible for? Nothing. Absolutely. Bloody. Nothing. It was not until this attack happened that WPP have decided to ramp up efforts, but it only entailed paying the few remaining IT staff that were not laid off or scoffed at being paid a fool’s wages and left to spare all their free time and resources to restore services, with some blokes on the other side of the pond working on 4/7 (their national holiday), then only hiring temporary staff when most of the work have been performed.

    I’ve more to say, but I shall address them in individual replies.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020