back to article Scotiabank slammed for 'muppet-grade security' after internal source code and credentials spill onto open internet

Scotiabank leaked online a trove of its internal source code, as well as some of its private login keys to backend systems, The Register can reveal. Over the past 24 hours, the Canadian financial giant has torn down GitHub repositories, inadvertently left open to the public, that contained this sensitive information, after The …

  1. Daedalus Silver badge

    Mean Mr. Mustard

    "Public" GitHub means one thing to me: it didn't cost them anything. Had they paid the rent, they would have had a private repository from the beginning. Either the bean counters refused to OK the fees, or somebody set this up intending to take it private Real Soon Now.

    1. Sgt_Oddball Silver badge

      Re: Mean Mr. Mustard

      They couldn't even be bothered to scratch up the cash for a private server and run something like GitLab instead.

      At least the should be able to white list access then as a most basic noddy step.

    2. cornetman Silver badge

      Re: Mean Mr. Mustard

      I am a soon-to-be ex-customer and I have to say this doesn't shock me in the least.

      1. Anonymous Coward
        Anonymous Coward

        Re: Mean Mr. Mustard

        A little late there; but then again, maybe you can find a secure home with say, Capitol One.

        1. cornetman Silver badge

          Re: Mean Mr. Mustard

          People are telling RBC or TD are the best bet at the moment.

          1. Robert Moore
            Facepalm

            Re: Mean Mr. Mustard

            People are telling RBC or TD are the best bet at the moment.

            TD tried to charge me an account closing fee when I decided to leave them after they refused to help me with a company I could not stop from removing money from my account. Much profanity was heard by everyone in the bank, near the bank, or within about 0.5Km of the bank. I then went home transfered my remaining funds onto a third party credit card, and never used them again. When they sent me mail, I wrote "Deceased" on it and stuffed it back in the mailbox.<BR>

            For whatever it bight be worth, I am now with RBC for many years, and have been very happy with them.

      2. Anonymous Coward
        Anonymous Coward

        Re: Mean Mr. Mustard

        Are you a soon-t-be ex because of this or similar cockups, or is it coincidence?

    3. JimboSmith Silver badge

      Re: Mean Mr. Mustard

      I find the term Muppet grade highly offensive to Muppets.

      I find the banks attitude to security worrying in the extreme.

      There's something quite impressive about just how crap they are. Maybe it should qualify for a Guinness World Record?

      1. Anonymous Coward
        Anonymous Coward

        Re: Mean Mr. Mustard

        IME the cobbler's children go barefoot.

      2. Mr.Nobody

        Re: Mean Mr. Mustard

        Sadly, this is the same story everyone is dealing with in the cloudy world and developers.

        Anyone who has let devs run free with a credit card or an account in the cloud winds up with stories like this. I am not suggesting that security and systems engineers don't make similar mistakes, but developers just don't think about, or have a lot of experience with locking down these environments, but the bean counters and CIOs that want to be "cloud enabled" and "flexible" and want to "innovate" just keep allowing this to happen.

        We had a bunch of open S3 containers with data on it we didn't want in the wild. We got the email from AWS telling us about it, and it took days not only find out who was responsible for it, but who had the creds to do anything about it. It was a sole developer.

    4. MAF

      Re: Mean Mr. Mustard

      Worse yet, one of the positive things about Microsoft buying Github was that they changed the free tier to include unlimited private repos. So not an excuse even for cheap-skates...

    5. iron Silver badge

      Re: Mean Mr. Mustard

      Private repos have been free on GitHub for quite some time and setting a repo private is very easy. Imo either Scotia employ brain dead devs who shouldn't be allowed near code or one of them set these repos public on purpose. Perhaps a disgruntled and now former employee?

      1. Blank Reg Silver badge

        Re: Mean Mr. Mustard

        The real question is why does any large corporation store any proprietary code on an external repository. I understand using something like Github if you're an individual or just a small company with little infrastructure, but surely a bank can afford to host their own repositories internally.

        1. fajensen Silver badge

          Re: Mean Mr. Mustard

          Because they would only stuff that up also, with the entire blame falling squarely on them?

    6. Anonymous Coward
      Anonymous Coward

      Re: Mean Mr. Mustard

      Bank source code should not be kept in any kind of cloud storage in the first place. It should be on internal systems on site and on failover systems at a backup site only with maybe 3rd party escrow with a trusted source. Even private cloud storage can be viewed by the admins running it especially with amateur hour sites such as github.

      This is what happens when you have idiots straight out of university running important systems.

      1. Anonymous Coward
        Anonymous Coward

        Re: Mean Mr. Mustard

        Viewed? Even being able to view is dangerous... but what about editing?

  2. John 104

    Morons

    Why would you put credentials for anything anywhere online in the first place? Private or public, its a bad idea.

    1. Cuddles Silver badge

      Re: Morons

      Convenience. The perennial enemy of security.

      1. Tom Paine

        Re: Morons

        Convenience, and not knowing any better.

      2. Anonymous Coward
        Anonymous Coward

        Re: Morons

        There is no convenience in compliance, an auditor once said.

    2. macjules Silver badge

      Re: Morons

      Because those morons (and I'm looking at YOU TCS) do not know what security is, do not care that they do not know and even if they did would not know what to do about it.

      "Other People's Money" and all that ..

      Oh, and by the way has anyone looked at https://www.scotiabank.com/global/en/global-site.html and checked the response headers? Just saying, like.

  3. AdamWill

    called it

    Oh man, I was counting the days until this happened ever since reading this story:

    https://www.theglobeandmail.com/business/article-brian-porter-is-making-big-changes-at-scotiabank-but-will-investors/

    as I wrote in the comments there:

    "As someone who works in IT and was previously considering switching to Scotia...the above has convinced me definitely not to do that, ever, or to invest in them."

    1. Anonymous Coward
      Anonymous Coward

      Re: called it

      paywall :-(

      1. TFL
        Trollface

        Re: called it

        Only if you have JavaScript enabled...

      2. AdamWill

        Re: called it

        Ah, sorry, I thought you got one or two free from the Globe before they cut you off. Here's the relevant quote:

        "Early on, Mr. Porter took senior bankers and board members to Silicon Valley for a lesson in pace.

        “He wanted that mindset to be understood,” Mr. Zerbs says. “A mediocre outcome is clearly not okay any more. Like, we have to be the best … and also make mistakes along the way.”

        Much of the heavy lifting on technology occurs behind the scenes, as the bank strips costs out of its legacy infrastructure through automation, cloud computing and advances in artificial intelligence. But to incubate the necessary sense of urgency, Scotiabank set up "digital factories” in five countries – Canada, Mexico, Peru, Chile and Colombia. The labs undertake rapid-fire projects aimed at solving bank-wide pain points for customers, sometimes rewriting computer code that can be deployed in the span of a few days."

        1. Rich 11 Silver badge

          Re: called it

          sometimes rewriting computer code that can be deployed in the span of a few days

          ...QA optional.

          1. Jimmy2Cows Silver badge
            Unhappy

            Re: called it

            Optional would suggest someone had considered QA a possibility.

            This is more like typical banking 'Agile'; release it and let customers do the QA.

            1. Rich 11 Silver badge

              Re: called it

              Beta permanence.

        2. sal II

          Re: called it

          Aaah DevOps... <3

        3. Blank Reg Silver badge

          Re: called it

          I can remember when banks would take 12-18 months validating new versions of an OS before they would deploy it in production and require a guarantee of at least 10 years of support for that particular version. Now they want to push out code in a few days? What could possibly go wrong?

      3. Bronek Kozicki

        Re: called it

        lynx rules

  4. IGotOut Silver badge

    You see Dev's

    This is why your IT guys and gals are "being obstructive" when they make sure you follow procedures. But hey don't bother with them, just use 'the cloud", far easier.

    1. iron Silver badge

      Re: You see Dev's

      You see Ops it's a shame the devs have to point out you can't handle a simple apostrophe. It makes you look stupid.

      Here's a hint, nothing belongs to the devs in your sentence so you don't need it.

      I'll leave your incorrect use of commas for another day.

    2. Law

      Re: You see Dev's

      "This is why your IT guys and gals are "being obstructive" when they make sure you follow procedures. But hey don't bother with them, just use 'the cloud", far easier."

      Whatever... where I work it's management and IT driving us into the cloud, because it'll solve all those pesky issues of IT having to maintain servers or deliver a usable internal network for our build infrastructure to run on. But hey, why not blame all the developers...

      1. Anonymous Coward
        Anonymous Coward

        Re: You see Dev's

        But hey, why not blame all the developers...

        Why not both? Just rebrand the team to 'DevOps' and let rip!

  5. JaseCoulls

    Sorry I'm late to the party...

    If anyone has any questions, I'll try to answer.

    1. Anonymous Coward
      Anonymous Coward

      Re: Sorry I'm late to the party...

      Why Donald Trump. And how the hell Brexit?

    2. Giovani Tapini
      Devil

      Re: Sorry I'm late to the party...

      Where did you find them? and why did you bring them to me?

    3. Kane Silver badge
      Boffin

      Re: Sorry I'm late to the party...

      Tea or Coffee?

    4. Tom Paine

      Re: Sorry I'm late to the party...

      The story says you previously discovered / disclosed some Scotia fail in 2017. Is it purely coincidence that you stumbled over stuff from the same bank twice, or do you keep a specific lookout because their record suggests you know there'll be another security snafu along before long?

      1. JaseCoulls

        Re: Sorry I'm late to the party...

        I keep a special eye on this bank. Originally, as a customer, I was concerned that they were putting me in danger. In 2016 I told friends and family to stop using their mobile products after I spotted the insult screen aimed at Kony Inc, hidden in the Android app (they only started Android java obfuscation in summer 2019). That told me right there that anyone could pull off an inside job, because if you can add an entire screen to the app and the bank doesn’t catch it, adding two lines to siphon bank credentials from the login process was going to be a walk in the park. That got me hooked on watching the stupid things they do. Until they disbanded the CCIRC, I would report the big things like that to them, practicing my observation skills along the way. Later, in 2018, Scotiabank and I got into a tussle where they showed a side of customer service that was reprehensible. Since then, I’ve upped the ante (including automation to keep tabs on stuff), and aim to document as much as possible to show orgs like OSFI and the Privacy Commission of Canada that these people don’t know what they’re doing.

        1. Tom Paine

          Re: Sorry I'm late to the party...

          Thanks for the excellent answer, very interesting stuff. I worked in the (enormous) security team at a well-known international megabank where money was no object, for a while. Any one of those incidents would have caused massive ructions, sudden resignations in management, reorgs, total revamps of SDL and testing processes, etc. Current berth (not a bank) has a bit more of a Scotia attitude (it seems) in that they're throwing money at half-baked attempts at agile and what they think security is* whilst carrying a gigantic tech and security debt load. In a way, it's just as interesting an experience... (After all, it's not _my_ firm. If that what they want to do...)

          *(Expensive blinkenlights and minimum wage analysts in a very, very cheap offshore location, mostly)

    5. Anonymous Cowtard
      Trollface

      Why is orange jam called marmalade?

      eh?

      1. BebopWeBop

        Re: Why is orange jam called marmalade?

        cos it contains peel

        1. The Oncoming Scorn Silver badge
          Alert

          Re: Why is orange jam called marmalade?

          I prefer shredless, is it still Marmalade then?

    6. BebopWeBop
      Pint

      Re: Sorry I'm late to the party...

      importantly, before I get into life, the universe and everything, where's the free beer?

  6. Anonymous Coward
    Anonymous Coward

    Hi grandma, may I use your fax machine?

    I have bad experiences with them and technology. Did you have to notify them via signed fax, courier, or registered mail?

    I would never attempt to configure critical services myself but, if I did, I would try hitting them with my cellphone to see if anything was world visible before loading up the data. (no, not your flip phone)

  7. Michael H.F. Wilkinson
    Facepalm

    Flabbergasting level of incompetence

    They might want to make a new Ig Nobel award category for this.

    Or perhaps we should have a Dr. Bunsen Honeydew Prize for Muppet-Grade Security Flaws

    1. Tom Paine
      Boffin

      Re: Flabbergasting level of incompetence

      Or a Professor Denzil Dexter Award for fail in the name of innovation?

    2. Not Elvis

      Re: Flabbergasting level of incompetence

      Perhaps a financial Darwin award is in order here?

    3. Robert Moore
      Happy

      Re: Flabbergasting level of incompetence

      we should have a Dr. Bunsen Honeydew Prize for Muppet-Grade Security Flaws

      This totally NEEDS so become a thing.

      1. Tom Paine

        Re: Flabbergasting level of incompetence

        https://pwnies.com/

  8. Aristotles slow and dimwitted horse Silver badge

    Whilst...

    Yes, whilst this is totally idiotic and obviously Scotiabanks problem to resolve, I do wonder if this is a fuckup caused by in-house resources or by the cheap and inexperienced offshore labour in India being so remote from the business that they don't properly understand the implications, or perhaps give a f**k.

    Either way, you do have to smirk (a bit)...

    1. JaseCoulls

      Re: Whilst...

      It's definitely gotten worse since many more devs were brought on in Chile, Peru, and Mexico.

  9. Groaning Ninny

    Totally off topic, another Canadian bank (ANC) keeps sending account info to one of my gmail addresses. Clearly there's no verification going on there. Additionally, I've tried to get them to stop, but any email I send seems to go unnoticed.

    Sadly, the account owner keeps going overdrawn, and it's not a lovely thing to see.

    1. Anonymous Coward
      Anonymous Coward

      Have you tried dropping the account owner a line? They have motivation to go properly ballistic and they are "authorised" to whinge about it.

      1. Prst. V.Jeltz Silver badge

        I doubt the email contains enough info to identify the owner .

  10. Steve Graham

    I did some development work for a UK bank back in the nineties. The restrictions, rules and regulations were strict, and were strictly enforced. After all, it was people's money that was at stake.

    Clearly. times have changed.

    1. andyL71

      Because, Agile.

      1. Kubla Cant Silver badge

        Because, Agile

        Crap QA has nothing to do with agile.

        There's nothing in agile development that says you have to be careless or deploy untested code. In a well-run project, nothing gets into development without a clear set of acceptance criteria that are subsequently used to validate the work.

        I don't recall any of the waterfall projects I used to work on having superior QA. The way waterfall works tends to mean that QA is crammed into whatever time is left over before the delivery deadline, and the components being tested are more complex.

        1. BebopWeBop

          Yeh, true, but then ISO9000 didn't say anything about quality - yet, people used it as shorthand for such.

        2. Anonymous Coward
          Anonymous Coward

          @Kubla Cant - Yeah, right!

          Security ? We've heard of it.

        3. cdegroot

          Indeed. Well before the term was coined, I worked on projects in the pharmaceutical industry, heavily regulated code (Good Clinical/Laboratory/Manufacturing Practice, FDA inspections, that kind of fun), and we could still be light on process and heavy on rapid learning and quick delivery; the acceptance criteria just specified lots of testing, including automation, and including printing out test reports that the project lead had to sign and file. I had similar experience building software for some banks.

          However, now that "agile software development" has turned into "Agile for Enterprise" and all sorts of nonsense (SAFe anyone?) - that sort of stuff just makes project managers fight different political battles and QA is still out the door. Quality is a mindset, not a process.

      2. Blank Reg Silver badge

        It sounds like they have embraced the "fail fast, fail often" mindset to the fullest.

    2. jsioui

      I worked for Scotia in security back in the 90's. All was good until IBM took over. It's all about the money (pun inteneded or not) which reminds me, have to sell some stock.

  11. DontFeedTheTrolls
    Headmaster

    Perhaps banks should be required to have "Accident" boards on their front webpage a bit like constructions sites on their fences.

    It has been 3 days since our last security breach

  12. Anonymous Coward
    Anonymous Coward

    Former Scotiabanker

    "By the time you read this, the GitHub repositories, which presumably were accidentally misconfigured by Scotiabank's techies, should be hidden or removed."

    As a former developer at Scotiabank, I can tell you they were not "accidentially misconfigured"; the proper adverb is "incompetently". There are good coders at Scotiabank. Too bad their entire day is spent trying to stem the tide of the legions of horrid typers (they aren't coders because a coder makes code, not just keystrokers).

  13. Anon Coward (there are nutters out there - I've worked with them)

    huh?

    what is all this GitHub billix about?

    I'm seeing it all over the shop.

  14. Anonymous Coward
    Anonymous Coward

    Security mindset from way back

    Back in the 80s, I accidentally left my Scotiabank credit card behind at a Business Depot (but did not realise it until a week later). The first fraudulent purchase was at a gas-station just down the road. Then came a string of fraudulent purchases at one store over the course of a few weeks. Scotiabank's grunt-and-poke security talked to me, asking if a relative or friend has visited before the theft. Security angle: my first name is a very common Anglo-Saxon name, the fradulent purchaser was a female, and I learnt that the person signing for the purchase need not be the card owner. (And no, they never bothered to visit the store in question.)

  15. mstreet

    What amazes me...

    Is that more than 12 hours after this story was posted, not a single Canadian news outlet seems to have picked up the story.

    Apparently our half-wit PM wearing black face at a party 20 years ago, is far more important...

    1. The Oncoming Scorn Silver badge
      Pint

      Re: What amazes me...

      Now you are just being kind about sock boy.

      Roll on October hopefully he can do a bollywood dance out of the door & I hope it hits his ass hard on the way out.

      1. This post has been deleted by a moderator

    2. JSIM

      Re: What amazes me...

      Yes, they're focusing on the breaking brownface news because they know people like you will lap it up. At least something on a slow news day and in a quiet election run-up, guaranteed to send Trudeau's haters into joyous convulsions. This minuscule "scandal", will be forgotten in a few days, and the 3 anti-Trudeau comentards here and their ilk will have left their usual dinners of nothing and nothing with an extra helping of nothing. Nothing to do except troll around IT forums spreading hate.

  16. Alistair
    Windows

    Excuse me ScotiaBank, would you mind informing us who it is you've outsourced your IT to lately?

    /walks down the hall whistling dixie.

    1. Anonymous Coward
      Anonymous Coward

      Can you make it up?

      You must have some good stories to tell of all the great projects you've been working on. ;)

  17. Barry Rueger

    Surprised? Hardly.

    It's only been a year since Scotia Bank finally relented and made passwords case sensitive and allowed special characters.

    Lately though they forced mobile customers to "upgrade" to a new improved app that lacks several heavily used features, and which is inarguably harder to use for everyday tasks.

    For instance, instead of tapping a button to confirm an action you now need to drag a slider across the screen.

  18. jsioui

    What the...

    I would ensure that the Cheif Risk Officer, CISO (who I knew well when I was there), CTO and the CEO are all held accountable for the dreadful manner in which they handle information assets. I'm almost ashamed to say I worked for Scotiabank. Makes you leary of how they handle other facets of the business. On another note their stock is up today. Incredible.

  19. LeahroyNake

    Muppet grade security

    Almost lost my drink out of my nose.

    Best headline ever lol

  20. queueback
    WTF?

    Not just incompetence, also a culture of unethical conduct in Scotiabank

    The story and comments are rightfully talking about incompetence. Keep in mind the Scotiabank’s security department, which is called Information Security and Control inside the bank, embodies underhand and unethical conduct. There is a branch in the security depart called Information Security Advisory Services that is supposed to do security risk assessments of all Scotiabank technology products and services as a gatekeeper function. Unfortunately their Security Advisory is marred not just by poor competence, but also because of falsified assessments and risk profiling.

    This Security Advisory group is well groomed to make any serious security gaps and risk impact findings, in their so-called assessment reports, as low impact or low risk (no matter what the real risk is). If you read security risk assessment documents of Scotiabank, most, if not all, threats from their services are classified as low or inconsequential impact.

    So, the incident or breach like this one should not come as a surprise. If you don’t believe me ask Scotiabank CISO to make available reports called TRA (threat and risk assessment) for their customer, Internet facing services.

  21. Lion

    A security breach, even when tens of millions have been exposed to possible exploitation, seems to not move the shock meter a single click any more. The known damage and the potential damage is treated as a mere notification. We have been numbed of our senses.

    Canadians think highly of their banks, so I am not expecting front page headlines giving Scotiabank the WTF treatment or a run on the bank by its business clients and private account holders. The corporate suits are counting on this and they will spew out the usual PR to cover it. This complacency, if not checked now, will end in a world of hurt for the bank's customers. If the leaked data has been collected, it will be appraised and sold to miscreants, i.e. organised crime and rogue states. The grim reality is that the bank will probably escape with a small fine by the regulators. They can also keep lawsuits in the courts for decades. They accept responsibility for causing the leak, but they are never held absolutely accountable.

    1. raving angry loony

      The same people who own shares in the banks also own and control 99% of our "free" press in this country. So no, I don't expect ANY public notice to happen in said publications. As it wasn't client information that was leaked, which would be covered by legislation, "muppet grade" security (and that's an insult to muppets) is not covered by legislation. No crime, no news, no mention. Move along, nothing to see here, everything is fine.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021