back to article Two-day Bitbucket borkage has devs tearing their hair out

Atlassian's code version control system Bitbucket has been suffering an ongoing failure in its storage layer, affecting developers across the globe. Problems arose yesterday (January 9) at 13:47 UTC, when the service website performance and SSH/HTTPS transactions slowed. After four hours of degraded performance, Bitbucket said …

  1. John Sturdy
    Coffee/keyboard

    There's always batbucket! https://batbucket.org/

    1. Anonymous Coward
      Anonymous Coward

      correcthorsebatterystaple

  2. wolfetone Silver badge

    "@bitbucket is really making our lives hell today. And before you bitch, yes, we pay for it."

    That's your own fault for giving a 3rd party the responsibility for handling something you really should keep in house, and it's your own fault for not having a back up plan in place for such an event.

    1. Brewster's Angle Grinder Silver badge
      Pint

      Pub o'clock

      Maybe the plan is to let the devs twiddle their fingers: the cost of them playing doom for a few hours being cheaper than the cost of employing people to manage an in-house system 24/7 (with the attendant risk that an outage might still leave devs playing doom).

      I'd want it all in house myself. But I know a old-time controlfreakosaurus.

      1. Anonymous Coward
        Anonymous Coward

        Re: Pub o'clock

        In-house systems tend usually to be simpler because they don't have to support millions of devs, which makes them more stable and easier to fix. Also, if your system fails, mine usually keeps on working, being separated.

        Decentralization had its benefits, it looks now they are forgotten, and to save a few dimes everybody is willingly to puts all of his eggs in someone else's basket.

        It's good that for security reason we are forbidden to store company IP outside the company systems... so our manglement can't awake with bright ideas...

  3. cambsukguy

    Who is the Vendor?

    "We are in the process of fixing the overall issue with our vendor, but cannot currently give an accurate eta for when this will be completed."

    AWS, Googe, Azure, some other lot?

    I assumed they owned and operated their own servers/storage.

    1. foxyshadis

      Re: Who is the Vendor?

      You've never heard of Dell EMC, NetApp, Nexenta, IBM, HPE, Pure Storage, etc? Even when you own and operate all of your own gear, you still have a vendor that you occasionally make a panicked call to.

  4. Anonymous Coward
    Anonymous Coward

    Meltdown?

    Have they applied patches and seeing the effects of performance hit on systems that were previously running a bit to close to the limit for comfort?

    Quite a few systems I am aware of that were ticking over, just about getting by on only had about 20-30% "headroom" are suffering squeaky bum time after patching and seeing performance nose dive on I/O intensive stuff (hence AC!)

    Or is it unrelated coincidence?

    If its unrelated, they should have just seized the opportunity and cynically blamed the patches anyway, isn't that sleaziness what PR people get paid for?

  5. Anonymous Coward
    Anonymous Coward

    This is a external hosting story

    nothing to do with Bitbucket.

    We use Bitbucket Server and host it ourselves, and it's a superb product and very reliable.

  6. Justin Clift

    Hmmm.....

    Sounds like a raid rebuild in progress. Not sure what storage tech they're using underneath, but it's probably something along the lines of it.

    Though, more likely the distributed version of it.

  7. Anonymous Coward
    Anonymous Coward

    Another win for Agile and Millennial developers!

    All development stops for Millennials when bitbucket goes down. Older developers just don't notice - carry on being productive as usual! You know - getting stuff done. Millennial developers oil their beards and tend to their man buns whilst moaning on twitter.

  8. MrKrotos

    N000bz :P

    "@bitbucket is really making our lives hell today. And before you bitch, yes, we pay for it."

    So you dont have a local server to fall back on if say..... Interwebs goes down?

    Yeah good luck with that, 1 point of failure anyone?

  9. Apprentice of Tokenism

    Did I miss the bottom line?

    What's the whining and moaning about? It is git after all.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like