back to article Software devs' new mantra: Zen dogs dream of small-sized bones

One of the primary principles of DevOps is moving from large software releases to a series of small batches. What do we mean by “large”? Six-to-12-month (or longer) projects that follow the infamous “water-scrum-fall” model. While development teams may create builds weekly, the code isn’t deployed to production and used by …

  1. tiggity Silver badge


    Some customers like to do their own exhaustive QA testing on software & do not want once a week (or more frequent) updates to existing systems in "live" use, they want infrequent releases so they can test it for as long as they want, feel confident nothing is broken and then have it deployed "live".

    The more "mission critical" an existing piece of software is seen to be, in general the less likely that customer want ultra frequent updates to their "live" system.

    However, plenty of customers that want infrequent "live" updates happy to see feature test builds frequently updated so they can play around with new features in their test environment and feedback on how the software is proceeding.

    So you can have scenarios of "live deploys" on an infrequent basis, but testbed evaluation deploys frequently.

    Just don't expect all (or even many) customers to be happy with daily "live" updates.

    Article needs to emphasise difference between "live" and "demo" / "testbed" systems IMHO as often very different update rates for sensible reasons (once a piece of software is relatively mature and in mission critical use then caution tends to prevail on upgrades)

  2. Anonymous Coward
    Anonymous Coward

    too fast for accurate bug tracking

    "deploying a lot less code each time, hopefully just a handful of changes, or even just one"

    Once you start iterating changes faster than QA can test and report on them, a whole world of versioning pain opens up. We already have to waste ridiculous amounts of time confirming which revision each bug report actually applies to. Start pushing such fine grained updates so quickly and that can only get worse.

    1. allthecoolshortnamesweretaken

      Re: too fast for accurate bug tracking

      "deploying a lot less code each time, hopefully just a handful of changes, or even just one"

      Or even just none. Once the customer gets used to his "daily" update, he'll be expecting them. No bugs to fix, no code to tighten? Yust change the colour of that button...

      In other news: another day, another DevOps plug.

    2. Lysenko

      faster than QA can test and report on them...

      The "point" here is to shove QA down a couple of tiers.

      You run your code through some unit tests and if it passes you deploy and wait for the beta testers end users to find and report the fault.

      This shouldn't interrupt your continuous delivery (of more faults) of course so your new metric is apparently MTTR (Mean Time to Repair) rather than MTBF, which is in line with the SJFL (Ship Junk Fix Later) overall strategic vision.

    3. Anonymous Coward
      Anonymous Coward

      Re: too fast for accurate bug tracking

      "Once you start iterating changes faster than QA can test and report on them, a whole world of versioning pain opens up."

      If you're not automating your regression tests, you're already doing it very, very wrong. Once a new feature and it's manual test pack passes QA for initial production release, it should be immediately automated as in theory it's stabilised. QAs should be focusing entirely on the main source of instability in the system which is the new/changed code. Also making QA sign-off an exit criteria for an iteration does wonders for making sure the process works.

    4. Anonymous Coward
      Anonymous Coward

      Re: too fast for accurate bug tracking

      What do you mean testing? Devs are never wrong.

  3. JacobZ

    Cloud vs on prem

    A lot of what you describe here is goodness, but I think it's important to add: It's orders of magnitude more realistic to aspire to this if you are running a cloud-delivered business:

    1. You only have one image to manage, and you control it. You don't have to rely on customers consuming and deploying your updates.

    2. You can do partial roll-outs, e.g. with feature flags, to progressively test whether (a) something works and (b) makes the users' lives better...

    3 ... and if it doesn't, you can [more] easily roll it back.

    And that's not just for consumer stuff. Look at for an example.

  4. Adelio Silver badge

    Change management

    Yep, many projects (irrelevant of size) make no sense to deploy to live BEFORE they are complete )how would you deploy half a customer form) . But lots of interim builds with testing and customer feedback make a lot of sense. Customers get to see how the work is going and provide timley feedback.

    Once a project is deployed to live then small changes are usually OK.

  5. The Mole

    Shock News

    “We did an analysis of hundreds of projects over a multi-year period. The ones that delivered in less than a quarter succeeded about 80 per cent of the time, while the ones that lasted more than a year failed at about the same rate.”

    So the simple projects (note no comment about release schedules) are much more likely to succeed than more complex projects. In other news...

  6. happy but not clappy

    Yes this, but...

    most of the use-cases for devops to live are only for systems where a bug in live is a low risk. Furthermore live devops to stateful systems is ridiculously immature.

    One poster suggested devops to pre-live, and that's good, though unpicking one change from a bunch is difficult.

    The key here is what is called "QA", though really they are talking about QC i.e. testing. This is the hurty point now, not the technology, and is why the business are nervous. They want good arse-coverage, and this doesn't give it to them, yet.

  7. Anonymous Coward
    Thumb Up

    Spot on IMHO

    All the points and numbers in this article concur with my experience in freelance team webdev work. 1-3 month projects seem to work best (aim for 1, expect 2-3), along with occasional small updates/fixes on live sites.

    The longer a project goes on, the more time it takes to keep track of everything up in the air - diverging codebases, db schemas, artwork, server config changes... which leads to more friction and bugs at deployment time. We forget things. People change jobs. Other projects push long-running projects onto the back burner. That's more than enough to erase any gains to be had by queueing up changes.

  8. Anonymous Coward
    Anonymous Coward

    Risk Management

    AC for the normal reasons.

    The author identifies some "risks," and I have some thoughts about them:

    1. "Bug Swarms" - There are other good ways to manage this risk. Unit testing and proper QA procedures come to mind. It's important to realize that in DevOps, QA is just as rushed as development, and simply can't cope with insidious bugs at all, let alone multiple bugs at once.

    2. "Useless Software" - So bug-riddled, poorly functioning software is useful in some respect? However, let's limit our discussion to some imaginary, properly functioning but unwanted feature of a software package. In cloud-cuckooland, users effectively convey their dissatisfaction to the developers, who immediately re-imagine the feature, push it to production, and it works so well that users carve marble statues of the developers in the foyer during all the time they save. Now, over to the real world, where the developers never imagined the feature. Management imagined it, with the help of a tea-leaf diviner called a "usability consultant;" its inception involved focus groups, and a great deal of money. The UX team hears rumblings of dissatisfaction, but since they're privy to the development process, they conceal this for a while, hoping it'll go away, because they're the messengers that will be shot by the manager championing the feature. After a month or so, the complaints keep coming in, and UX informs the manager, who finds out this code now has multiple dependencies because the whole program is constructed like a painting made by throwing buckets at a canvas while standing in jet engine exhaust, and the crappy feature remains indefinitely, or at least, for a very long time.

    3. "Stymied Innovation" - My experience is that users expect consistency in design rules, a modicum of sense applied to the organization of features, and software that works. Then, once they have it, they desperately hope that nobody will seek to "delight" them by screwing with it. Users are especially frustrated when those delightful innovations are not uniformly applied across the UI. A recent example of this is Windows 10 Mobile.

    4. "Budget Overruns" - OK. Supposing I have 1000 units of currency to budget for the month, and I decide the following: I will place this money in a pool, and when I decide that I want something, I will create a new budget for exactly the cost of the item I want. That way, I'm more likely to decide to spend it! I will try to avoid multi-purpose items, preferring to purchase multiple single-purpose items that are less expensive per unit. Each time I go to the store, I will purchase only a few items at a time, to ensure that, when I use them, it's easy to figure out which ones are not satisfactory. When I make a poor purchasing decision, I don't worry too much, because it's a learning experience!

    5. "Schedule Elongation" - It makes sense that this would be a problem. After all, if I can't construct a budget and adhere to it, how could I possibly construct a schedule and adhere to that? And, you know, there's always "more hardening to do." Like ripping your hardcoded AWS keys out of the application binary. Until DevOps happened, I couldn't even imagine something so absurd.

  9. JLV

    "Joe, how's your paycheck?"

    "Off by $500"

    "Me too. But that's alright. Payroll told me it's IT's ongoing feature experimentation process. It'll be fixed next week "

    "They know what they're doing"

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2022