back to article Big, fat fail? Here's how to avoid that: Microservices and you

Sure, we've all heard about "microservices" but just what use are they and why would you want them? How do you even start designing microservices? Most applications built more than a couple a years ago would more than likely use classic application design methodology that is monolithic in nature, i.e. everything is bundled …

  1. Warm Braw Silver badge

    Every component of the application (should) be retested

    I'm not clear to me how changing an interface from a subroutine call to a RESTful API magically removes the requirement to peform integration testing when a component is altered. Wishing doesn't make it so.

    1. Anonymous Coward
      Anonymous Coward

      Re: a RESTful API magically removes the requirement to peform integration testing

      It doesn't. As with much of current IT development best practice this is geared towards stopping bad engineers doing terrible things. In theory it stops people breaking the whole application because they only break a single microservice. How much damage that does is down to how many other bits are using that service, and that's where integration testing comes into it's own.

      In all the projects I've worked on in the last few years the unit tests have been a pain in the arse, only ever failing when behaviour changes and the tests aren't updated. The real work, both testing and development, is in integration. That's not to say that unit tests are useless (though in my experience their usefulness is limited) but that all they really tell you is whether the system's ready for proper testing or not....

    2. Lysenko

      Re: Every component of the application (should) be retested

      It doesn't. There is no difference between a REST/JSON API and a process dumping CSV files to disk for another process to read. A decade ago we would have been listening to the same guff about SOAP/XML.

      A shell script is bunch of interacting "micro services". This is just the latest attempt to spray fairy dust on the mundane and get the PHBs swooning.

      1. Anonymous Coward
        Anonymous Coward

        Re: Every component of the application (should) be retested

        Lysenko nailed it.

        Just to add, if you need to do stuff, it doesn't matter how you break that up into sub-stuff, it still has to plug together as a 'system of systems'. So a big app needs sub-unit testing and integration testing. Micro-services need testing and the integrated whole needs testing.

        The real bummer is if you need something slightly different from the micro-service. It takes ages to bash an API into a usable shape.

        And we had this discussion ages ago regarding how it was 'bad' to do linux as a monolithic kernel and it should be micro-services...

        1. gumbril

          Re: Every component of the application (should) be retested

          > The real bummer is if you need something slightly different from the micro-service. It takes ages to bash an API into a usable shape.

          This. Completely. It's the secret sauce.

      2. Grikath

        Re: Every component of the application (should) be retested

        Lysenko indeed has it.

        Ah well.. at least we know the next reason to gently detach the Cattleprod from it's comfy wall mount..

      3. LDS Silver badge

        A decade ago we would have been listening to the same guff about SOAP/XML.

        And some time earlier about CORBA, DCOM and DCE/RPC.... the latter at least was a far better RPC method than the ugly REST calls over HTTP...

        And before we had "microkernels" - same philosophy, break a monolithic kernel into microservices...

        Moreover the complexity of a lot of APIs and messaging among services is not inferior to the one of a monolithic application, and poses additional issues when large data needs to be moved and/or shared across "services" possibily running on different nodes - whoever tried to build a true multi-tier application had to cope with that.

        1. HmmmYes

          Re: A decade ago we would have been listening to the same guff about SOAP/XML.

          CORBRA..... shiver .... interworking ..... language bindings ...... wierd state .....

        2. sed gawk Silver badge

          Re: A decade ago we would have been listening to the same guff about SOAP/XML.

          1) REST is not RPC. Do RPC explicitly in another manner, e.g. using plain TCP or perhaps apache thrift.

          2) HTTP is pretty rubbish for RPC but REST is fairly useful for writing "generic" api clients/services.

          It doesn't really matter what service the client/service pair does, if it's REST, I can assume a lot about my services, and that layer of base assumption makes layering API semantics over REST a win.

          If all my api clients specify the "Accept:application-v1/json" header when negotiating an representation of a resource, I get clients which are not going to break when I upgrade my services, and I can upgrade each endpoint at a time.

          That's worth restructuring your service interfaces away from RPC if possible, not silver bullet, but surprisingly useful.

          1. Anonymous Coward
            Anonymous Coward

            Re: A decade ago we would have been listening to the same guff about SOAP/XML.

            REST *is* a form of RPC. I understand RPC got a bad name - but it just mean "Remote Procedure Call" - and a "procedure" is an API or "service" - just those terms maybe was not yet fashionable back then.

            REST does exactly what RPC does, in the clumsy web way hammering everything into HTTP, URLs and strings - the only things web developers are able to understand.

            Generic "APIs" are exactly what DCE/RPC, CORBA, DCOM and SOAP implemented before REST. Maybe in a more complex (and performant) way beyond most web developers knowledge.

            Sure, maybe REST somewhat abstract more - but not always its a good thing - more layer of abstractions means more processing power wasted into turning data to and from representations CPU can actually handle (take that, scalability...) - string exchanges are good for humans, not computers. Clueless developers without a good debugger like them, though, easier to print out...

            REST is strongly designed around the HTTP communication model, which unluckily is a size that doesn't fit all. There's a reason why HTTP 2.0 is so different from 1.x, because it was a protocol not designed for what it became used - and just because too many new "web" developers hadn't a clue about distributed systems development (websockets are the pinnacle of such sillyness).

            1. sed gawk Silver badge

              Re: A decade ago we would have been listening to the same guff about SOAP/XML.

              REST is an architectural composition of services.

              The idea is simply to overlay one interactions over an application layer protocol, to avoid explicitly writing business logic when it can be defined in terms of the protocol.

              RPC is just marshalling -> addressing -> unmarshalling -> processing -> marshalling.

              You can do RPC over HTTP but you really shouldn't.

              These do the same thing but in the RPC example I get to write all the code and the logic, in the REST example, it comes for free by virtue of layering my application concern "Chat rooms live on separate nodes" over the redirect status codes offered by HTTP, and make sure that my identifiers are URI.

              This adds a side benefit in that in the REST case, redirects are cachable so my application is more network friendly.

              Example: Chat rooms. (RPC)

              (C) >> (S) "Find Room1"

              (C) << (S) "Room1 is at 1.2.3.4"

              (C) >> (Room1) Join

              (C) << (Room1) Welcome to Room1

              Example: Chat rooms. (REST)

              (C) >> (S)

              GET /rooms/room1 HTTP/1.0

              (C) << (S)

              301 http://1.2.3.4/room1 HTTP/1.0

              (C) >> (S)

              GET http://1.2.3.4/room1 HTTP/1.0

              (C) << (Room1)

              200 OK

              Welcome to Room1

      4. KitD

        Re: Every component of the application (should) be retested

        "A shell script is bunch of interacting "micro services". This is just the latest attempt to spray fairy dust on the mundane and get the PHBs swooning."

        Well, except that your shell script was (probably) only written by one person.

        The principal benefit of microservices IME is that it fits naturally with Conway's Law [1] by design, which is generally A Good Thing. Not saying it is all sweetness and light, and yes, conceptually it is very similar to the long list of modularisation techniques that have gone before, but there are certain key technologies in the limelight now (eg Docker, REST, CD) that make it worth pursuing IMHO.

        [1] - https://en.wikipedia.org/wiki/Conway%27s_law

        1. ahmanwhathandle

          TDD should save the day though...

          If a microservice is built well, to genuinely do *one* small job then it should lend itself to easy testing, programmatically - TDD / unit tests etc.

          If devs make changes to their microservices they should have tests to run that prove the outputs of that service are the same for given input(s) etc.

          Therefore it follows that full integration testing really isn't needed (I agree in theory) to release a new service version into a wider system.

          The services just need to be small enough.

          Would one retest an entire operating system when a new version of ping or ls was built?

          1. This post has been deleted by its author

          2. Anonymous Coward
            Anonymous Coward

            Re: TDD should save the day though...

            "Therefore it follows that full integration testing really isn't needed (I agree in theory) to release a new service version into a wider system."

            There's a whole historic world of pain that disagrees with this. If you make changes to microservices you've changed it. Somehow. You can prove outputs are the same for given inputs when things are in a test state but no more than that.

            And it needs a lot more than that - unintended consequences...

            "Who'd've thought that minor change would **** it up" will be on your mind when applying for a new job.

        2. Anonymous Coward
          Anonymous Coward

          Re: Every component of the application (should) be retested

          Hmmm, anyone who's ever been vaguely involved in crowbarring an ERP into a company may disagree.

          You've misunderstood the shell script analogy, the shell may be written by one person but the micro-services were not.

          Every few years old/new/old stuff is trotted out, be it 'the cloud', neural network analytics, yet another language to save the world and microservices. Which without a coherent data model ends up as stovepiped nonsense.

          A business is not like a lot of distributed microservices. It may look like it on the surface, but its not.

    3. Steve Davies 3 Silver badge

      Re: Every component of the application (should) be retested

      And how does it run compared to the Subroutine Call?

      with Vendors like Oracle etc charging arms, legs amd everything else on a Per CPU license all this SOA/Restful/Crapware modularisation costs CPU Cycles and may need the organisation to spend lots more $$$$/£££££/Magic money on more CPU licenses just to get the same performance (if you are lucky) than before.

      Sure you might possibley/probably/are you joking (delete as appropriate) some money in support but that could easily be outweighed by a new Oracle/SQLServer/Database of your choice license.

      And you still may never get the performance back to what it was before the 'DevOps' magic sauce was taken.

  2. Ru'

    TL;DMDO*;DR

    *didn't mention DevOps

    1. LDS Silver badge

      Yes, it did mention DevOps...

      "Such compartmentalisation lends itself to continuous development of different components of the applications (services), with each service continuously developed and rolled into new releases."

  3. This post has been deleted by its author

  4. SVV

    There is no silver bullet

    Please can all tech journalists re-read this chapter before writing overenthusiastic articles such as this one, hailing the latest in a long long line of near magical solutions that will solve the issues with testing and sytem integration that have existed since the dawn of IT.

    I agree totally with the previous posters, who obviously all have experience that leads them to understand that microserevices don't negate the need for full system testing any more than previous technologies such as Web Services, CORBA, etc did.

    Distributd architectures always require at least one instance of each system that is used to implement them. At the most, microservices with really good manaqgement and scheduling tools look to be a good way of managing varying load demands at different times by enabling computational power to be used more efficienlly by scaling the number of instances running to handle those loads up or down more easily.

    Service Oriented Architectures do work well in my experience, but making everything stateless isn't a solution when you need transactional integrity, as all systems writing data to a database do.

    1. Valerion

      Re: There is no silver bullet

      Service Oriented Architectures do work well in my experience, but making everything stateless isn't a solution when you need transactional integrity, as all systems writing data to a database do.

      This is indeed the problem. The other trend I'm seeing now is each microservice to store data pertaining to that service in its own database. Which is fine, until you need to do something transactionally across multiple services, i.e. create an order and update the user's account at the same time.

      So what ends up happening is someone writes an orchestration layer that wraps up calls to those individual services in a transaction-friendly way, but by then you're pretty much back to what you started with 5 years ago.

      Overall though I do like the idea of microservices, especially the easy-release cycle that it allows. But - as every single poster has correctly pointed out - it does NOT negate the need for proper integration testing!

    2. Steve Davies 3 Silver badge

      Re: There is no silver bullet

      Said Journo's might learn something from going over back issues of Computing and Computer Weekly for the artivles on another bit of snake oil from the 1970's called

      'the Last One'.

      Where is that today eh?

      The hype over it was huge (in journo land that is).

    3. Anonymous Coward
      Anonymous Coward

      Re: There is no silver bullet

      My TL;DR interpretation of the article(s) is: Same Shit, Different Decade.

  5. Anonymous Coward
    Anonymous Coward

    I'm starting to feel like Jimmy in the recent south park episodes.

    "Ad" "Ad" "Ad" "News" "Ad"

  6. Black Road Dude

    Its about testing and scaling.

    Docker which is what this article is really about helps with scalability but does demand you architect your systems differently so they can be multiple independent services.

    Docker also means we can automate integration testing which is what makes the whole thing mean anything. The micro service architecture on its own has some value but until people understand the value it can bring to testing its just going to look like SOA all over again.

  7. Anonymous Coward
    Anonymous Coward

    *nixes do a lot of single things...

    ... just I still have to find one they do well... especially since their true philosphy is stll that of chained batch files on systems that could barely run a task at a time for each of the few users.

  8. Anonymous Coward
    Anonymous Coward

    Microservices = premature optimisation

    Anyone thinking of getting into microservices should read this article by Martin Fowler: http://martinfowler.com/bliki/MonolithFirst.html

    Basically - start with a monolith and break it up when you need to.

    (Post anonymously 'cause we've got a microservices system that is a bugger to work with.)

    1. Someone_Somewhere

      Re: Microservices = premature optimisation

      You said 'premature' and followed it with a word ending in 'ation' (fnarr, fnarr)

  9. breakfast

    As others have said, the problem is not that there are no ways to create working distributed architectures, the problem is that distributed programming is hard and whatever architecture you use will require you to solve some very tough problems. Splitting a service into decoupled microservices is not a terrible idea if they are truly decoupled, but most applications don't work that way in real life, so we tend to end up having semi-coupled complex systems that don't do what you expect when you expect as well as opportunities for the creation of endless race conditions that you never thought of during development.

    If it was easy, anyone could do it.

  10. DCLXV

    I enjoy distributed solutions, though my refusal to use software by Oracle has quite a lot to do with that.

  11. Someone_Somewhere

    It's a whole new paradigm, I tell you!

    And I'm going to start remodelling my life around it!!

    Instead of calling Interflora and getting them to send some flowers to my wife (who's in Mexico City, teaching a course) for her birthday, I'll call 118118 and get the number of an English speaking local telephone operator to get the number of an English speaking local business directory, so that I can contact a local (English speaking) courier service to collect them from the local florist I will speak to* and deliver them to the university campus .

    I'm not sure whom I'll need to call in order to have a card with a birthday message added to the mix, but I'm sure it'll be easily sorted out by calling 118118 again and (eventually) getting the number of an English speaking birthday card manufacturer, getting it collected by courier, delivered to a card wrting service, collected by courier and delivered to the campus.

    I'll just have to cross my fingers and hope that both the flowers and the card arrive in the same place at the same time - or perhaps there's some service provider over there that will organise that for me too.

    I don't know how Interflora do it now, but they're a monolithic service provider, so that has to be bad, right? It'd be much better to make use of the microservices supplied by all the different providers instead.

    * After I've called 118118 again and got the number for the local telephone directory service that will provide me with their number.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2021