back to article Containers have security problems and flexibility issues. VMs will make them viable

Welcome to the latest Register Debate in which writers discuss technology topics, and you – the reader – choose the winning argument. The format is simple: we propose a motion, the arguments for the motion will run this Monday and Wednesday, and the arguments against on Tuesday and Thursday. During the week you can cast your …

  1. ciaran

    Vi or emacs?

    This sounds like the eternal vi-emacs argument.

    Personally I'm vi - simple, available everywhere. Like VMs.

    One thing I particularly don't like about containers is the relative lack of storage or persistence.

    However nothing I'm working on needs me to do the cost-benifit analysis. VMs let anyone administer the solution, its a known quantity, good enough for me.

    1. Ben Tasker Silver badge

      Re: Vi or emacs?

      We always took the approach that using containers (well, specifically Docker) was fine *if* you could provide a sane justification for it.

      It makes deployment much easier, sure - but in a network where deployments are relatively rare, that's outweighed by having to get the support team/ops comfortable with managing and troubleshooting Docker.

      The result is that (in terms of projects) VMs have tended to be more common than containers.

  2. raesene

    Containers vs VMs - probably a false dichotomy

    For me containers make most sense when defined as a way to package and distribute software irrespective of the isolation mechanism used. If you deploy containers on Google Cloud Run they're isolated using a gVisor sandbox, if you deploy on ECS Fargate, it uses a micro-VM with Fargate.

    So it's not one or the other, it's getting the right isolation mechanism for the deployment. I'd say that, if you're doing something like multi-tenant Kubernetes clusters, you want a VM/Sandbox isolation setup, rather than runc based things like ContainerD and CRI-O as the attack surface of a shared Linux kernel is pretty big and there's a steady stream of vulnerabilities to try and exploit.

    However if you've got a set of relatively trusted applications, runc based isolation may be fine and it's going to be lower overhead than using a Sandbox or MicroVM

    1. Anonymous Coward
      Anonymous Coward

      Re: Containers vs VMs - probably a false dichotomy

      This is the right sort of mindset. Your comments regarding the typical isolation seen when actually running the containers is of course paramount and are some of the only comments actually discussing technical detail and how it could be done. It is also worth mentioning that the two cases you covered are some of the most popular ones - the majority of people deploying on the cloud are using one of these two. Alternately, assuming plain Kubernetes, perhaps a micro-VM-per-tennant (per Kubernetes node) is best within which the tenant's containers execute - preferably including gVisor/an equivalent to provide tighter cap/syscall controls than what Kubernetes currently offers.

      Of course this can be taken up another level - where do the micro-VM's execute? Can you smash the walls on the micro-VM? Should multiple tenants live inside the same CPU ring? What happens if another meltdown-style attack surfaces? Whilst everyone says "security is paramount" the reality is that absolutely guaranteed security is very hard (and usually comes with heavy performance penalties) and a combination of the above probably gets you a good balance.

    2. teknopaul Silver badge

      Re: Containers vs VMs - probably a false dichotomy

      I just discovered that the whole of /proc is accessible between containers. Some is masked but its still reachable.

      Containers are handy and undeniably a useful security layer but I don't think its safe to rely on the isolation of a container in the way you can with VMs.

  3. This post has been deleted by its author

  4. steelpillow Silver badge
    Trollface

    The OS as an app

    Early VM hypervisors needed the hardware to be fairly close to the VM's expectations, later ones play fast and loose with same. Early containers struggled to share every last OS feature, later ones deliver the goods. Consider how much crapware gets hosted by the average web browser these days - web apps and all kinds of ess-aitch-one-tee. People have even tries to make the browser to OS (or is that make the OS the browser?). In the end, VM and containers are just named stopovers on a continuum from multi-instance mainframes to nesting one app inside another. Both will find their niches. Both will get periodically compromised, fixed and compromised again, playing marketing leapfrog ad nauseam. This is one pointless debate.

    1. diodesign (Written by Reg staff) Silver badge

      "This is one pointless debate"

      So pointless you contributed to it -- thanks!

      C.

      1. Anonymous Coward
        Anonymous Coward

        Re: "This is one pointless debate"

        "...The format is simple: we propose a motion..."

        Where? And who's the running politician?

        Speaking of pointless, pointless is asking for a vote on something when nothing definitive is in question. Maybe it's hidden in there somewhere to be abstracted and also in an ambiguous way, exactly like government politics but, I've read the article twice and there's either nothing or many things to vote on.

        I vote Swiss Cheese.

        1. katrinab Silver badge
          Paris Hilton

          Re: "This is one pointless debate"

          You are voting on whether or not you agree with the motion that containers will kill virtual machines.

          It is not explained whether or not this means that running Docker inside or indeed outside VMWare / Hyper-V / etc will kill the vm instance; or whether it means that the existance of Docker means you will no longer need VMWare / Hyper-V etc.

          But neither is true, so I voted against the motion.

          1. Michael Wojcik Silver badge

            Re: "This is one pointless debate"

            Having the voting links labeled "For" and "Against" doesn't help, particularly when the canonical question isn't restated in the voting box. I scrolled up to the top of the article to double-check the phrasing of the question before I voted. (No points for guessing how; either you've read my posts in the forum for TPM's piece, or you haven't.)

  5. Claptrap314 Silver badge

    Regarding the arguments in this article

    Color me really, really unimpressed. He makes a lot of hay about security matters, but check the CVSs against, oh, VSphere and get back to me. I agree that container technology currently has a higher bug flow rate, but has the author admits, VMs have been around since before I was born, and they remain a steady source of security fails. This is like comparing GCP outages their first year to AWS outages that same calendar year.

    Step up on level in the arguments, he more-or-less admits that VMs are becoming more container-like in order to stay relevant. If that's not admission that containers have already effectively won, I don't know what is.

    Having said all that, I'm reminded about what happened when Apple went from the PowerPC to Intel. This move completely destroyed their business model, and I observed, "Apple Computer is dead". A year later, the iPhone came out, and Apple changed it's name from Apple Computer to Apple. The company did not die, but it had to invent a market & rebrand itself to survive. I call that a vindicated observation.

    I've not played around enough with VMs, or, really, containers to make any strong judgements. But if our understanding of what a VM is and what we expect of it significantly changes, this it is certainly true that "VM"s, as they are currently known, have been eliminated. This article looks to me to be an admission that this is probably the case.

  6. Mike_in_the_house

    It’s all about operational Costs.

    Well, this statement of a custom kernel was un impressed, and I find it really useless the need of a customer kernel on a VM to append the app. “When was the last time you have a custom kernel needed to install an app?” Windows ? Well, if you consider the said mention of boot times, one of the key values of container is that fail over of a containerized app happens at the Aplication layer, so the RTO of that said app to fail over from one HW to another is a substantial benefit vs VMs. I’d like to see some figures on the Bare Metal OS with Containers (say MariaDD, etc, etc) vs a Hypervisor OS + Guest OS +App. How much performance can you hardness out off the same hardware platform vs the other, this is the key value of the Container platform. Now set it up to fail over to another hardware, the RTO are in the 2-4 sec vs minutes… the Container wins IMHOP.

  7. Robert Grant Silver badge

    Most of this is obviated by the fact that with things like K8s, you run containers on VMs (where else?) and you can specify certain containers to run or not run on certain sets of VMs if you want to separate them.

    This industry standard already is a better best of both worlds than the article's conclusion - as long as a node can run kubelet, it can participate, which allows security separation where needed, and different architectures to be brought in.

    1. raesene

      With Kubernetes (and without add-ons like OPA/Kyverno) scheduling isn't really a security barrier. If the attacker can schedule pods, they can just bypass the scheduler altogether (specify nodeName in the manifest, and tolerate any taints that might be present on the node)

      For general breakout, it often depends on what else is deployed in the cluster and what rights the service accounts have. One of the big problems with k8s security and that setup is that if you deploy things with rights to the k8s API into the cluster, any attacker who can break out onto the underlying node can go looking for service account tokens which belong to any other container scheduled to that node.

      There's the option of using different runtime classes and deploying some containers into VMs (e.g. kata/firecracker) but I've not seen that used, that often.

      1. Robert Grant Silver badge

        Sure - I was more referring to the idea that if you want to stop one pod's breach from affecting other pods, you can do that by making sure that the two pod types don't run on the same VM.

        Hacking k8s itself is a different ballgame; more comparable to hacking the Hypervisor or its management software.

      2. Anonymous Coward
        Anonymous Coward

        I am pretty sure if you abuse your VM deployment enough, you will get similar security outcomes as allowing your containers to run as root or have admin access to the K8S APIs. This kind of argumentation is just naive.

  8. YTC#1

    Missing "container" option

    The article fails to mention Solaris Zones, which are essentially containers. These allow branded versions whereby older versions of Solaris can be run (ie S10 on an S11 system) and Kernel Zones that allow other levels of S11 or different patch levels to be run within a S11 instance (and then allow zones within that KZ).

    With ZFS datasets delegated the zone gets a high level of control over storage.

    If you don't like Oracle, then there are other options available via the illumos https://www.illumos.org/ (opensolaris) route.

    1. katrinab Silver badge
      Happy

      Re: Missing "container" option

      Or FreeBSD jails.

      For a new project, I can't really think of any reason why you would pick Solaris over FreeBSD. Obviously if your existing setup is Solaris, then migrating it to FreeBSD probably isn't worth the hassle.

    2. DevOpsTimothyC Bronze badge

      Re: Missing "container" option

      The article fails to mention Solaris Zones

      I'd say it was skipped on purpose. All of the unix like OS's had (or have) things like this. Linux has chroot.

      The core difference here is VM's and containers are typically configured internally and are meant to be inaccessible to the host.

      I'm quite squarely in the "containers" camp as I see it more of a concept difference than a technology one. To me

      * A "container" is a single application and it's immediate system dependencies. eg a web server.

      * A "VM" is much closer to what people traditionally call a server. It will typically have multiple applications running eg web server AND database.

      I'd say the whole Containers vs VM's is a little moot in the way the article is presenting it. Both containers and VM's require a "hypervisor" of sorts. For VM's the hypervisors aren't changing as frequently or as radically as the equivalent for containers. As such it's just alot easier to run containers in a VM. Perhaps in 5-10 years.

      One of the great things about docker style containers is it's layering of the filesystem allowing for better resource utilization. The same can be said of the rest of the elements, who have a means of logging in, shipping logs etc when that isn't needed IN the application (they are needed elsewhere to support the application)

  9. spireite Silver badge

    Tomarto - Tomayto

    Quite frankly, they are both only as good as the person who configured the environment/image/whatever.

    I've seen more than my fair share of VMs which have the same volume of security issues as a collander

    Same is true for containers.

    At the end of the day someone needs to oversee all this stuff, and periodically run a security scan on them to see what new CVEs have come out.

    For example, we use docker containers in k8s.

    During deployment. they get scanned for CVEs etc

    Once deployed, they don't get scanned for vulnerabilities ever again..... until, someone makes a change triggering a redeployment. At this point, you get buried under severe alerts because it's not the latest curl, ssh, etc, etc, etc.....

    One of the arguments for not being proactive on the scanning is "It's in our AWS/Azure environment", which is bollocks. You're assuming your security on the cloud environment is appropriate, but someone misses something inevitably

    The key is LOCK IT DOWN AT ALL LEVELS... as a matter of default.

  10. CookieMonster999

    container insecurity

    Container insecurity is very theoretical imho, could you share any real life incident where hackers could utilize it ?

    1. Aitor 1 Silver badge

      Re: container insecurity

      You could compromise one and then read the shared memory, spectre and meltdown being two examples of the issue.

      It is essentially docker breakout or "escapology", so yes, it is a thing.

    2. Michael Wojcik Silver badge

      Re: container insecurity

      Sure, if you can't be bothered to do any research whatsoever.

      Try searching for "container escape exploit". There are dozens of PoCs.

    3. raesene

      Re: container insecurity

      TBH it's not really theoretical at all, there've been plenty of attacks on containerized environments, and CVEs are becoming more frequent as it's used more heavily.

      In general hackers recognize that exploitable container runtimes or orchestrators allow for arbitrary code execution (by design) so clusters and Docker instances have been attacked by people looking to run their applications (most typically crypto-coin mining).

      https://www.aquasec.com/news/2021-cloud-native-threat-report-reveals-new-threats/

      is one example of a recent threat report on container attacks but there are several :)

  11. John Sager

    No obvious 'winner'

    I only voted now (16:15BST 23/6/21) as I don't have a dog in this fight. Interesting to see that when I voted it was exactly 50/50!

  12. QUADRANTDIONYSUS
    Mushroom

    How many barriers do you want in the way of your ransomware encounter?

    Using VMs and containers seems to me like the belt-and-suspenders approach, one that is valued by those who have met Mr Murphy or one of his more evil associates, ransomware.

    Containers, OS's and hypervisors all have security vulnerabilities that are discovered and exploited from time to time. Keeping the containers in a VM both adds flexibility and security by adding another layer of isolation and abstraction between the application and the metal. Sure it does make for a bit more admin effort and a bit more CPU and RAM overhead, but I think it is likely worth those costs. In my opinion the technologies are complementary and most use cases would be best served by using them together.

    Just imagine the result of a container escape to the bare metal running all the rest of your containers - not too hard since it is just like not using any containers or VMs.... Just read the news.

    The results have been bad enough that some CEOs are finding that their stock options are worthless and their future prospects not so bright after all. Fining the corporation just gets passed on to the customers, who were usually also the victims of the corporation's failures. Univerally, the customers who are impacted do not think that is in any way enough of a penalty.

    People's reactions are getting to be like this: Free credit monitoring? YGTBSM! Those lazy incompetent SOB's running things did it so badly that my heart attack treatment was delayed long enough that I went into full arrest and now I can't walk, let alone work, and I need a transplant - AND the letters I am getting from lawyers say that most men my age and health with a first heart attack get a clot buster injection and maybe some stents...

    So far nothing anyone has done about any of this has even tried to make the real victims whole, and in some cases that is not even possible since in some cases customers (usually hospital patients) are dying. Governments are facing pressure from the public to do something more than just issue fines that get passed on the the victims.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like