back to article Want to stay under the radar for a decade or more? This Chinese hacking crew did it... by aiming for Linux servers

A group of hackers operating as an offshoot of China's Winnti group managed to stay undetected for more than a decade by going open source. A report from BlackBerry outlines how the group, actually a collection of five smaller crews of hackers thought to be state-sponsored, assembled in the wake of Winnti and exploited Linux …

  1. Tom 7 Silver badge


    Any idea how many machines may have been compromised?

    1. Anonymous Coward
      Anonymous Coward

      @Tom 7 - Re: Penetration?

      That's the question to be asked. All news on this matter mysteriously lack this crucial detail. They only describe in detail what happens after the attacker has managed to get in and obtain root privileges.

      I suspect the attackers are using flaws in Internet facing applications as well as sysadmin incompetence, especially when servers and applications are being managed by developers themselves.

      1. Anonymous Coward
        Anonymous Coward

        Re: @Tom 7 - Penetration?

        The simplest thing to do is to block ports unless necessary and to not allow SSH or other services to any unknown addresses.

        1. Claptrap314 Silver badge

          Re: @Tom 7 - Penetration?

          Simplest, yes. But like a mask, not a 100% solution. And like a mask, if you're not doing this, you're kidding yourself.

      2. bazza Silver badge

        Re: @Tom 7 - Penetration?

        The worrying part is that they seem to have been careful to not betray their presence on systems. If they've been using exploitable flaws in Linux, rather than configuration mistakes by admins, there's a possibility that they're on a lot of stuff an no one knows it.

        To me their apparent gently-gently approach smacks of an intent to go after some major targets. Hmm lets see, who runs Linux? Google? Amazon? Outfits like that? Can you imagine the consequences if it turns out they've been running round inside, purely as a hypothetical example, Google's infrastructure unobserved for years?

        1. Palpy

          Re: Major targets

          I believe that the US military uses Linux for some purposes. So do some of the US three-letter agencies, again, not exclusively by any means but for certain purposes.

    2. This post has been deleted by its author

  2. Pascal Monett Silver badge

    So, one Linux myth bites the dust

    Not the one where Linux didn't have any viruses. That myth has sunk a while ago already. But the myth that Linux, being a niche product, did not attract hacker attention.

    With hindsight, that myth couldn't hold water as soon as half of the Internet started running on Linux. Linux on the desktop is still a pipe dream, but Linux in the server rooms is very real, and I'll wager there are more Linux server than there are Windows servers at this point in time.

    And those hackers were attracted to the servers, and their sweet, sweet data.

    I'm sure that admins that are on the ball already have proper firewalls and anti-virus tools in place, plus maybe monitoring and intrusion detection for the best of them, Linux or no. For the rest of you, this is your wake-up call. Linux _is_ vulnerable.

    So patch and fortify your defenses.

    1. IGotOut Silver badge

      Re: So, one Linux myth bites the dust

      But aren't most Firewalls Linux based?


      1. Anonymous Coward
        Anonymous Coward

        Re: So, one Linux myth bites the dust

        BSD is quite a bit more common

      2. Anonymous Coward
        Anonymous Coward

        Re: So, one Linux myth bites the dust

        Even firewalls can be compromised....

      3. Anonymous Coward
        Anonymous Coward

        Re: So, one Linux myth bites the dust

        In our computer room?

        Firewalls, switches, routers, intrusion prevention, proxies, network monitors, IPAM, DNS, DHCP, and NTP servers, ASRs, and a few other things, I know I'm missing some other things that are linux powered if you look under the hood.

        There are a couple of obsolete core switches that are being replaced running something else. And some of the servers, of course, (a mix of Solaris, Windows, and Linux).

        I haven't got a clue what the tape library and backup server run, but that's the extent of non-linux/non-server devices in the room.

    2. Anonymous Coward
      Anonymous Coward

      @Pascal Monett - Re: So, one Linux myth bites the dust

      I'll start by addressing your statement about Linux viruses. As long as you don't give us some serious examples of viruses actively infecting Linux servers, the myth still stands. Proof is that actually nobody runs antivirus on Linux servers, except of course for those managed by Windows admins in Windows shops since they can't trust anything that doesn't run an anti-virus.

      When you're talking about Linux on the desktop as a pipe dream, are you talking about widespread adoption or Linux as a desktop alternative. That 1% at planetary scale still makes for millions of Linux desktops, sorry if this is upsetting your stomach.

      Third, as long as I don't have the attack vector, I can't tell if Linux itself is vulnerable. Is it the kernel, the user-land, applications, we don't know yet. And the same is to be said about Linux viruses, those few that are known so far lack any mention of infection mechanism (it seems they rely on goodwill of half-competent sysadmins).

      So hold your horses for a while!

      1. fnusnu

        Re: @Pascal Monett - So, one Linux myth bites the dust

        "Proof is that actually nobody runs antivirus on Linux servers"

        Absence of evidence is not evidence of absence

        1. Tom 7 Silver badge

          Re: @Pascal Monett - So, one Linux myth bites the dust

          I've done AV checks on many machines. Its only ever windows generated content that has ever tested positive.

          Now this 'new' method has been highlighted I will test all my local machines when I can find a sure way of doing so but given Blackberrys reluctance to provide evidence of its spread in the field I'm not sure if its common enough not to be more of problem testing for it.

        2. Andy Non

          Re: @Pascal Monett - So, one Linux myth bites the dust

          The best bet is to keep Linux servers at least two metres away from the internet.

          1. BebopWeBop

            Re: @Pascal Monett - So, one Linux myth bites the dust

            I believe that the Chinese sports federation have been training paclets to leap small air gaps.

    3. Anonymous Coward

      Re: So, one Linux myth bites the dust

      It's not just a Linux myth, it's the myth that any OS can be run without knowledgeable sysadmins.

      Security always costs money and time. If you don't invest in it you are setting your company up for failure.

    4. martinusher Silver badge

      Re: So, one Linux myth bites the dust

      Linux differs form Windows because it is inherently and openly modular. This means that Linux as such can't have vulnerabilities but individual components can, and have had, vulnerabilities. These invariably get patched in very short order but unlike Windows there isn't a megabuild with the attendant risk of introducing yet more problems with the fix.

      Linux is very commonly used, its used in most servers and as the base for Andriod so there is quite a bit of effort taken to attack it and sometimes those attacks succeed. Linux desktops themselves are quite common although you might only see them as "Chomebooks". They're not very popular in business because of the enormous IT control infrastructure that Microsoft has created in the name of security -- management likes the Microsoft computing model even if developers of non-Microsoft based applications hate it.

      1. NATTtrash

        Re: So, one Linux myth bites the dust

        ...but unlike Windows there isn't a megabuild...

        *cough* ~~systemd~~ *cough*

        1. jake Silver badge

          Re: So, one Linux myth bites the dust

          Except the systemd cancer is not Linux and Linux does not need the systemd cancer to operate. Make it your mantra and the world of admining Linux suddenly gets much, much easier.


            Re: So, one Linux myth bites the dust

            I will disagree on both of your points.

            1. systemd at this point, as far as enterprise is concerned, is Linux, just like GNU is Linux. Are there distros that don't use systemd? Sure, popular choices are Devuan and Artix are systemd-free forks of Debian and Arch respectively. Are there distros that don't use GNU? Sure, Alpine Linux is a popular alternative, and it also doesn't use systemd; if you've ever used Docker containers, you likely have used Alpine, since they use it as the default container layout for its incredibly small size and rock-solid stability. But fan forks of the distros that dominate the hobbyist scene and a barebones lightweight OS that prides itself on its minimalism cannot compete with the financial and developmental powerhouses of RedHat, Canonical, Oracle, and to a degree SUSE. And who do you think are some of the main proponents of the cancer known as systemd? The very first two companies I listed AFAIK. You will not enter an enterprise environment running anything but systemd-encumbered enterprise Linux distros like RHEL, Ubuntu, Oracle Linux, or SUSE Linux—maaaaybe Debian, buy only with a third party enterprise support package, or because it's required for or bundled with some application. Only SMBs would ever consider doing otherwise, or some very strange/money-constrained companies with decent admins.

            2. I will be that guy and say with abject honesty that—at least right now, and as long as nothing breaks—systemd administration is usually pretty straightforward, much due to how infested baked in it is with the enterprise distros. The CLI tools are all pretty self-explanitory and you should never have to touch unit files or binary-only formats unless you're a developer, and even then it's often a one-and-done affair.

            Now. None of what I said changes the fact that I hate systemd from all administrative, security, usability, and development standpoints. Only the barest fraction of its codebase could and/or should be salvaged (eudev is ok) while the rest is left to rot in an unmarked grave. Forgetting about systemd is all well and good, but the second you enter the enteprise arena it is very hard to ignore.

            1. jake Silver badge

              Re: So, one Linux myth bites the dust

              Don't be daft. Linux is the kernel, and the systemd cancer is not now, and never will be, a part of the kernel. So no, the systemd cancer is not Linux no matter how hard you squint at it.

              I have never employed the systemd cancer in any enterprise, nor do I intend to start any time soon. Why would I? BSD is a better server OS than Linux (not by much, but it doesn't take much). For the desktops I use an easy to customize, non kitchensinkware variation of Linux called Slackware. You may have heard of it.

              It's not a matter of ignoring the systemd cancer, it's a matter of understanding it, and being able to explain to my clients why, exactly, it's not a very good variation on the init theme. To date they have all agreed with me after a thorough explanation.

              Obviously YMMV ... but feel free to rejoin the light side. It's very liberating.


                Re: So, one Linux myth bites the dust

                You miss my point. "Linux" is both the proper name for the kernel and demonym for all that it touches. Like how I compared "like how Linux is GNU", it may not be true, but the layperson would not know or likely care the difference. I hate it just as much as you do; no need to call me to the "light" side as I am already bathed in the Holy Glow of Alpine Linux on every system and container I personally administrate.

                It sounds like you get to recommend how to build servers, and that's just great. Unfortunately being a developer, maintenance programmer, or systems administrator usually means you don't get that luxury. To CIOs everywhere, the allure of shiny enterprise support packages and bundled applications that "take 5 minutes to deploy" (sic) greatly outweigh the ease of mind that comes with less encumbered, less bloated, first-party-developed systems. It's even worse when the person you end up replacing never had the spine to stand up to such decisions so you are stuck with a mesh of overlapping enterprise closed-source nightmare fuel that breaks should you glance in its general direction.

                Again, I hate it too. But without enterprise support, how would we get by with 1 programmer-cum-administrator paid at the lowest possible salary bracket for his position?? Hiring more people is obviously completely absurd!

    5. Anonymous Coward
      Anonymous Coward

      Re: So, one Linux myth bites the dust

      According to this 2012 article there have been 13 Linux viruses from 1996 to 2010. There is plenty of other Linux malware that is not technically a virus.

      1. Sitaram Chamarty
        Thumb Down

        Re: So, one Linux myth bites the dust

        So I clicked the link.

        Every single one is "risk level: low" and "wild level: low".

        Existence is moot if it does not propagate.

        To be clear, we're not saying Linux is invincible. As someone else mentioned, Linux encompasses all the software that runs on it (at least in people's minds). The Equifax hackers who used Tomcat bugs (if memory serves me right) could easily have written it to *also* propagate, but server to server propagation of binary exploits is not that easy (other than via hacked JS that gets included in a "partner" site).

        But a thriving virus ecosystem, with almost every single unmanaged computer probably hosting at least a few, likely more, viruses? Only in Windows

    6. jake Silver badge

      Re: So, one Linux myth bites the dust

      Monett, your screed is fundamentally flawed for one simple reason: Linux attracted hackers right from the year dot. That's what happens when a new OS built by hackers for hacking appears on the horizon. (Perhaps you don't know what the term "hack" means? If not, you are incapable of intelligently commenting on the subject, by definition.)

      As for Linux on the desktop, it works for MeDearOldMum and my Great Aunt. The only major change after switching them from Windows to a subset of Slackware (as built by me, for them specifically) is that support calls from them have dropped to near zero per year, down from several times per month each.

      Of course Linux is vulnerable. All complex code has vulnerabilities. It's just not as vulnerable as other alternatives. And those vulnerabilities get fixed, usually within hours of being found ... unlike alternatives I could mention.

      Also note that TOA didn't make specific references to any actual vulnerabilities. Nor does the freely available info from BlackBerry. It wants you to provide personal details to them before you can see whatever those details might be. In other words, it seems to be nothing more than marketing bait, and is thus probably worth somewhat less than the paper it's printed on.

      1. Anonymous Coward
        Anonymous Coward


        This article was the equivalent of 'passing meteors threaten earth'

        I was looking for more specific details of the vulnerability and how to detect it

        I run RK hunter daily and if I see a change not triggered by an update, I rebuild the server.

        Would love to know more when you have more

    7. mevets

      Re: So, one Linux myth bites the dust

      "... Linux, being a niche product, did not attract hacker attention..."

      This myth was from MicroSofts talking points on why Windows had so many problems; the implication that it is being victimized because of its own success. In marketing innovation, it is a coup worthy of Jobs: blend a little bit of fact, a mirror, and some smoke, and whoosh a whole new story.

      "Features" of windows, such as the ability to attach lumps of executable code to emails, and have LookOut! helpfully run them on the recipients machine with no action on behalf of the recipient, are more the cause of windows success than the product of it. Of course, no other vendor would dream of doing something so irresponsible. Had they known they would be richly rewarded, they might have.

      The central architecture of running untrusted lumps of data, and opening vast kernel attack surfaces, was what made it the favoured hacking target; it was so easy school children were routinely producing malware for it. The fact that you could spread your malware to just about any machine you could find was merely a bonus.

  3. Stuart Castle Silver badge

    Interesting, and worrying, that they were able to avoid being noticed by going for Linux. Also a good reminder that no matter how good the security of the OS, it can still be compromised, and companies still need fairly rigorous security procedures in place.

    1. jake Silver badge

      "Interesting, and worrying, that they were able to avoid being noticed by going for Linux."

      Proof? Show me the code. Merely telling me it exists doesn't work, I'm a sysadmin not a religious fundie.

  4. Anonymous Coward
    Anonymous Coward

    More like they targeted the multitude of Chinese made devices with defective linux preinstalled...

    set top boxes... usb powered computers....., medical equipment.... IOT....


    Seen this pre-2010......... which is why i wont have any Chinese devices near my home... unless the firmware is replaced........

    1. Irongut

      Ok Donald you can go back to sleep now.

      1. KSM-AZ

        Flamable but true

        Sorry, but I'm dealing with a bunch of chinese built camera's and dvr's, running abandoned firmware. There is a TON of this ship and forget tech out there, sitting on networks, vulnerable.

  5. amanfromMars 1 Silver badge

    In Free AI Spaces of Quiet Contemplation ....

    .... Do New Life Forces Spring into NEUKlearer HyperRadioProACTive IT

    Do state sponsored hackers break down or more simply open novel doors onto platforms and into applications granting full access to the treasures and temptations therein, uncovered and discovered for exercising and fine tuning to a Base Master Root Mutually Satisfying Perfection Assuring and Ensuring All Heavenly Performance?

    If you be of the latter persuasion with a passionate unbridled desire for the delivery and sharing/co-hosting of Such Almighty Satisfactory ACTivity, .... Pleased to See You, To See you Nice. :-) Be Sure to Not Leave a Stranger if Never Tempted to be Fully Satisfied by Heavenly Performance with a Universal Power in Energy Command and Control ...... for it would suggest there be Alternate Fully Satisfying Routes in Other AIMarkets which one may or may not yet have any knowledge of. And that's a hell of a lot to have many much further and deeper chats about. You know some such are Enlightening and Prone to Non-Priming of Viable Defence Counter Measures.

    They used to be thought Problematical rather than considered Virtually Ideal and Practically Perfect. :-) Honest. I Kid U Not.

    1. jake Silver badge

      Re: In Free AI Spaces of Quiet Contemplation ....

      "Do state sponsored hackers break down or more simply open novel doors onto platforms and into applications"

      The latter. Brute force leaves broken bits and bytes behind. Opening and then closing doors behind after entry leaves no traces (idealy ...). You know this, so why ask?

      1. amanfromMars 1 Silver badge

        Re: In Free AI Spaces of Quiet Contemplation ....

        The latter. Brute force leaves broken bits and bytes behind. Opening and then closing doors behind after entry leaves no traces (idealy ...). You know this, so why ask? ....jake

        Because, jake, not enough folk know that is a secret which gives hackers state, and it also has to be said, non-state sponsorship in exchange for a remote virtual command and control leverage .... which is both conveniently invisible and practically intangible ie untouchable.

        You can think of it as a public service if you like from that which flourishes and buries itself deep within the private and pirate sectors securing future programmable events.

  6. AnAnonymousCanuck

    Meaningless Group Of Statements Made Up Out of the Air

    The entire article is BS!

    > Going after Linux servers also has the added benefit of yielding massive caches of data when an attack is successful.

    Really, any evidence to back that statement? Or any other statement in the article?

    Not even close to El Reg's usual carrion quality.

    A very disappointed AAC

    1. Anonymous Coward
      Anonymous Coward

      Re: Meaningless Group Of Statements Made Up Out of the Air

      If they're anything like our BOFHs they haven't really got a clue what's running.

      Rogue processes mounting NFS shared drives and running rampant deleting files? No problem, you're on your own, that must be unsupported software because we don't have anything that does that.

      Tivoli filling up disk space because the event logs haven't been transferred to another system for processing and storage in months? No problem, just disable Tivoli and delete the logs.

      No operating system can cope with stuff like that. I wouldn't be surprised if one day they said they'd found a Counterstrike server hosting games held between Russian and Chinese teams with the NSA spectating.

    2. MarkSitkowski

      Re: Meaningless Group Of Statements Made Up Out of the Air

      If a company has had its data stolen over a period of ten years, I'd expect one of two things to occur:

      1. The company discovers it, and says so

      2. The hackers try to make use of it, and someone notices.

      Since neither of these events has happened, I suspect that either the hackers are content to steal data and wallpaper their rooms with it, or (which is far more likely) that it never occurred,and someone is after free publicity/notoriety.

      1. Anonymous Coward
        Anonymous Coward

        Re: Meaningless Group Of Statements Made Up Out of the Air

        "1. The company discovers it, and says so"

        More likely: Company discovers it and says nothing.

        In two international organisations where I worked in the last 20 years, both had/have policies under which, any employee publicising details of a security incident at the organisation could expect to be summarily dismissed.

        I am aware of serious security incidents at both organisations - kept entirely suppressed at one organisation and unpublicised by the other.

    3. eamonn_gaffey

      Re: Meaningless Group Of Statements Made Up Out of the Air

      Agreed, this dross is not up to the usual Register standards...better not to publish it at all. Must be anxious to fill up content space (sort of understandable in these times).

  7. amacater

    Patch / update / keep current / monitor / check logs

    Nothing new here - the report suggests Red Hat / CentOS / Ubuntu versions - the kernel versions they suggest are mostly CentOS 6 era. Patch / update / keep current. Red Hat licences persist across versions: there's no penalty for moving from one version to the next. Just do it, people. Sysadmin 101. If you don't need a GUI - NEVER install one. Limit the number of services you run: audit logs : baseline to find out what's anomalous -tedious, but nothing unexpected.

    1. Anonymous Coward
      Anonymous Coward

      Re: Patch / update / keep current / monitor / check logs

      That means “employ diligent skilled sys admins, support them to do their jobs, have enough of them”. Not doing this is the first failure of many organisations.

      A couple of other things:

      1. Red Hat have guidance:

      2. Get your logs into a SIEM

      3. Use a respectable EDR solution


    2. adam 40 Silver badge

      DON'T Patch / update / keep current

      Once you have it working, don't be pulling in those vulnerabilities from upstream.

      Freeze the system, make the binaries sit on read-only partitions.

      1. jake Silver badge

        Re: DON'T Patch / update / keep current

        a) You're replying to an 8 month old conversation.

        b) Don't be daft. Sometimes bugs are found in even decades old code. This will be true as long as there are humans in the loop.

  8. jake Silver badge

    It's not a security advisory, people.

    It's an invitation to download an opinion piece in exchange for some personal details.

    In other words, BlackBerry is looking for suckers to market at.

    Nothing to see here. Move along.

    1. GrumpenKraut

      Re: It's not a security advisory, people.

      Thanks, I was suspecting about that after reaching a web form...

      I once filled out such a web from offering some white paper (not pitched by El Reg but a German outlet). Oh how I regretted it, spammy mac spamface from the department of spray-shit in spam hell! Icing of the cake: the white paper was never send.

  9. cam

    You can only do what you can do.

    - Close all unused avenues of approach.

    - Keep systems patched and updated.

    - Keep users access limited to minimum necessary

    - Double down on password practices and policies

    - Have all IT Sec folk aware and up to date on what's what

    You know. The usual stuff that hopefully works, until it doesn't. ;)

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like