back to article This major internet routing blunder took A WEEK to fix. Why so long? It was IPv6 – and no one really noticed

Last week, an internet routing screw-up propagated by Verizon for three hours sparked havoc online, leading to significant press attention and industry calls for greater network security. A few weeks before that, another packet routing blunder, this time pushed by China Telecom, lasted two hours, caused significant disruption …

  1. This post has been deleted by its author

  2. Peter Stevens

    It's not surprising it wasn't noticed. Routing follows the most specific route, so the cloudflare incident was obvious because Cloudflare annouced a /20 and somebody else announced a more specific /21. In this case it's the opposite, everything in 2400::/12 that's actually in use is already announced by a more specific route, somewhere around /29 to /48. The only affected addresses would be ones not in use.

    Nobody noticed the unused addresses in a large IPv6 block going missing because they weren't in use so nobody was affected.

    1. diodesign (Written by Reg staff) Silver badge

      "they weren't in use so nobody was affected"

      Yeah, as we said, doesn't speak well for IPv6 and future routing cockups.

      (Yeah yeah we know IPv6 space is huge and this probably collided with nothing.)

      C.

      1. Chris Fox

        Re: "they weren't in use so nobody was affected"

        "Yeah, as we said, doesn't speak well for IPv6 adoption."

        Perhaps that was meant as a joke, but if not... IPv6 is intended to be sparse: the fact that a large number of unused addresses are not in use merely means those addresses are not in use. It says nothing about IPv6 uptake, or the number of addresses that are in use, or even the number of announced prefixes (either in general or within this block). This error will not have any impact on any conventionally announced addresses. If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes.

        1. diodesign (Written by Reg staff) Silver badge

          "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

          Hm, I see where you're coming from. We'll keep it in mind in future.

          (Edit: Tweaked the article to include your counterpoints. Completely accept that IPv6 is vast, that it didn't break despite this error which is a good thing, and that more specific routes would have been used. As watchers of IT blunders on a daily basis, who see failures developing a mile off, we were concerned that no alarm was raised, and no fix was applied, for several days, which makes us contemplate more problems in future.)

          C.

          1. Anonymous Coward
            Anonymous Coward

            Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

            IP6 is big. Really big. You just won't believe how vastly, hugely, mind-bogglingly big it is. I mean, you may think a really big network is a /16 IPv4 network at your employer, but that's just peanuts to IPv6.

            1. Anonymous Coward
              Anonymous Coward

              Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

              Thanks for applying a classic. LOL.

            2. EnviableOne Silver badge

              Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

              There are enough IPv6 addresses to give 7 to every atom in every human on earth.

              Personally i think its excessive, and if you are using v6 with wifi and have 4 addresses in the header, its starting to become a serious overhead.

              1. Jellied Eel Silver badge

                Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

                Personally i think its excessive, and if you are using v6 with wifi and have 4 addresses in the header, its starting to become a serious overhead.

                It's only the assignment policies that are excessive, ie originally it was proposed to assign end users a /48, which is a few trilion addresses. That changed to recommending a /56 for a home user, which means enough for a few billion IoT gizmos per home. Then there's /64 for a single LAN, mainly due to v6 being designed to encapsulate MAC addressess, so your single LAN can contain all of them, which I don't recommend due to likely congestion.

                So most IPv6 space is going to be wasted, but luckily /56 policy allows for a good few more routes than v4 did..

                1. cdegroot

                  Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

                  The numbers work, of course (I have a /56 and use only one /64 in that - calculate the number of addresses I'm not using!). Say my ISP gets a /32, that means that we can have 4,294,967,296 ISPs on the planet. Each of them can hand out 2^(56-32) = 16,777,216 customers (and once you get over that, you nab a new ISP block).

                  The biggest advantage of this very sparse ("wasteful") allocation strategy is that you can have very small (compared to IPv4) routing tables. "The internet" only needs to know about active ISP prefixes, ISPs only about what prefix they handed out to customers. I guess that once the designers of IPv6 realized just exactly how mindbogglingly vast the space of 340,282,366,920,938,463,463,374,607,431,768,211,456 addresses is, they realized that this wasteful approach could actually make things much more efficient without risking another address space problem.

                  Also, I like all the addresses I have at my disposal. My laptop's WiFi adapter has a bunch active: a link-local one, for low level techie stuff (fe800::...); a network-local one for things that, well, shouldn't leave my network or be accessible from outside (fd00::...); and a proper publicly addressable one. Plus temporary versions of the latter two to help thwart (ad) tracking - a lot of the 2^64 address space I have is used for that - and a predictable version of each based on my device uid. Initially, it all smells like horribly over the top but once you dive into it it starts making more and more sense.

                  1. Jellied Eel Silver badge

                    Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

                    The biggest advantage of this very sparse ("wasteful") allocation strategy is that you can have very small (compared to IPv4) routing tables.

                    In theory, yes, although each v6 route takes up more FIB/RIB resources. There's handy stats for v6 (and v4 of course) here-

                    http://bgp.potaroo.net/v6/as2.0/index.html

                    with 208792 routes visible in APNIC's test RIB. So that puts it.. err.. around mid-90's in terms of v4 table size. And equates to 0.003008% of available space (ie in /64 terms). It's also interesting that there's only a small number of multi-origin prefixes, which hints at corporate adoption (or lack thereof).

                    Plus temporary versions of the latter two to help thwart (ad) tracking - a lot of the 2^64 address space I have is used for that

                    Curious how effective that would be, ie to narrow down to a 'user', ad slingers will only really need to track down to the /56, unless they're really desperate enough to track down to individual hosts/users. Which they probably are, but can manage that by aggregating data from all the cookies & crumbs they scattter.

              2. Yes Me Silver badge
                Facepalm

                Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

                "if you are using v6 with wifi and have 4 addresses in the header, its starting to become a serious overhead."

                Try again. You may have 4 MAC addresses in the WiFi header, but you will never have 4 IPv6 addresses (unless you decide to tunnel IPv6 inside IPv6, of course, but that would be a bit silly).

              3. James_K

                Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

                Why would 4 addresses be used? IP only uses 2 addresses, whether IPv4 or IPv6. Perhaps you're thinking of the MAC addresses. WiFi frames use up to 4 MACs.

              4. James_K

                Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

                "There are enough IPv6 addresses to give 7 to every atom in every human on earth."

                There are enough to give every person on earth over 4000 /48s.

            3. JJKing
              Facepalm

              Re: "If anything, it is a demonstration of how robust IPv6 can be in the face of such mistakes."

              IP6 is big. Really big. You just won't believe how vastly, hugely, mind-bogglingly big it is.

              My ISP has assigned either 14,000+ Quadrillion IPv6 addresses or 253,000+ Quadrillion IPv6 addresses for my personal use. At present I only use about 20 addresses. It's gonna cost me a Bezos fortune to just make a slight dent in those numbers.

              Yes, it is a mind-bogglingly number.

      2. Nick Kew

        Re: "they weren't in use so nobody was affected"

        Not sure it says much about adoption one way or t'other. IPV6 addresses are by design much sparser than ipv4.

        What it says to me is that ipv6 is harder to read, patterns are harder to interpret. That's partly inherent: much longer numbers to read. And partly familiarity: anyone who deals with networking has long familiarity with ipv4 and recognises many things like a mother tongue.

        What it says to me is that we should think about a new generation of tools to work with ipv6, rather than just lazily adding it to the tools we've been using for a (human) generation. Which is, I daresay, already obvious to commentards who've given it a bit more thought.

        1. Nanashi

          Re: "they weren't in use so nobody was affected"

          Is it really harder?

          Look at this address: 2001:db8:abcd:1::2. The first three segments ("2001:db8:abcd") are the prefix assigned by the ISP. The next segment ("1") is the subnet ID I assigned (matching VLAN 1, perhaps). Finally, the "::2" part identifies the host on the network.

          Compare that to v4, where the same machine is something like 203.0.113.51/27. Which parts of that are the prefix, which parts are the subnet and which are the host? I definitely can't tell at a glance.

          v6 isn't hard. People are just scared by new things.

          1. Michael

            Re: "they weren't in use so nobody was affected"

            20 years old is new? Seriously?

          2. James_K

            Re: "they weren't in use so nobody was affected"

            "v6 isn't hard. People are just scared by new things."

            +1

  3. Julian Bradfield

    Why would anybody notice, particularly? Surely the individually allocated IPv6 blocks are all much smaller than /12, so their announced routes will take priority over the /12 route. (APNiC allocates /32s, I think.)

    1. diodesign (Written by Reg staff) Silver badge

      'Why would anybody notice, particularly?'

      Doesn't bode well that an advertisement like this is only picked up days later. Maybe we're worrying over nothing; maybe not.

      C.

      1. doublelayer Silver badge

        Re: 'Why would anybody notice, particularly?'

        I wonder though. It's true that an announcement of an IPV4 block gets reported immediately, but what if I did something like this and announced (we're presuming I have the ability to announce and be taken seriously) a new route for a /2 block, which is around the same size as this block? Once again, almost everyone has a more specific route taking them to the various parts of the network, and completely skips what I said. I think it would be noticed a bit faster, but I doubt there would be "discussion on social media within minutes" because it wouldn't break much. The reason the more typical reroutings do get announced so quickly is that either people start noticing the traffic taking a long time and check the route or the new announcement isn't paired with an ability to actually get to the resource meaning things are obviously broken. If my announcement gets ignored, someone has to notice the anomaly manually and deal with it at that point.

      2. John Sager

        Re: 'Why would anybody notice, particularly?'

        I don't think we are worrying over nothing. IPv6 implementation is a golden opportunity to do the admin 'properly', as the address block allocation is well-structured from the start. So you would expect that RIRs could work with ISPs to implement processes to implement and check announcement filtering as a 'it just happens' thing.

  4. Jellied Eel Silver badge

    But... why?

    The /127 would be 2. Just 2 IP addresses.

    Ok, so I guess it bodes well(ish) for someone's address conservation, but it's not just 2. It's 2400::0 & 1. Which means subnet zero's finally a thing! Or I'm curious what config someone was trying to achieve that wouldn't have required more witchcraft.. I mean hexes.

    1. doublelayer Silver badge

      Re: But... why?

      I think it was probably something like 2400:xxxx:.../12, I.E. a more specific address that then got truncated by the typo. For example in IPV4, if someone did 12.34.56.78/8, some programs would just assume that to mean 12.0.0.0/8.

      1. Jellied Eel Silver badge

        Re: But... why?

        For example in IPV4, if someone did 12.34.56.78/8, some programs would just assume that to mean 12.0.0.0/8.

        Yup, that's correct, ie IPv4 and v6 work on an address, and the netmask/route. Programmes shouldn't rewrite or assume it means 12.0.0.0, but should route as being part of 12/8

        IPv6 complicates things with a lot more reserved addresses & scopes.. But xxx.0 has always been a bit special, hence the comment about subnet zero, which was to counter a hold-over from pre-CIDR days and the first address being the broadcast address. Which becomes irrelevant on a P2P link with only 2 endpoints. Many v4 addresses have been wasted by people using /30's instead of /31. So it could have been an attempt at configuring a link, or misconfiguring anycast.

        Either way, the /127 route shouldn't have escaped into the global table when that ideally wouldn't contain anything more specific than /56 customer allocations.. Which is back to Cloudflare scenarios and ISPs not having sensible route filters.

    2. Anonymous Coward
      Anonymous Coward

      Re: But... why?

      /127 (& /31 in the IPv4 world) have been supported for point-to-point links for a while now. See RFCs 3021 & 6164.

    3. Anonymous Coward
      Anonymous Coward

      Re: But... why?

      Yes, due to security issues serial links are strongly recommended to be /127's.

      https://www.ripe.net/publications/docs/ripe-690

    4. Yes Me Silver badge
      Headmaster

      Re: But... why?

      /127 subnets are sometimes used for point-to-point router links.

      See RFC6164.

    5. James_K

      Re: But... why?

      "The /127 would be 2. Just 2 IP addresses."

      That would be a point to point link. However, one thing a lot of people fail to understand is that routing is often done over link local addresses, not GUA, with only a /128 address to provide an address for testing, etc., but not for routing. I had a problem with my ISP a few months ago and I had to explain that point to 2nd level support.

    6. vabello

      Re: But... why?

      Point to point link? Or you just need two /128’s routed to devices as loopbacks? Also, IPv4 addresses ending in .0 and .255 are also perfectly valid and usable IP addresses in several scenarios (various routed IP pools of any prefix length, or /23 and shorter prefixes will contain them). You assume the world is all /24’s or are LANs? The major issue with them is that some programmers have made the same mistake assuming something ending in .0 or .255 is always a network or broadcast addresses which is false. As a result, using these IP’s unfortunately results in some user experiences being broken, so it’s best to avoid using them despite there being no technical reason to do so.

      1. James_K

        Re: But... why?

        "Point to point link? Or you just need two /128’s routed to devices as loopbacks?"

        Point to point links need 2 addresses, one for each end. However, IPv6 does not need a global or even unique local address for routing, as link local addresses are often used. A /128 means the address is used to identify a system, but is not used for routing. For example, my firewall/router has a /128 address. I can use that address for testing with ping, traceroute, etc. I can also use it as a VPN end point. Any traffic for that address will travel from the ISP using the link local address.

        The IPv6 loop back address is ::1.

        1. Jellied Eel Silver badge

          Re: But... why?

          The IPv6 loop back address is ::1

          Indeed.. Which is why I doubted the explanation, ie if it was someone misconfiguring a network within 2400, then I'd have expected more hex in the advertisment. It doesn't seem right that it was just missing a digit off the subnet size. But it's also why I don't really like v6, ie it's not very human friendly. As an experiment, try getting someone to correctly configure address/netmask when you've got 2 engineers in noisy machine rooms.

  5. -tim
    Facepalm

    Noticed, reported up stream, fixed days later

    We noticed. We lit up a new IPv6 link and our provider is still using 1999 BGP concepts on filters so we had to debug links without being able to see what filters they had, what routes they were accepting, a silly process that won't allow us to talk to the NOC combined with a "order" system that not only is clueless about IPv6 but crashes when it finds IPv6 addressees where it expects IPv4 ones.

    RFC 7454 would imply more ISPs need to look into the "GE" and "LE" values on their BGP filter lists.

    In our cases, the unused parts of our /32 were all going to Hurricane Electric San Jose. Makes be wonder if the routers are a fan of Dionne Warwick.

    1. vabello

      Re: Noticed, reported up stream, fixed days later

      Announce your aggregate /32 instead of only the used networks and stop polluting the Internet routing table. You wouldn’t see your unused networks going anywhere else then when this happened.

  6. Lee D

    Well, at least we know for certain that The Reg couldn't have ever been affected by this one. Right?

    1. Jamie Jones Silver badge
  7. vtcodger Silver badge

    What transition?

    But if ever there was a symbol of how miserably the transition from IPv4 to IPv6 is going

    The world is a different place than it was when IPv6 was developed fifteen years ago. For one thing, it turns out that the exhaustion of IPv4 address space is much less catastrophic than was anticipated. It can be, and is being, worked around. For another the potential security and social problems associated with billions of poorly designed and badly implemented consumer products directly addressable by manufacturers, hackers, advertisers, nation states and other scoundrels -- What is coming to be called "The Internet of Shitte" -- were not well understood. If they had been, it's possible that IPv6 would be a substantially different protocol. Third, it was assumed that people would have no choice but to "upgrade" to IPv6 so backwards compatibility with IPv4 was not included. It is claimed that it's not technically possible to do that -- an assertion that strikes me as quite improbable. Anyway, the result was that network people worldwide looked at the effort and budget required to "upgrade" and decided to wait a while -- next year. Or maybe next century.

    My suggestion. IPv6 people should recognize that they screwed up. Cancel the "transition" Come up with something serious and effective to tame IoT problem (which they didn't cause, but truly must not enable). Then design a new protocol compatible with both IPv4 and IPv6. And give a lot of thought to ease of implementation this time.

    1. Gerhard Mack

      Re: What transition?

      "My suggestion. IPv6 people should recognize that they screwed up. Cancel the "transition" Come up with something serious and effective to tame IoT problem (which they didn't cause, but truly must not enable). Then design a new protocol compatible with both IPv4 and IPv6. And give a lot of thought to ease of implementation this time."

      Please explain how to extend a fixed 32 bit header field without slowing down packet processing or introducing incompatibilities.

      1. vtcodger Silver badge

        Re: What transition?

        Well they **COULD** have dedicated one (1) IPv4 address for use as a flag that a 128 bit address will be found elsewhere in the header. That's probably not the best answer, but it's what pops immediately to mind. Truth is surely that the IPv6 designers thought (incorrectly) that they were dealing with a captive audience that had no choice but to follow the IPv6 game plan.

        1. Gerhard Mack

          Re: What transition?

          "Well they **COULD** have dedicated one (1) IPv4 address for use as a flag that a 128 bit address will be found elsewhere in the header. That's probably not the best answer, but it's what pops immediately to mind. Truth is surely that the IPv6 designers thought (incorrectly) that they were dealing with a captive audience that had no choice but to follow the IPv6 game plan."

          So adding a branch to the middle of packet hadling code forever. Congrats. Your plan slows down packet processing. A branch may not seem like much, but consider what happens with a device is used for routing and/or exists on a high speed interface. Those branches add up.

          The IPv6 designers knew the transition was going to be painful and there was no other way around it (you can even read their logic if you bothered to look) so they went out of their way to make it only needed once. You could try looking it up yourself instead of assuming the worst possible motives for people you haven't met.

        2. Nanashi

          Re: What transition?

          Truth is that you're not familiar enough with the problem space, or the existing solutions, to be able to criticize the people who are.

          It'd be possible to assign a flag address like that, but no existing v4 hosts would recognize the address or be able to handle the extra addressing bits, so doing so wouldn't make existing v4 hosts be compatible with v6.

          Plus we actually did something very much along these lines -- we defined a new protocol (6: TCP, 17: UDP, 41: 6in4) which places those extra address bits just before the payload. This is basically the same as your suggestion except that the magic value goes in the protocol field rather than the address field. But them doing what you suggested apparently wasn't enough to stop you from writing a great big post on how they screwed up.

    2. John Sager

      Re: What transition?

      I invite you to write a RFC. Surely it can't be that hard?

      1. vtcodger Silver badge

        Re: What transition?

        "I invite you to write a RFC. Surely it can't be that hard?"

        Actually, I've written a few specifications in my time. Sometimes it's not all that hard -- **IF** you understand the problems, talk to everyone involved and can reflect their needs. ... And sometimes it 's difficult or impossible. But the issue isn't my of skill level. It's that the IPv6 spec writers made two major mistakes -- one not their fault. Those REALLY need to be resolved before IPv6 adaptation will become (near) universal.

    3. Lee D

      Re: What transition?

      It took years to be in EVERY major operating system, every modern protocol and standard, and be tested extensively. People don't avoid IPv6 because it's not ready, they avoid it because THEY aren't ready (*cough* REG *cough*).

      DOCSIS3 (cable modems), LTE (4G and above) all specify IPv6 as a core requirement. Your phone must be able to support it. Google see 25% of it's search traffic come in over IPv6: https://www.google.com/intl/en/ipv6/statistics.html

      Because you're ALREADY USING IT and didn't even realise.

      How long would it take to gain worldwide agreement on your IPv4.5, get code for it into every major router, switch, network device and OS, deploy it long enough that the vast majority of hardware used and sold worldwide supports it, test it and convince everyone to use it? Because it will take half that time for IPv4 to run out (and it has run out... don't think it hasn't. At least one continent cannot issue any more IPv4 at all... that's why Carrier-Grade NAT is being deployed as a interim measure) and for you to get on board IPv6.

      The reason IPv6 continues to lag is that IPv4 is still going. That's not true indefinitely. When it is true, you know that you can turn on your PC, and your OS, router, web browser, all of the major web servers in the world, the DNS, etc. all support the upgrade from the second it's necessary and it's being used daily by millions upon millions of people already. It's so seamless, you don't even know that your phone is doing it, and yet you're contributing to the 25% of Google searches that use it.

      You want to throw that all in the bin and *start again* on something completely different and without a single deployed line of code, that - at best - will only be another interim measure before you need something like IPv6 anyway? That's the very definition of insanity.

      How about you just take a few hours and IPv6 any website you own (*COUGH* Reg *COUGH*), any internet service you run, any domain you control? You do own a portion of the Internet, right, and not just speaking from your laptop on your sofa about how people should deploy the Internet? Guess what... I did. It took me an hour or so, about 8 years ago, to deploy IPv6-working NTP, HTTP, SSH, SMTP, IMAP and numerous other services on all my Internet-facing servers / gateways (you don't even have to IPv6 everything - just the gateway!). It's that low impact that you can do it now, today, without effort.

      What you will still be able do is browse to websites that are completely lazy despite being tech-focused (*COUGH COUGH* REG! *COUGH COUGH*). But you're already seeing hosting companies that charge for every single IPv4 address, are reallocating all their spare addresses, and giving away IPv6 address ranges for free.

      Unless you're on the back-end, providing services, IPv6 isn't going to affect you one bit. Don't worry. One day your ISP will turn it on and all your routers and laptops will still work exactly as they do now - because they all support IPv6, it's just not turned on or be default, and even when they do, they can still exist on IPv4 forever and just NAT you if they need to.

      If you're on the back-end, providing services, you'll see that it takes minutes to deploy and works just the same without any hassle at all, no more reading up than a little primer and "what's the command again?". Everywhere there's an A record, put in an AAAA. Everywhere you assign an IP, assign an IPv6 address. Done.

      Windows 7, 8, 10 have *all* had IPv6 loaded in them and, by default, bound to all network interfaces. Windows 7 is now officially obsolete. You think you're going to get that kind of deployment, vendor support and testing overnight with any half-assed middle ground? That's before you even get close to the people actually pushing packets around networks (i.e. switch and router manufacturers) where it actually *matters*.

      1. vtcodger Silver badge

        Re: What transition?

        You think you're going to get that kind of deployment, vendor support and testing overnight with any half-assed middle ground?

        Overnight? Of course not. Only IPv6 fanbois think (or used to think -- maybe they've learned a bit) implementation details are simple. OTOH, everybody supports IPv4 also, so maybe it'd be good idea next time to plan to support it as a subset of the network protocols.

        I actually don't have any problem with running major network backbones on IPv6. It seems that might be a pretty good idea. Folks at that level have the resources to deal properly with IPv6 and I'm under the impression that IPv6 includes real improvements in message handling. For all I know, a well planned IPv6 roll out might have been built around exactly that.

        However, I don't see anybody ready to deal with the IoT issue. It does have to be dealt with. Dropping IPv4 before those monstrosities are housebroken looks to be a really bad idea for most end users.

        BTW, IMHO what is half-assed is the IPv6 rollout. You folks should possibly quit blaming everybody else for your self-inflicted problems.

        1. Jamie Jones Silver badge
          FAIL

          Re: What transition?

          "Only IPv6 fanbois"

          All credibility lost right there.

          1. vtcodger Silver badge

            Re: What transition?

            I Agree. Jamie. You clearly have a credibility problem.

      2. Lee D

        Re: What transition?

        "6 thumbs up & 4 thumbs down" - Irony.

      3. James_K

        Re: What transition?

        "Windows 7, 8, 10 have *all* had IPv6 loaded in them and, by default, bound to all network interfaces."

        Actually, IPv6 was mostly there in XP SP3.

    4. Brewster's Angle Grinder Silver badge

      Re: What transition?

      But it's the human side they really screwed up on. They underestimated the drag of millions of sysadmins having to unlearn everything they know and learn a new, harder system. (Probably because sysadmins were fewer in number, smarter, and better paid, back in the day.) If sysadmins perceived IPv6 as the newest and greatest, they would be pestering bosses for the cash and smuggling in the kit; the transition would happen. Instead, we're still debating the notation. And given that we're still debating whether to use tabs of spaces (and how many spaces) I don't see that debate ending any time soon.

      1. JJKing
        Mushroom

        Re: What transition?

        They underestimated the drag of millions of sysadmins having to unlearn everything they know and learn a new, harder system.

        And all those noobies coming into the profession will only need to learn IPv6 and then be streets ahead of those who do not update their "skills" and IT knowledge.

        I came from IPX/SPX, NetBEUI and had to learn that oh so difficult IPv4. Yes IPv6 is different but it is part of our required knowledge and to be honest, it is not like it was just released a week ago. This is something any IT Professional should have been looking at and learning about for the past 10 to 15 years. Yes I found IPv4 easy, especially when I could remember the IP range of each site I worked in. It is a lot easier to remember 12 digits then 128 alpha numeric. That is never going to happen so it is not on my to do list. Instead I will see what the IT Pros cleverer than me come up with so I can adapt and overcome.

        It's not hard, just difficult, for a short period of time, until it is not difficult anymore and yes, I am an IPv6 fanboi and proud of it. It is the way of the future like the automobile was better than the horse and buggy. Bet new horseless carriage drivers found it difficult to learn after spending all their life on a horse.

      2. vabello

        Re: What transition?

        I did this ten years ago at my last job. The kit we had already supported it back then. I just had to get a /32, configure it across all our backbone and start assigning it and using it. It solved quite a few problems for us and made management of our services easier.

    5. Jamie Jones Silver badge

      Re: What transition?

      IPv4 gets hobbled by nat hacks, due to address exhaustion.

      IPv6 fixes this.

      Your argument against this is that the fix is bad because of potential broken implementations?

      As for backwards compatibility, you have dns64, nat64, 6over4, 6in4. How would you get true backwards compatibility on a lower level?

      Sure, the protocol could have been designed so that ipv4 routers could still route traffic if the ip6 header was contained in an ip4 header with an ip address of "some ip6 gateway", but:

      1) That will not help end systems, only core routers, and as with other things, it's the "last mile" that's the issue. Most routers are already natively IP6.

      2) Extra overhead and processing required unnecessarily for Ip6 networks.

      3) You are basically just enforcing an always "6in4" - no point. For those routers that require it, just run your ip6 in 6in4 without forcing it where it's not needed.

      so that's the network sorted.

      For the hosts, how do you make a new Ip protocol with a bigger address range that is compatible with current IP4 hosts?

      Ipv6 is already as fully backwards compatible as it can be.

      1. Peter2 Silver badge

        Re: What transition?

        IPv4 gets hobbled by nat hacks, due to address exhaustion.

        And routing ports via NAPT is well understood by everybody doing anything with IPv4.

        IPv6 fixes this.

        Yes. It fixes a problem that had already been fixed, and creates an entire array of problems that require remediation, most of which won't actually be fixed creating a security nightmare.

        And the "this requires that everybody be retrained" bit is actually the real killer, because IPv6 obsoletes any knowledge of how networking works. You have people in IT who have paid multiples of their yearly salary to go on all the CCNA courses and the people who have just figured it out themselves. The people who have been on the training courses (or their employers) don't want to pay for another set of training courses and the people who haven't done the training don't want their hard earned knowledge obsoleted. And the beancounters refuse to pay silly money for new equipment (firewalls, modems, wap's etc) because it costs too much.

        Net result: practically nobody actually wants IPv6. Techs as a general rule would rather disable IPv6 and just use IPv4 for their internal networking, and generally do unless they specifically must work with IPv6 at a carrier level.

        What would have been a better idea? The problem: IPv4's address space of 254*254*254*254= 4,162,314,256 addresses (four billion, one hundred sixty-two million, three hundred fourteen thousand, two hundred fifty-six addresses) is almost exhausted.

        The solution: create IPv4+2, which has 6 coulons. 254*254*254*254*254*254 = 268,535,866,540,096 (two hundred sixty-eight trillion, five hundred thirty-five billion, eight hundred sixty-six million, five hundred forty thousand, ninety-six addresses)

        Yes, this would require a massive rewrite of networking code on the same scale as IPv6. However, if it was written it would have been (and still would be) implemented with a speed that would make your head spin because everybody knows precisely how it works and isin't inconvenienced by it in the slightest as their existing ipv4 knowledge and practices are carried straight over.

        Obviously it's a fudge but it would still give everybody alive on earth several hundred thousand addresses each. /problem solved.

        Obviously, it'll never happen. But that's a practical and relatively moderate solution that could/would command immediate widespread support due to a positive message of why people should upgrade (look, it's just what you want! East upgrade! No training costs! Works just as you expect!) unlike the unwanted travesty that some people are trying to ram down everybodies throats with a negative message of "WE KNOW YOU DON'T WANT THIS BUT YOU HAVE TO HAVE IT ANYWAY OR ELSE."

        How's that going again? Ah, yes. Predictably badly.

        1. JJKing
          WTF?

          Re: What transition?

          Net result: practically nobody actually wants IPv6.

          Really?

          World IPv6 Deployment

          My sincerest apologies for the link target but my sciatica is being a tad extra naughty today which resulted in larger dose of brain fog producing oxycontin, hence the lazy search.

        2. James_K

          Re: What transition?

          "And the "this requires that everybody be retrained" bit is actually the real killer, because IPv6 obsoletes any knowledge of how networking works. You have people in IT who have paid multiples of their yearly salary to go on all the CCNA courses and the people who have just figured it out themselves. The people who have been on the training courses (or their employers) don't want to pay for another set of training courses and the people who haven't done the training don't want their hard earned knowledge obsoleted. And the beancounters refuse to pay silly money for new equipment (firewalls, modems, wap's etc) because it costs too much."

          Actually, for the most part IPv6 works exactly the same as IPv4, other than larger addresses. There are differences, such as fixed length headers and extension headers, which improve router performance, ICMP6 is used for a lot more than ICMP in IPv4, which rationalizes a lot of things, including getting rid of ARP, which actually predates IP. There are other things that improve security too. As for Cisco certification, IPv6 has been part of the curriculum for several years, so anyone with a current certification should know IPv6.

          If you're doing a massive rewrite of networking code then do it right and start from scratch, rather than building in more hacks.

    6. James_K

      Re: What transition?

      "My suggestion. IPv6 people should recognize that they screwed up. Cancel the "transition" Come up with something serious and effective to tame IoT problem (which they didn't cause, but truly must not enable). Then design a new protocol compatible with both IPv4 and IPv6. And give a lot of thought to ease of implementation this time."

      This is the real problem, hack after hack to get around the limitations of IPv4 address space. As a result, we have a hack called "NAT", which breaks the end to end nature of IP. Then we have hacks like STUN, to get around the problems NAT causes for VoIP and games, etc..

      Instead of coming up with all these hacks, just start from a clean slate and come up with something adequate, such as IPv6. As for your idea, as others mentioned, it will add to the processing workload, something IPv6 was designed to avoid, with fixed length headers and more.

  8. Anonymous Coward
    Anonymous Coward

    If an IPV6 network falls over in the forest......does anybody care??

    Nobody uses it, nobody affected....just wait until it's more popular and something like this goes down.....

    1. Nanashi

      Re: If an IPV6 network falls over in the forest......does anybody care??

      There are over a billion v6 users worldwide. That's far from "nobody".

      1. Hans 1
        Boffin

        Re: If an IPV6 network falls over in the forest......does anybody care??

        There are /127 sorts of people on this planet, those who understand IPv6 and those who do not.

  9. gnarlymarley

    My ISP didn't have problems

    no one really noticed

    This is because my IPv6 ISP had it setup to reject bogus advertisements. I didn't notice downloading issues, but I did notice cloudflare's certificate that was advertised on thereg's https servers had expired. Maybe related?

    1. JJKing
      Thumb Up

      Re: My ISP didn't have problems

      no one really noticed

      gnarlymarley, I couldn't agree more. My ISP told me several years ago that IPv6 was not in the foreseeable future. I was pretty pissed off when their business clientele were provided with static IPv6 addresses a couple of years later. In January this year (2019) they started a trial rollout to their residential customers and I did not discover this until I read about it in

      Whirlpool Forums

      about a week ago. Absolutely seamless!

      I was still pissed off because I missed out on being a beta tester for the trial; yet another chance to learn more about IPv6. However I now have many quadrillions of addresses that I can use. YIPPEE!!

  10. Anonymous Coward
    Anonymous Coward

    Warning. You are announcing over 2 billion addresses! Proceed Y/N?

    Can you do something like that?

    1. James_K

      Re: Warning. You are announcing over 2 billion addresses! Proceed Y/N?

      "How? Not possible while any site you might need uses IP4 only and some gadget that can't be updated only uses IP4. And while a single ISP (fixed or mobile) doesn't do IP6 properly."

      Look up "bogon filtering", which routers can use to block invalid address space.

      1. vabello

        Re: Warning. You are announcing over 2 billion addresses! Proceed Y/N?

        Not even bogon filtering, but the upstreams and/or peers of that network should only be accepting routes their customer or peer are allowed to announce by using filters, or ideally RPKI. It would have never propagated beyond the source.

        1. Jellied Eel Silver badge

          Re: Warning. You are announcing over 2 billion addresses! Proceed Y/N?

          That's what Kieran's getting at with BCP for ISPs. There's a lot of trust involved with BGP & peering, and it's easy to make mistakes. So it's beholden to upstreams to minimise propagation of downstream's fat-fingers.

          Which in theory is easy(ish) to do, ie in general terms, you should only ever advertise routes you or your downstreams are allocated, and can be identified in a handy RIR db. Then to avoid bloat, don't allow more specifics than say, /56 or /48s, and aggregate at your borders. There shouldn't be any real need to advertise the trusty old 'Class C' or /64 for customers/downstreams. With business networks, assigning a /56 per site should be more than enough for most customers.

  11. Adrian 4 Silver badge

    How about

    'millions, if not billions of addresses will receive your announcement'

    it's got form.

    1. JJKing
      Happy

      Millions, if not billions of addresses

      No Adrian, not billions but quintillions, well at least from my ISP. The numbers are mind bogglingly crazy.

      I read an article that said the full IPv6 range had more addresses than atoms in the Universe. I wonder who counted them?

  12. Mage Silver badge
    Big Brother

    Shifting to IP6 properly?

    How? Not possible while any site you might need uses IP4 only and some gadget that can't be updated only uses IP4. And while a single ISP (fixed or mobile) doesn't do IP6 properly.

    Privacy is a huge issue on IP6.

    Interworking.

    Gadgets.

    Originally the designers SO naive that the IP6 was based on MAC. At least that's fixed.

    The very idea of it seems to have ignored issues of end users configuring and privacy and concentrated on big and the silly idea that EVERYTHING SHOULD connect uniquely to the internet, which is great for exploitive Mega Corps and evil regimes and terrible for every ordinary user.

    The majority of businesses and homes have no sysadmins

    1. Anonymous Coward
      Anonymous Coward

      Re: Shifting to IP6 properly?

      You'll be relieved to learn that IPv6 has had privacy extensions for more than a decade, enabled by default nowadays. They use temporary addresses: randomly generated addresses are used for outgoing connections, and they change regularly. So, it's actually quite difficult to tie a specific user to a single address.

    2. Yes Me Silver badge

      Re: Shifting to IP6 properly?

      "Privacy is a huge issue on IP6."

      No it isn't. It's a huge issue in the application layer of the Internet; IPv6 has more privacy features than IPv4.

    3. James_K

      Re: Shifting to IP6 properly?

      "How? Not possible while any site you might need uses IP4 only and some gadget that can't be updated only uses IP4. And while a single ISP (fixed or mobile) doesn't do IP6 properly."

      There is something called 464XLAT for just that purpose. It converts between IPv4 and IPv6, as needed. My cell phone is on an IPv6 only network and uses 464XLAT for access to IPv4 sites and for apps that are IPv4 only.

  13. Anonymous Coward
    Anonymous Coward

    This actually impacted quite a lot

    route-views>sh bgp ipv6 unicast 2400::/12 longer-prefixes | i /

    * 2400:0:500::/40 2001:1860::223 0 101 101 174 4766 i

    * 2400:C0::/28 2001:1860::223 0 101 101 174 4766 i

    * 2400:144::/48 2001:1860::223 0 101 101 174 4766 i

    * 2400:181::/64 2605:8200::18 0 19653 6461 4766 i

    <lots omitted>

    Trust me when I say this impacted thousands of prefixes. In fact, I suspect the reason it didn't cause issues is that a lot of transit's etc filtered it off.

    1. Julian Bradfield

      Re: This actually impacted quite a lot

      But as pointed out in the very first two comments, longer prefixes were not impacted!

      1. Anonymous Coward
        Anonymous Coward

        Re: This actually impacted quite a lot

        You're quite right, bad use of language. Should have said "affected" rather than "impacted". I was referring to the up-thread musing about empty address space :-)

    2. vabello

      Re: This actually impacted quite a lot

      Nope. It had no impact on any of those prefixes. They’re longer and a longer prefix is always preferred. The only impact would be on unannounced prefixes in the /12, which were... unannounced and not publicly used.

  14. Anonymous Coward
    Anonymous Coward

    So, yes, I did not notice

    And yet I use IPv6 quite literally all the time on my personal devices, phone and computers. It all kept chugging along, quite nicely.

    Since when is the ability to break half the internet with a simple mistake considered a useful feature of IPv4?

  15. Gerryb

    BT upgraded my router and enabled IPV6 without asking and destroyed the routing of my tiny system so I could no longer get to my office from the outside. Looked as if offline. Disabling IPV6 on the router fixed it.

    BT tech helpline were useless and saw nothing to apologise about.

    Is that what small offices and domestic users have to do... turn off IPV6?

  16. James_K

    "Is that what small offices and domestic users have to do... turn off IPV6?"

    Someone at BT clearly did something wrong, as IPv4 and IPv6 are different protocols. There are many, many people around the world who have no such problem.

  17. jpo234

    > Google currently claims that 28 per cent of its visitors are using IPv6. We don't buy it

    Akamai's numbers are in line with Google's: https://www.akamai.com/us/en/resources/our-thinking/state-of-the-internet-report/state-of-the-internet-ipv6-adoption-visualization.jsp

  18. David Crowe

    On the Amsterdam Internet Exchange IPv6 traffic is a remarkable 2.6% of all traffic over the past year. It appears to be declining as it was only 2.4% over the past month.

    https://stats.ams-ix.net/sflow/ether_type.html

    1. Nanashi

      That isn't an accurate measure of v6 traffic on the internet though.

      Here's another measurement: EE see about 70% of traffic from dual-stack devices going over v6. (Source: question at 16:50 of https://youtu.be/16FdlxyFQgY), which incidentally represents a cost saving to them.

      This isn't an accurate measure of v6 traffic on the internet either; my point here is that the value you get depends on where you measure. There's a lot of reason to believe that most v6 traffic avoids hitting the Amsterdam Internet Exchange - for example, I'd imagine most of Youtube's traffic goes over private Google links and then via peering directly with eyeball ISPs, and a lot of Netflix's traffic is from local cache boxes. Most v6 traffic is generated by big networks, and those are also the networks most likely to have private peering. Traffic over private peering isn't going to show up in AMS-IX's graphs.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like

  • Cloudflare menaces virtual desktops with isolated browser access to internal networks
    Gives cloudy email a kicking, too – but VDI should be safe in its bastions

    Cloudflare has added the ability to access private networks to its browser isolation service, and suggests the combo represents an alternative to virtual desktop infrastructure.

    Browser isolation requires organizations to have a Cloudflare Zero Trust account, and to install a client on users' devices. Cloudflare runs a browser in its cloud and users browse as usual – but Cloudflare intervenes so that users don't make it to whichever web server they intend to visit.

    Cloudflare browses to the server and then redraws the web page on the client browser. The user's device therefore never touches the web server, so anything nasty on a page is snuffed out by Cloudflare in its cloud instead of poisoning a local PC.

    Continue reading
  • Google, EFF back Cloudflare in row over pirate streams
    Ban akin to 'ordering a telephone company to prevent a person from having conversations' over its lines

    Google, EFF, and the Computer and Communications Industry Association (CCIA) have filed court documents supporting Cloudflare after it was sued for refusing to block a streaming site.

    Earlier this year, a handful of Israel-based media companies took Israel.tv to court, accusing it of streaming TV and movie content it had no right to distribute. The corporations — United King Film Distribution, D.B.S. Satellite Services, HOT Communication Systems, Charlton, Reshet Media and Keshet Broadcasting — won the lawsuit after Israel.tv's creators failed to show up to their hearings, and the judge ordered Israel-tv.com, Israel.tv and Sdarot.tv each pay $7,650,000 in damages. 

    In a more surprising move, however, the media outfits also won an injunction [PDF] in the United States in April against a slew of internet companies, among others, banning them from aiding Israel.tv in its piracy.

    Continue reading
  • Cloudflare explains how it managed to break the internet
    'Network engineers walked over each other's changes'

    A large chunk of the web (including your own Vulture Central) fell off the internet this morning as content delivery network Cloudflare suffered a self-inflicted outage.

    The incident began at 0627 UTC (2327 Pacific Time) and it took until 0742 UTC (0042 Pacific) before the company managed to bring all its datacenters back online and verify they were working correctly. During this time a variety of sites and services relying on Cloudflare went dark while engineers frantically worked to undo the damage they had wrought short hours previously.

    "The outage," explained Cloudflare, "was caused by a change that was part of a long-running project to increase resilience in our busiest locations."

    Continue reading
  • Cloudflare's outage was human error. There's a way to make tech divinely forgive
    Don't push me 'cos I'm close to the edge. And the edge is safer if you can take a step back

    Opinion Edge is terribly trendy. Move cloudy workloads as close to the user as possible, the thinking goes, and latency goes down, as do core network and data center pressures. It's true  – until the routing sleight-of-hand breaks that diverts user requests from the site they think they're getting to the copies in the edge server. 

    If that happens, everything goes dark – as it did last week at Cloudflare, edge lords of large chunks of web content. It deployed a Border Gateway Protocol policy update, which promptly took against a new fancy-pants matrix routing system designed to improve reliability. Yeah. They know. 

    It took some time to fix, too, because in the words of those in the know, engineers "walked over each other's changes" as fresh frantic patches overwrote slightly staler frantic patches, taking out the good they'd done. You'd have thought Cloudflare of all people would be able to handle concepts of dirty data and cache consistency, but hey. They know that too. 

    Continue reading
  • Cloudflare says it thwarted record-breaking HTTPS DDoS flood
    26m requests a second? Not legit traffic, not even Bill Gates doing $1m giveaways could manage that

    Cloudflare said it this month staved off another record-breaking HTTPS-based distributed denial-of-service attack, this one significantly larger than the previous largest DDoS attack that occurred only two months ago.

    In April, the biz said it mitigated an HTTPS DDoS attack that reached a peak of 15.3 million requests-per-second (rps). The flood last week hit a peak of 26 million rps, with the target being the website of a company using Cloudflare's free plan, according to Omer Yoachimik, product manager at Cloudflare.

    Like the attack in April, the most recent one not only was unusual because of its size, but also because it involved using junk HTTPS requests to overwhelm a website, preventing it from servicing legit visitors and thus effectively falling off the 'net.

    Continue reading
  • Man gets two years in prison for selling 200,000 DDoS hits
    Over 2,000 customers with malice on their minds

    A 33-year-old Illinois man has been sentenced to two years in prison for running websites that paying customers used to launch more than 200,000 distributed denial-of-services (DDoS) attacks.

    A US California Central District jury found the Prairie State's Matthew Gatrel guilty of one count each of conspiracy to commit wire fraud, unauthorized impairment of a protected computer and conspiracy to commit unauthorized impairment of a protected computer. He was initially charged in 2018 after the Feds shut down 15 websites offering DDoS for hire.

    Gatrel, was convicted of owning and operating two websites – DownThem.org and AmpNode.com – that sold DDoS attacks. The FBI said that DownThem sold subscriptions that allowed the more than 2,000 customers to run the attacks while AmpNode provided customers with the server hosting. AmpNode spoofed servers that could be pre-configured with DDoS attack scripts and attack amplifiers to launch simultaneous attacks on victims.

    Continue reading
  • Big Tech shrank the internet while growing its own power
    Classic internet ideas matter less now that CDNs and private networks dominate traffic

    Comment The internet has become smaller, the result of a rethinking of when and where to use the 'net's intended architecture. In the process it may also have further concentrated power in the hands of giant technology companies.

    Given the ever-expanding content and resources available online, and proliferation of connected devices, the notion that the internet has shrunk is counter-intuitive. But shrunk it has – to the point at which some iPhones do not immediately connect to the open internet.

    Those phones are iPhones running the latest version of Apple's iOS and the opt-in service called Private Relay. The iGiant bills Private Relay as a privacy enhancement because it obscures users' DNS lookups and IP addresses by funneling traffic over networks operated by Cloudflare, according to specs set by Apple.

    Continue reading
  • Cloudflare stomps huge DDoS attack on crypto platform
    At 15.3 million requests per second, the assault was the largest HTTPS blitz on record lasting 15 seconds

    Cloudflare this month halted a massive distributed denial-of-service (DDoS) attack on a cryptocurrency platform that not only was unusual in its sheer size but also because it was launched over HTTPS and primarily originated from cloud datacenters rather than residential internet service providers (ISPs).

    At 15.3 million requests-per-second (rps), the DDoS bombardment was one of the largest that the internet infrastructure company has seen, and the largest HTTPS attack on record.

    It lasted less than 15 seconds and targeted a crypto launchpad, which Cloudflare analysts in a blog post said are "used to surface Decentralized Finance projects to potential investors."

    Continue reading
  • Developer adoption is our priority, profits second, Cloudflare tells bankers
    We seem to give away stuff for free at just the right time, says CFO

    If Cloudflare CFO Thomas Seifert's take on his company's direction is accurate, expect future strategy to focus on how it can use its slew of newly announced tools to make the biggest dent in existing markets. Profit motivations come a distant second, as least for now.

    Speaking at the Morgan Stanley Technology, Media and Telecom conference, Seifert told analyst Keith Weiss that 2022 will be all about growing Cloudflare's Zero Trust solution as well as Workers, its serverless code platform.

    Even with those products, Seifert said, the security-focused content-delivery network's strategy isn't about earnings – it's about gaining users. "We think primarily about adoption in the developer community penetration and less about dollars and revenue at this point in time," Siefert told the audience of investors and financial analysts.

    Continue reading
  • Cloudflare, Akamai: Why we're not pulling out of Russia
    Yanking connectivity would do more harm than good, they say

    Though Cloudflare and Akamai have voiced their opposition to President Vladimir Putin's invasion of Ukraine, they have stopped short of pulling completely out of Russia despite mounting pressure to do so.

    In a March 6 statement, Cloudflare CEO Matthew Prince said his company, which provides DDoS protection and other internet networking and security services, has received "several calls to terminate" all business inside Russia. He added that "we've watched in horror the Russian invasion of Ukraine," adding: "Our thoughts are with the people of Ukraine and the entire team at Cloudflare prays for a peaceful resolution as soon as possible."

    That said, after discussing the situation with government and private-sector experts, Prince said Cloudflare concluded: "Russia needs more internet access, not less."

    Continue reading

Biting the hand that feeds IT © 1998–2022