back to article There is a path to replace TCP in the datacenter

One of the most entrenched standards of the last forty years, the Transmission Control Protocol (TCP), might be seeing the end of the line, at least for applications in some of the world's largest datacenters. For the rest of the world though, the hassle factor of a shift might be too heavy to bear, even if 100X faster message …

  1. Sub 20 Pilot

    Translation.

    Working system that has just done the job for 40 years to be superseeded by some complex, process intensive turdulence that does NOTHING except crash regularly, fail to do the job and requires patching every week until it almost works when it will be replaced by something even worse.

    It works, leave the fucking thing alone.

    1. Anonymous Coward
      Windows

      Re: Translation.

      Ir's attitudes like that which keep COBOL programmers in business (and I should know since I benefitted from them).

      1. jake Silver badge

        Re: Translation.

        Good thing, too. COBOL programmers keep business world-wide running.

        1. chivo243 Silver badge
          Pint

          Re: Translation.

          Good thing, too. COBOL programmers keep business world-wide running.

          I was just telling my 12 year old this... Now where can I send him to learn COBOL??

    2. Doctor Syntax Silver badge

      Re: Translation.

      The job TCP was designed to do, has done and continues to do is to provide what looks like a reliable connection over a wide-area network which was not necessarily reliable and just fired packets around which is not a connection-oriented thing. There was always UDP as an alternative, relying on higher level protocols to fix up the reliability bit if it was needed.

      In an environment where the connectivity can be taken for granted TCP isn't necessary so they could have been using UDP anyway although I doubt that that's what he's suggesting.

      We use TCP/IP on our LANs because it's there and easier than having to worry about whether your printer is local rather than in head office 2000 miles away. It's worth remembering that your LAN's TCP packet, inside which your data sits, itself sits inside an IP packet which is designed to be routed over a WAN even though it's delivered locally inside and that sits inside an Ethernet packet. If this is using point-to-point fibre it won't need the IP packet or the Ethernet packet.

      Before that there were other networking protocols for local networks so in a sense it goes back to those days.

      1. Allan George Dyer
        Trollface

        Re: Translation.

        @Doctor Syntax - "easier than having to worry about whether your printer is local rather than in head office 2000 miles away"

        So no worries walking 2000 miles to pick up your printout?

        Printing is one of the few tasks where knowing the location of the device serving you is always going to be significant.

        icon - I'll just pick on the poorly-chosen example and ignore the significant points.

        1. J. Cook Silver badge
          Coat

          Re: Translation.

          Singing Oh, I would walk 500 miles, and I would walk 500 more, just to be the man to walk 1000 miles and go "PC LOAD LETTER?!?!?!?!? WHAT THE F-"

          Mines the one with The Proclaimers embroidered on the back.

          1. Robert Carnegie Silver badge

            Re: Translation.

            Letter, from America. :-)

            1. TRT Silver badge

              Re: Translation.

              Printed out on A7?

              1. jake Silver badge

                Re: Translation.

                20# bond, green bar, 14 7/8 x 11.

              2. Robert Carnegie Silver badge

                Re: Translation.

                No, more.

        2. jake Silver badge

          Re: Translation.

          Amdahl had a largish printing facility just off Central Expressway in Sunnyvale (Cupertino?) in the 1980s. Virtually every mainframe print-job generated within the company, world-wide, was printed at said facility, and then next-dayed to whatever office had requested the print job. They had several gawdawfulfast channel attached printers, and a fleet of trucks delivering & sending out paper. Was awesome to watch in full-swing, if you had adequate ear protection.

          However ... it sounded daft then, and still sounds daft now.

          1. John 104

            Re: Translation.

            LOL. Sounds very British...

        3. eldakka

          Re: Translation.

          > So no worries walking 2000 miles to pick up your printout?

          I might not be the one the printout is for.

          That's the thing with computers and WANs, what you are doing may not be local to you. I've had cases where someone in a remote office has contacted me for some form, and I've provided the form to them by sending a print job of that form to their local printer in their remote-to-me office 13000km away from me, but 13 meters away from them...

        4. Doctor Syntax Silver badge

          Re: Translation.

          "I'll just pick on the poorly-chosen example and ignore the significant points."

          Whoooosh!

          edakka saw the point even if you didn't. The print-out might not be for the user who generates it, it might be for someone else in a different place. They don't want to walk 2000 miles over here to pick it up.

          Fair enough, the use case might be a picking list to be sent to the warehouse printer but it might still be 10 miles away. Even if it's just next door they don't want to walk over to collect it.

          1. jake Silver badge

            Re: Translation.

            I used to send source-code listings and other such tat to the much faster line printer at the school. It was quicker to go pick 'em up on my bike[0] than it was to wait for my little dot matrix to do the job. Cheaper, too ... I didn't have to pay for paper & ink.

            [0] Not the car ... the Bultaco allowed me to take the shortcut through San Francisquito Creek ... I'd probably get tarred & feathered and run out of town on the rail if I tried that these days.

            1. Anonymous Coward
              Anonymous Coward

              Re: Translation.

              A Remote Job Entry terminal for a customer in Africa used a state-of-the art 2400bps modem link. The users in an office on the other side of a small town could input jobs on cards - and get their output on the printer.

              Their previous method had been a man on a butcher/grocery delivery style bike with a large front basket. He took great delight in being able to turn computer jobs round faster than the RJE terminal.

      2. Roland6 Silver badge

        Re: Translation.

        >We use TCP/IP on our LANs because it's there and easier than having to worry about whether your printer is local rather than in head office 2000 miles away.

        I think you are getting confused, yes we use TCP/IP across our LANs and WANs, however to resolved the local/remote printing problem we use services based on mDNS such as Bonjour, which locate devices such as printers on the same network segment as your computer; the computer transfers the print job to the printer via TCP/IP using Airprint, Mopria etc..

        The only problem is with RDS servers either not correctly detecting the site a user is connecting from or the user selecting their favourite printer, forgetting they aren't actually at that site...

      3. Pomgolian
        Trollface

        Re: Translation.

        "easier than having to worry about whether your printer is local rather than in head office 2000 miles away"

        https://www.theregister.com/2022/07/26/windows_10_printer_bork/

        With the latest Windoze updates, a 4,000 mile round trip is the least of your problems.

        1. The Oncoming Scorn Silver badge
          Paris Hilton

          Re: Translation.

          Somewhere in the bowels of El Reg (Who Me\On-Call?) there is a story of updating a printer driver on a remote users PC & printer over some issue. The remote tech had tried every driver without resolving the issue to the users satisfaction, as he reached the end of his tether he tried a different driver from the norm & said "OK lets see how this one is!" prior to hitting apply.

          It's still the same!

          I haven't sent the page yet..........Are you still looking at the first printout?

          Yes - Why?

      4. MJI Silver badge

        Re: Translation.

        I remember IPX

        That was quite good

        1. Sudosu Bronze badge

          Re: Translation.

          It worked much better for Starcraft when playing head to head back in the day.

    3. Brian Miller

      Re: Translation.

      No, not quite at all.

      My secondary shell is Wireshark, and I've been doing network programming since, ah, 1992 or so. There is much to be desired about what is going on upon the wires. The TCP start takes three packets before the actual data stream starts. If it's a secure connection, then more packets are exchanged. As the stream progresses, acknowledgement packets must be sent back relatively frequently, enough to be a burden on the traffic.

      The RFCs have plenty of solutions that have been tried over the years. Simply using UDP can be just fine, but all of this takes programmers who really know their stuff. Bluffing doesn't cut it with network performance.

      I haven't read all of John Ousterhout's paper, but there isn't anything in there about HOMA being published in an RFC. At least there's a GitHub project: https://github.com/PlatformLab/Homa

      1. Roland6 Silver badge

        Re: Translation.

        > but there isn't anything in there about HOMA being published in an RFC.

        There does seem to be very little actually published about the actual details of the HOMA protocol specification, as you intimate with the GitHub link, it does seem the protocol specification is the source code...

        Aside: I would be interested to know whether HOMA includes QUIC functionality.

        1. Anonymous Coward
          Anonymous Coward

          Re: Translation.

          FFS quic is just re-inventing TCP over ip in stupid ways

      2. martinusher Silver badge

        Re: Translation.

        The part that most programmers don't get about TCP is that its a byte oriented protocol -- it might exchange network packets but internally the protocol is managing a byte (octet) sequence. This has enormous implications. One is "when do I send a packet?". If you send one every time the software wants to send a byte then you'll dramatically increase the network traffic. So the fix is 'activate a timer for x mSec and send packet data after that timeout or when the send buffer has over a threshold number of bytes in it". This leaves you juggling latency and efficiency. Then the listener receives some data...is it all the data or is it just part of the block of data that the sender wanted to send?

        If you think of it as something that's great for a serial type terminal using a network then everything's fine. You're typing lines that don't happen very fast and your data is automatically framed (<CR>). Same with terminal output -- its just a block of characters that are displayed as they're received.

        Computer to computer talk requires a lot of this implicit stuff like framing and timeouts to be made explicit. Over the years there's been numerous bodges to make it all work, sort of, but they invariably are trade offs -- latency goes up, excess traffic is generated, you name it. There's also the fact that you have to install a fairly complex stack to support TCP on remote devices -- the IoTs of this world -- and with that stack comes all sorts of security issues. You can provide similar capability for UDP with a tiny piece of bomb-proof code (and you don't even need an OS...).

    4. martinusher Silver badge

      Re: Translation.

      It actually doesn't work, and I wasn't the only person to discover this 40 years ago. It was a handy full duplex serial line replacement but its implementation always suffered from the problem that it was unstable outside the lab. Various and sundry paches over the years have made it a bit of a monster that kinda-sorta works but it still can't get over basic problems such as it being intolerant of dropped connections and programmers writing clunky DiY datagram protocols to run on top of it.

      Let's face it, the only reason its popular is that programmers just open a socket, connect and send stuff. They don't know what's going on under the hood and just point fingers when things don't work as they expect it to (and, unfortunately, it does quite often -- its only when links are unreliable or have longer latency or smaller MTUs than the ones they're used to that it falls apart.....but then its always someone else's fault).

      Incidentally, TCP is really process intensive.

  2. Ken Hagan Gold badge

    For most programs, the abstraction offered by TCP is just a reliable stream (pipe) between two endpoints that can be addressed by a short piece of text.

    If you have the source code, it really wouldn't be hard to slip in an alternative "reliable stream" transport protocol.

    1. bazza Silver badge

      I think it interesting that they've identified a stream transport as a problem. If one looks at how we actually use networks, it's via libraries like ZeroMQ, or (as the paper / article states) some sort of RPC, etc. All of these synthesize a message passing paradigm on top of a stream protocol. Even "streaming" of media these days boils down to discrete https transfers of files - messages - rather than having a continuously open socket through which bytes are streamed. Programming with a raw socket is a battle for the developer to synthesize a message protocol on top of tcp.

      So it is perhaps a good idea to bump that message paradigm down a few layers, get some benefits from having done so.

      The peak of folly will be somewhere having to transistion between Homa - a message orientated transport - to http - a message/file orientated protocol built on top of a streaming transport like tcp. Could make for a very busy edge of one's data centre.

      1. ebyrob

        ZeroMQ is it. I was going to bring this up myself but you already nailed it. I don't know why anyone would use some "HAMA" when tools like ZeroMQ are available. And thing about ZeroMQ is, you don't have to run it on top of TCP. Implement it exactly as you want to take advantage of your network microcosm.

        That's it. These problems are already solved. and RPC of all things is hardy generic data transport. So you want to write an application without using TCP? Go right ahead. It sure as hell isn't going to replace TCP!

        Perhaps the problem is we don't have 7 whole layers to play with any more like we did with OSI.

        1. Roland6 Silver badge

          If I understand correctly ZeroMQ doesn't actually do transport, hence HOMA would be just another transport sitting under ZeroMQ.

        2. Spamfast
          Stop

          Perhaps the problem is we don't have 7 whole layers to play with any more like we did with OSI.

          We never did.

          OSI was designed for circuit switched networks aka PSTNs. Packet switching was bolted on afterwards and to the best of my knowledge there was only one practical implemementation using OSI from the ground up - DecNET OSI - which went the same way as every other alternative to TCP/IP for systemwide high-level networking.

          Packet switching predates OSI and TCP/IP was developed contemporaneously if not slightly before.

          The only useful thing to come out of OSI was the pretty layer diagram and even that doesn't fit TCP/IP or most other networking technology very well.

          1. Roland6 Silver badge

            The only useful thing to come out of OSI was the pretty layer diagram ...

            When I worked on OSI back in the day, it was always my understanding the reference model was more about the presentation and communication of the functionality required, it wasn't prescriptive about the implementation. I took the work being done by ISO on the individual layers to be just one implementation and if anything the Mk1 implementation ie. a learning implementation. Even in the late 1980's there were protocol implementations that combined layers eg. X.400 and others that 'broke' the strict layering rules eg. SNMP and X.500.

            Looking back, it is clear the reference model is now a bit dated - probably because of its focus on the point-to-point, if we look at services such as QUIC (which implements Session layer functionality) we see OSI had little on one-to-many networking, with thinking largely restricted to multicast streaming and nothing on the orchestration of streams (which is QUIC does) and I suspect the operation of swarms and other highly dynamic networks also don't fit well into the model.

      2. Greybearded old scrote Silver badge

        I worked on a project that used ZeroMQ recently. We found many messages getting lost, and when I looked at the docs I found that it explicitly drops stuff on the floor under heavy load. (I'd provide a link, but I can't be arsed.)

        I can't imagine a use case where that's acceptable.

        1. Androgynous Cupboard Silver badge

          Always drops? I ask because MQTT (also message based) lets you identify your message as "send it once or maybe more times", "send it once exactly" and "send it once and make sure it gets there" (each with increasing amounts of overhead to manage the transmission). I'm surprised ZeroMQ doesn't have that option, bit of a shame if it doesn't - I looked at it for a project, and it looked very promising.

          As for this protocol in the subject of the article, I read his paper - I'm a developer not a network guy, but I recognised a lot of truth in it. There is very little pure stream communication done these days, most of it is in chunks and designing a low-level protocol to reflect that makes a lot of sense to me.

          Get it done, slap a POSIX interface on it and we'll find a use for it, no question.

          Edit. For comms between VMs on the same hardware I imagine this would absolutely smoke TCP.

          1. Greybearded old scrote Silver badge

            I couldn't find a way to make it be sensible. Our half-arsed roll-your-own replacement turned out much faster anyway. Despite unspeakable abuse of a database that Is Not Meant for such a thing.

            1. Androgynous Cupboard Silver badge

              > half-arsed roll-your-own replacement turned out much faster anyway.

              Good man :-) It's very, very hard to beat a tool built for a single purpose.

      3. Abominator

        ZeroMQ is a hunk of shit.

  3. JimC

    Multiple stacks

    I'm old enough to recall multiple network protocols on PCs and servers. It wasn't that big a deal, and with modern hardware ought to be straightforward enough. Its very easy to believe that there can be a better protocol than TCP for short range high speed networking.

    1. Warm Braw

      Re: Multiple stacks

      In many ways, it's surprising TCP has survived as long as it has - and more so that we've got this far with essentially one network protocol and one transport protocol and that they've provided a sufficiently adequate basis for everything from e-mail to high definition video streaming and the transition from noisy, slow analogue lines to fibre.

      It's not just that there are transport protocols better suited to data centre operations. IP (whether v4 or v6) combined with Ethernet framing (and the consequent need for mapping between datalink and network layer addresses) is less than optimal too. The requirement is for something that functions much more like a bus or backplane interconnect than a general purpose network.

      One of the (ultimately doomed) innovations of DECnet Phase V was that the equivalent of DNS recorded the protocol stacks available on the target host (at least a network layer and transport layer) and the resolver would return the protocol stack(s) that were mutually supported by the communicating parties along with the addressing information for each layer. Sounds like an idea whose time might finally have come.

      1. adam 40 Silver badge

        Re: Multiple stacks

        Probably it has survived is because it fixes a shitty hardware implementation underneath.

        So the corollary is HOMER (sic) had better have 5 9's or better hardware underneath it.

      2. Fred Goldstein

        Re: Multiple stacks

        TCP is 48 years old now and long past its sell-by date. IP split from "The TCP" in 1978 and is even more senescent. Moore Law keeps it going. While not widely implemented yet, RINA (Recursive InterNetworking Architecture) makes more sense for the future, as among other things it is more scalable and secure. DECnet was more advanced than TCP/IP but didn't have the ARPANET (free if you could get on it at the time) and Berkeleycode (free as in beer) to make it easy to adopt in spite of its flaws.

    2. david 12 Silver badge

      Re: Multiple stacks

      Its very easy to believe that there can be a better protocol than TCP for short range high speed networking.

      There always was. For those of us doing database programming using the MS native API's, the move to put SMB on top of TCP/IP at the turn of century was a move backwards, only partly relieved by the move from 10MB networking to 100MB networking.

    3. Paul Hovnanian Silver badge

      Re: Multiple stacks

      This, exactly.

      I remember when TCP/IP ran alongside NetBIOS, with some other DEC and IBM protocols. All on top of our Ethernet LAN. The whole "TCP/IP is entrenched" goes against the whole protocol agnostic LAN design philosophy. Go ahead and implement SANs using protocols better suited to co-located servers and disks where nothing more than LAN (or VLAN) technology is needed. TCP/IP can just plod along next to it for the connections that need to find their way through routers, across heterogeneous networks and maybe out into the world.

      1. Anonymous Coward
        Anonymous Coward

        Re: Multiple stacks

        Yeah.Too fucking right. NetBIOS, DECnet and IBM stuff has sucededed where IP seen proven to fail -- repeatedly. .Add ironicity to taste: if when reality finaliciity influence real world eality.

        1. jake Silver badge

          Re: Multiple stacks

          I think you just b0rkeded my parser ...

    4. ITMA Silver badge

      Re: Multiple stacks

      IPX/SPX, Token Ring anybody? LOL

      Have used both.

      What protocol did the original Banyan Vines use? I can't recall...

      1. jake Silver badge

        Re: Multiple stacks

        "What protocol did the original Banyan Vines use?"

        Banyan VINES ... The acronym expands to Virtual Integrated NEtwork Service.

        It was based on Xerox's XNS, as was Novell NetWare's IPX.

        1. ITMA Silver badge

          Re: Multiple stacks

          Cheers!

          It was a loooonnnggg time ago when I used it.

        2. Roland6 Silver badge

          Re: Multiple stacks

          >It was based on Xerox's XNS

          Not done a comparison recently, although still got the specifications in the attic, but I suspect there is still much useful (to users) functionality contained in XNS (and Apollo Domain) that hasn't been incorporated into the de facto Internet protocol suite.

  4. DS999 Silver badge

    If they are doing message passing

    Why not use UDP if the congestion avoidance and resending of packets is their issue? Or is it is just the congestion avoidance but they want the resending of packets to happen in the rare cases they are lost, RDP (reliable datagram protocol)

    What are the advantages of their new thing supposed to be, aside from not being supported by any layer 3 switch in the world?

    1. This post has been deleted by its author

      1. Old69

        Re: If they are doing message passing

        "Removing the need for every packet to be acknowledged by the receiver."

        TCP usually acknowledges periodically rather than every packet - with a cut off time if there isn't a continuous flow of data. It is an implicit acknowledgement of any previous packet up to that one. The window mechanism allows dynamic tuning of occasions when the sender might have to pause its traffic waiting for an acknowledgement.

        With TCP the Devil was always in the detail for any particular use.

    2. Anonymous Coward
      Anonymous Coward

      Re: If they are doing message passing

      > What are the advantages of their new thing supposed to be, aside from not being supported by any layer 3 switch in the world?

      At a guess and knowing how corporates work, it'll look good on someone's performance review on the way to a promotion / pay rise.

    3. Anonymous Coward
      Anonymous Coward

      Re: If they are doing message passing

      Essentially, TCP is a pickup truck, and can only be a pickup truck, serviceable for almost any transportation need but rarely best for any of them. A dump truck can haul more, a bus will carry more passengers, a sports car is faster, a motorcycle is more efficient, etc.

      A better protocol could be a supercar when speed is needed, or a semi truck when volume is favored, or an armored truck when reliability is critical, or a motorcycle when resources are limited.

      Imagine a protocol that can be optimized, or even better will self optimize for everything from SAN storage, to high latency satellite, to low reliability shortwave radio connected weather stations, and low power IoT devices. That protocol is definitely not TCP... Yet we use TCP in all of those places.

      UDP is great, until you need one of packet reordering, or error correction, or retransmission, or.... Then your stuck rolling your own, or just using TCP which may include a bunch of stuff you don't need that only slows you down.

  5. bazza Silver badge

    Half a Job?

    My title isn't fair - replacing Ethernet would be a major, major change.

    These days the key thing that makes performance increase is development in switch chips - be it for Ethernet or Infiniband or whatever. The price of developing a new one is very high, and there needs to be a very large market to justify it.

    There are alternatives to Ethernet, e.g. RapidIO. This always used to outstrip contemporary Ethernet, and was found largely high performance embedded processing systems, often for military applications. It reached a dead end, because to develop a new version of it and actually design the necessary switch chips was too much for the military market to bear. So, now it's all Ethernet. But it's a pity - with a proper peripheral for it transfer latencies were 75ns (from sending application's memory buffer, to the first bytes starting to show up in the destination application's memory buffer on another computer).

    When one looks at the most impressive super computers - e.g. Japan's Fugaku - the thing that makes them stand out is the inter-compute node transport. For Fugaku, the transport (called Tofu) is built directly into the processor chips, hotwired to the memory bus, with huge bandwidths, lots of point to point interconnect topology, and lowest possible latency.

    Each processor node is also a network switch, in the 6dimensional hypertoroid and 4dimensional hypercube of point to point links between processors. It's bumping the job of network routine all the way back up to the application layer. When one thinks about it, a classic Ethernet router is kind of a sop for developers; packets fired into the network will eventually get to where they're going, but the developer isn't telling the router in advance what to expect, it has to work that out as and when stuff turns up. Whereas making the software developer do the routing themselves means that the routing that gets done is optimisable around the application needs. This is hard on developers - Fugaku and K were / are apparently quite hard to develop for - but that's where you have to go to reach ultimate peforamance.

    You can think of a super computer these days as the ultimate design of a datacentre, and data centres are going to have to copy them.

    1. Greybearded old scrote Silver badge

      Re: Half a Job?

      I think you've got slightly mixed up. TCP != Ethernet

    2. l8gravely

      Re: Half a Job?

      Sure, you can get better performance with custom software and hardware. Homa's bet is that by targetting a specific niche area with known performance issues, and network latencies, and by putting the code into a library that's already well understood, you can gain alot of benefits.

      It's not meant as a general replacement for TCP/IP, especially not at the desktop. And it's not meant to replace TCP in all hyperscaler instances either.

      But for google/FB/AWS/HPC groups, if you can update a library and get better performance and not have to re-write your applications, that's a huge win. But this only works for a very specific use-case.

      TCP/IPs strength is that it works over 5 feet as well as it works over 15,000 miles, without any change needed to the program. But as someone who spent alot of time evaluating different tools (bbcp, lftp, rsync_parallel, etc) and various WAN accelerators (Silverpeak, and some others I forget about now) it's not an easy problem. TCP suffers horribly from the Bandwidth-Delay-Product, which means that as bandwidth goes up, the default size of buffers needed for in-flight packets rises quite a bit as well. So you either increase your buffers, or your goto parallel streams, or you have applicances which handle this at the edge for you and basically lie to the application that their packets made it there and to just keep sending data.

      So that's one instance (long distances, high RTT, high bandwidth) where TCP fails.

      But Homa is attacking the issues at the other ends of the scale, sending data 100m between lots of different hosts. And efficiently using the link speed we already have, isntead of just buying ever faster (10mbs/100mbs/1g/10g/25g/40g/100g/400g) ethernet links. Who else remembers FDDI? Woot! Token Ring? It got high utilization rates out of existing bandwidth.

      Ethernet wins because it just throws switching silicon at the problem. Which at short distances doesn't really address the problem Homa seems to be trying to solve.

      I for one would love to see something like this get deployed, I'm sure all the big guys are looking into something like this because if they *don't* have to replace TOR and Core switches all the time to get better performance, then that's a win. They can amortize the huge cost over a much huger installed base.

      And yes, Homa will never be a generic replacement for TCP/IP, it's not meant to be.

      No one runs Infiniband over WAN links, it's not what is solves, but people don't whine about how Infiband can't be used to send cat pictures from the UK to NYC.

  6. Anonymous Coward
    Anonymous Coward

    Nothing new under the sun

    TCP was the thing we "slipped in" to make local calls over simple sockets work of the long distances of the ARPAnet. The HPC crowd(called them kids at the time, when I was still a younger man) have been doing the same thing for the various MPI generations since forever. If you can think of a wacky high bandwidth technology, they have ported a stack to it a tested it at least. SerDes? Who needs the other side of the SFP, check. PCI-E? Just build an interposer to bridge those lanes, check. Thunderbolt 400 cables in a compact 3-d torus connecting nodes in a Beowulf cluster? The cables were damn short but it was fun in the late 90's, 10-4 on the 1394 good buddy.

    I remember the team having to explain to a Java programmer that had joined us that there was more than one kind of socket, and that UNIX sockets were a thing. If you haven't already been using short haul technologies, you may have left a ton of performance on the table. That said, many of these solutions were very brittle, taking custom drivers on specific hardware, or were eye wateringly expensive to buy as off the shelf parts, but there are 31 flavors of socket/RDMA/RPC/MPI layers out there already.

    If this new proposal leads to a fast local link technology in the 10-30m range that is reasonably priced and not direct link only, it may be a welcome addition, though in the end people will just write a TCP stack on top of it, so you may as well assume you have a three piece project, a transport layer(one that isn't solely optimized for TCP), a fast message passing stack that favors smaller messages that may sacrifice some things like order of delivery or reliable delivery, and because some idiot will do it anyway, am overlay stack that provides ip services, as you will end up bridging these zones somehow and hand having to route them in a virtualized network, even if there is a performance hit.

    From the paper, I feel like the author scores there best hits with discussion of the problems cause buy the streaming model and the legacy of the OSI layer model. Performance dictates that devices close to wires and metal benefit from being able to make smart choices for routing, instead of being dumb and blind and leaving those decisions to higher levels of the stack. While the claims TCP is "beyond repair" are an apparent attempt to get cheap heat, I agree that it's cleaner to design a parallel stack that people can choose to use as needed then to try to force TCP to change to meet their specific demands. The long push is getting buy in from enough big players to make this version relevant, stable, and reasonably priced. Though due to the choices the author made for the Homa stack there may be reasons to rely on other link architectures to push large messages or data streams. That and there a couple of places where the protocols assumptions could lead to fun behavior in the event of a bad acting host. In any event, I think what we are building towards is a network with more parallel transports that are well develped. Right now TCP can be generally assumed, and UDP is similarly pervasive. We need to raise the best of breed of some of the others to the same level, not sure if Homa will make the cut, but it seems designed to be good where the others are weakest, so there may be a place for it or something very like it. That said, there are standards aplenty.

  7. MikeTheHill

    Only for ipv4 networks inside a data-centre

    While the title makes it obvious, some people might have missed that the goal is only to replace data-centre traffic as Homa is current ipv4-only and relies on things like DSCP markings to survive transit between end-points.

    There is *no* suggestion that Homa will replace wide-area TCP.

    1. Roland6 Silver badge

      Re: Only for ipv4 networks inside a data-centre

      >Homa is current ipv4-only

      Personally, I would have explored putting HOMA directly on top of LLC and utilised network switches layer-2 routing capabilities.

      Okay this would limit the size of network/DC but if speed really is important...

  8. FrenchFries!

    Hmm...

    Isn't this why we use other protocols inside the DC? Sounds like he's just spouting his PhD. That's what prof's do!

  9. Anonymous Coward
    Anonymous Coward

    Yeah, yeah

    Considering that the IPv4 pool ran out years ago, that IPv6 had been around for decades, and that something like GitHub.com still doesn't support it at all*, what conclusions is one to draw about claims of revolutionary changes to our current global network infrastructure?

    * Which is especially daft as it's often relied on by "serverless", err… servers to pull packages from (NPM, Go and the like), which means paying for what's pound for pound the most expensive part of these public cloud setups**: an IPv4 address, when otherwise you could run the lot on IPv6.

    ** As for the "someone else's computer" argument, when running untrusted loads that comes in rather handy. :)

  10. Greybearded old scrote Silver badge

    ???

    Back when I were a brown-bearded mature student (OU FTW) scrote we learned about the OSI 7 layer model. Now I know TCP/IP blurred some of those together, but the application caring about anything at or below layer 6, WTAF?!

    This is why we can't have nice things.

    1. Androgynous Cupboard Silver badge

      Re: ???

      The OSI 7 layer model (other works of fiction are available).

      ZFS is a great example of why punching through layers is a good thing to do if it's holding you back.

    2. Fred Goldstein

      Re: ???

      The OSI 7 layer model was a mistake, taken from early Honeywell work and intended as a model for dividing the work among subcommittees in ISO/IEC JTC1. TCP/IP is not even that smart, though; while it left out the mistaken layers 5 and 6 (which were nulled out in the later, little-known "fastbyte" OSI option), it didn't have clean layering at all and made various mistakes about role assignments. Of course that's all they teach in most schools, so it's like studying political science in North Korea with an elective about China.

    3. Nate Amsden

      Re: ???

      I don't think TCP/IP blurs the OSI model. TCP/IP is two different layers you are referring to them as one. TCP (layer 4) and IP (layer 3).

      applications usually care about layer 7 only.

  11. John 104

    Not supporting or trashing this idea. However, we used to use IPX and managed to move away from that, we used modems and managed to move away from that. If there really is a benefit to the average user with this new technology, then I would expect it's adoption to occur over time. If not, it will be relegated to datacenter ops and most of us won't be the wiser.

  12. Plest Silver badge
    Joke

    Hmmm....

    When 90% of the world is just passing streamed vids of cats and kids falling off things, do we really need faster networks?!

    In fact, force Joe Public back to dial up speeds again and the rest of us actually doing useful stuff on "da inner-webs" can get out stuff done without worrying about morons chucking their life stories at each other over Facebook, Twitter and TikTok!

  13. Nate Amsden

    What interesting datacenter applications?

    The article concludes with the person who wrote this new thing claiming their new protocol will allow interesting data center applications to use networking better. What sorts of applications? How much better and in what scenarios/speeds?

    Please don't tell me these applications have anything to do with Web3, or blockchain BS.

    Things seem to be working just fine with TCP, and in cases where that may be too much overhead we have UDP too.

    One area where TCP may not be so well is tons of tiny connections(I used to manage an app that would sustain about 3000 HTTP requests/sec and it overflowed the # of ports on the linux systems until we got the load balancer to pipeline the requests which dramatically lowered overhead and improved performance), but in that case the solution is to keep a pool of connections open and send requests over them rather than open/close in rapid succession.

  14. John Smith 19 Gold badge
    Coat

    Not read article but I'm thinking......

    Asynchronous Transfer Mode rises again (maybe)?

    Small, fixed sized packed (58 bytes?) designed by telcos to be the future especially of digital telphony (not VoIP).

    Just my random musings.

  15. StrangerHereMyself Silver badge

    Ludicrous

    In my opinion it would be ludicrous to alter the internet protocols to favor a few Big Tech Cloud Providers. If they want to use their own protocols inside their data-centers that's fine with me, but don't even think about pushing these protocols into the wider world.

  16. KSM-AZ

    Seems a little silly...

    If you want maximum speed drop to layer 2. As long as you are at l4, you have to endure more overhead. If all your traffic is in the same layer2, ... Early on in the ip vs ipx wars... There is already a bunch of l2 protocol management going on as well.... lldp, spanning tree. To get data from point a to point b you need a handle on both ends. Mac addresses with ethernet. Not sure how you are gonna improve on switching that with asic's. Never did understand why usb was so popular for perhipherals, when the cost of a much faster ethernet chip was the same price. 8 port switches were $15, build it in. Ataoe never took off, but iscsi did. Designing a new use specific lightweight l3 would be ok, but why stick with ip at all? Customizing a new l2 seems dumb. (Can you say fiber channel?). Then again 802.1x...

    This whole thing is a bunch of hummice homa.

  17. Spamfast
    Stop

    El Reg's Irrational Downer On TCP?

    a protocol that's over the hill and under the gun

    Didn't we have this a few weeks ago in another El Reg article which was thouroughly debunked by the merry throng?

    TCP is still amazingly good at what it's supposed to do desptie the fact that the transports over which it is carried have changed significantly over the past forty years, the older ones of those having most certainly become surplus to requirements. It is also very simple to implement.

    That it is not suitable for certain niche applications is not news and does not mean it is broken or superannuated.

    Ms Hemsoth & her editors need to get a grip.

  18. spireite Silver badge

    Proven technology

    Like others in old enough to have used Broken String, and Novell NetWare implementing NetBIOS, IPX/SPX and the loading the TCPIP nlm.

    Back then TCP/IP on Novell was a bit flaky.

    I also found multiplayer Doom/Wolfenstein/Quake. (one of them) was much better

    1. ITMA Silver badge
      Devil

      Re: Proven technology

      "I also found multiplayer Doom/Wolfenstein/Quake. (one of them) was much better"

      That was our standard network "stress testing tool" LOL

      The trick was (with three or more players in Quake Deathmatch) to load the Bombs8 QuakeC weapon patch and not tell most of the other "testers".

  19. ITMA Silver badge

    Flashers!

    This has vague overtones of Flash Boys by Michael Lewis.

    Not just trying to get your links to exchanges data centres as fast a possible and thus latencies as low as possbile - but even going so far as to buy/bribe you way to get your servers as physically close as possible to the exchange's servers WITHIN the same data centre.

  20. grumpy-old-person

    Don't fix it if it not broken

    While TCP/ip may have many deficiencies it also has many positive aspects.

    Replacing it with something else would require a lot of thinking that makes the proposed replacement infinitely better than TCP.

    After all, if you are old enough to remember what it was like in the days of X.25, SNS, BNA and so much more that made it so VERY difficult to connect systems - TCP/IP changed that in a dramatic way, allowing just about anything to connect to anything.

    A replacement needs to have similar positive aspects while providing a significant improvement in performance.

    Not so simple, and very difficult to do.

    1. Roland6 Silver badge

      Re: Don't fix it if it not broken

      > TCP/IP changed that in a dramatic way, allowing just about anything to connect to anything.

      From memory, this only really came about because of Dan Lynch and the series of intensive, behind closed doors, no media access, technical Interop events he started in 1986. I suggest the key events were '86 through to '88, which the MAP/TOP/OSI Interop events paralleled (*).

      What these events meant in the everyday world was that those TCP/IP implementations shipped (for free!!!) with Unix systems, could readily communicate with other TCP/IP implementations over Ethernet/802.3 LAN, such as those for IBM mainframes etc. With newly released OSI implementations (late 1988) carrying a price tag and having no install base, it effectively won the battle against both OSI and proprietary networking suites. Obviously, it took a few more years of lobbying before Microsoft and Apple terminated their proprietary networking and also put their weight behind TCP/IP; just in time for them to take full advantage of Tim Berners-Lee's work on the WWW.

      (*) I have no idea whether there was some "copying" of ideas or whether because interoperability was a big issue back then it was totally logical to hold multi-vendor interop events and thus the parallelism was just to be expected. However, I am certain, that without these series of events we would not have had fully interoperable implementations of TCP/IP and/or 7-layers of OSI by the close of 1988 and so made interoperability a given rather than an aspiration.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon