How CXL may change the datacenter as we know it

Compute Express Link (CXL) has the potential to radically change the way systems and datacenters are built and operated. And after years of joint development spanning more than 190 companies, the open standard is nearly ready for prime time. For those that aren’t familiar, CXL defines a common, cache-coherent interface for …

[Anonymous Coward]

CXL == slower /tmp

CXL seems kind of useless for anything but a large cache blob like /tmp but, why not add more RAM and enjoy faster speeds? Can anyone give a real world example of where any data center said "I wish we could put RAM on the PCIe bus"? I've said it, but I'm not a datacenter and I'm not spending anything on futuring my hardware. However, every time I've said it myself I am not thinking about the wall clock, which seems like most _cloud_ data centers will.

Cloud costs will increase as existing configurations will remain the same price but using real RAM will become the "enterprise" option instead of the standard thus more $$$, similar to the "shared" and "dedicated" options we all know.

[DS999]

Re: CXL == slower /tmp

I've been pretty skeptical about CXL since I keep seeing people pumping it for all kinds of scenarios that make no sense to me, which is a red flag in the tech world. I've seen people claiming that servers would lose most of their DIMM slots and use CXL for memory expansion because they'd save pins on the CPU and board space in the server, which is about the dumbest idea I've ever heard since pins aren't at a premium and server density per rack is limited by power/cooling not by board size.

I do think the suggested use in cloud is probably the one place it may make sense, because you want the ability to move VMs around between physical servers but you don't want to spend money on equipping every one with the most RAM it can hold just in case it has to run a few memory hungry VMs. Having a pool of shared RAM out there that can be reallocated along with the VM's move avoids having to max out every physical server's RAM or limiting that VM to only running on the subset that have that maxed out RAM.

Now how common that is in the real world I have no idea. I've never dealt with hyperscale clouds but enterprise level VM farms seem to deal just fine with having physical servers of different configurations and restricting certain VMs to only running on a subset has never proven to be a problem. Even if you don't want to manage with a list of servers, something rule based "VM x can only run on servers with available RAM >= y" would seem to do the trick. I have to imagine Amazon et al have some pretty good automated management software that learns how much memory a given VM needs and can migrate it appropriately.

Time will tell whether CXL is the advancement some claim or a solution looking for a problem.

[Arbuthnot the Magnificent]

Re: CXL == slower /tmp

I work in HPC, we have an on-premises cluster used for large-scale numerical simulations. Composable memory fabric for our compute nodes is something we are interested in, since at any time we have jobs running with varying memory requirements.

[An_Old_Dog]

Re: CXL == slower /tmp

CXL will provide exciting opportunities in marketing, buzzword-generation, consulting, and certification programs!

CXL-attached RAM could accelerate multi-user timesharing systems, which I haven't seen in use since the mid-1990s.

Seriously, though ... if running multiple VMs on a single CPU is the new timesharing, CXL-attached RAM could allow more VMs per-CPU, but you could do the same by adding more conventional RAM to the system. Whether or not the CXL gear would be cost/performance-effective for this, vs just buying more nodes with their own CPU and dedicated RAM, is something we won't know until we have CXL hardware price info and benchmark results.

[Doctor Syntax]

No doubt even now the malware developers are studying the fine details.

[An_Old_Dog]

Got TEMPEST?

A super-duper-speed bus linking units in different racks sounds like an antenna waiting to be exploited to exfiltrate data from "secure, air-gapped" systems.

(Icon for data-escape.)