CXL

AI Hardware Updated Sep 2026

CXL (Compute Express Link) Overview

A conceptual/reference doc on Compute Express Link — protocol stack, device types, coherency model, generational evolution through CXL 4.0, and how it compares to NVLink/UALink. This rounds out the memory section alongside hbm (accelerator-local memory) and HBM Memory Explanation.pdf/nand (system memory and storage): CXL is the piece that lets memory be attached, pooled, and shared across a fabric rather than wired directly to one die. Numbers below are industry-typical figures as of late 2026 gathered from the CXL Consortium and trade press — cross-check a specific platform's datasheet before relying on exact bandwidth/latency figures, since CXL 4.0 only became a published spec in November 2025 and host/device support is still ramping.

1. What CXL Is

Compute Express Link is an open interconnect standard that runs on top of the PCIe physical layer but adds cache-coherent, memory-semantic protocols on top of PCIe's plain load/store I/O. The core problem it solves: PCIe lets a CPU talk to a device, but every PCIe transaction is an explicit I/O operation — there's no notion of the CPU and a device sharing a coherent view of memory the way two CPU cores do. CXL adds that missing piece, so a PCIe-attached device can be treated less like a peripheral and more like an extension of the host's memory system.

This matters for exactly the two problems hbm and nand describe from the memory and storage sides: an accelerator with its own local memory needs a coherent way to share data with the host instead of costly explicit copies, and a system that has run out of DRAM channels/slots needs a way to attach more memory without adding another CPU socket. CXL was built to standardize both at once, replacing what used to be a fragmented set of proprietary or narrower efforts (Intel's older QPI-based coherency extensions, Gen-Z, CCIX, OpenCAPI — CXL absorbed or displaced all of them over 2019–2022).

2. Protocol Stack: CXL.io, CXL.cache, CXL.mem

CXL is not one protocol but three, multiplexed over the same physical link (a mechanism the spec calls Flex Bus, which negotiates at link-training time whether a given PCIe-electrical link should run as plain PCIe or as CXL):

Sub-protocol Role Transaction model
CXL.io Device discovery, configuration, interrupts, bulk DMA — functionally identical to standard PCIe Non-coherent, PCIe-style
CXL.cache Lets a device cache host memory coherently, and lets the host snoop/invalidate that device cache Coherent, host ↔ device cache protocol
CXL.mem Lets the host access memory that physically lives on a device (device-attached DRAM/HBM) as if it were host memory Coherent, memory-semantic reads/writes

A given CXL device implements whichever subset of these three protocols matches its role — which subset it implements is exactly what defines the CXL "device type" (§3). The physical link itself is ordinary PCIe electrical signaling (same SerDes, same connector families, same PCIe generation's data rate), which is why CXL could ride PCIe's existing ecosystem instead of needing new connectors, new cabling, or a new physical layer of its own.

3. Device Types

Type Protocols used Has local memory? Typical use Coherency direction
Type 1 CXL.io + CXL.cache No SmartNICs, accelerators that cache host memory but carry no attached DRAM of their own Device caches host memory
Type 2 CXL.io + CXL.cache + CXL.mem Yes (own DDR/HBM) GPUs, FPGAs, AI accelerators — bidirectional coherence between host and device memory Both directions
Type 3 CXL.io + CXL.mem Yes (DRAM or persistent memory) Pure memory-expansion/pooling modules — no compute, no cache Host accesses device memory as an extra NUMA-like tier

Type 3 is the device class getting the most near-term deployment attention, because it directly answers a problem every large server already has: DDR channel count per socket has grown much more slowly than core count and workload memory appetite, and adding a second CPU socket just to get more DIMM slots is an expensive way to buy capacity. A Type 3 CXL memory expander plugs into a PCIe/CXL slot and presents itself to the OS as additional NUMA-distant memory — capacity without another socket, at DDR-like (not HBM-like) bandwidth and somewhat higher latency than local DRAM.

4. Coherency Model: Host Bias vs. Device Bias

CXL.mem's core design challenge is letting both the host and a Type 2 device's own compute engine touch device-attached memory without constant coherency-traffic overhead for every access. It solves this with a bias-based coherency model — each region of device-attached memory is tagged as being in one of two states:

  • Host bias — the device's own memory is treated like ordinary host-coherent memory; every host access is safe by default, but the device must request/resolve coherency (essentially asking the host) before touching that memory itself.
  • Device bias — the host is guaranteed not to have a cached copy of that memory region, so the device can read/write it directly at full local-memory speed with no coherency round-trip; the host must go through the device if it needs that data.

Software (typically the device driver, sometimes hinted by the application) sets which regions are in which bias state, effectively choosing per-region whether to optimize for "host touches this often" or "device touches this often." The host always sees one consistent, correct view of the data regardless of bias state — bias is a performance optimization, not a correctness mechanism.

5. Generational Evolution

Generation Released PCIe base Data rate Headline additions
CXL 1.0 / 1.1 2019 / 2020 PCIe 5.0 32 GT/s Baseline CXL.io/.cache/.mem; single device, single host, direct-attach only
CXL 2.0 2020 PCIe 5.0 32 GT/s Single-level switching, memory pooling (one device shared across hosts sequentially/statically), persistent-memory support
CXL 3.0 2022 PCIe 6.0 (PAM4) 64 GT/s Multi-level switching, non-tree fabric topologies, peer-to-peer device-to-device transfers, hardware-coherent multi-host sharing (not just pooling)
CXL 3.1 2023 PCIe 6.0 64 GT/s Fabric management refinements, scale to larger node counts (thousands of ports in a fabric)
CXL 3.2 2024 PCIe 6.0 64 GT/s CXL Hot-Page Monitoring Unit (CHMU) for memory tiering, Trusted Security Protocol (TSP), Post-Package Repair, online firmware activation
CXL 4.0 2025 PCIe 7.0 (PAM4) 128 GT/s Doubles per-lane bandwidth again, up to ~1024 GB/s per CPU link aggregate, native x2-width full-performance links, port bundling, retimer support for multi-rack topologies

The generational split roughly falls into three eras: 1.x established the coherency protocols on a direct-attach link; 2.0–3.x built out switching and fabric so memory could be shared/pooled across multiple hosts rather than hard-wired to one; 4.0 is chasing the bandwidth and topology needs of large multi-rack AI clusters, where CXL is increasingly being positioned alongside (not just behind) accelerator interconnects. Each generation stays backward-compatible — a CXL 3.x device negotiates down to whatever rate/feature set the link partner supports.

6. Switching, Pooling, and Fabric

CXL 1.x is direct-attach only — one device wired to one host, much like a PCIe card in a slot. Starting with CXL 2.0, a CXL switch sits between hosts and devices, and this is what unlocks the capabilities most discussed in 2026 server planning:

  • Memory pooling (CXL 2.0+) — a pool of memory devices sits behind a switch, and hosts are assigned regions of that pool, typically at boot/allocation time rather than dynamically. This turns memory into more of a provisionable resource, similar in spirit to how storage was virtualized decades earlier — buy a pool, carve it up as workloads need it, instead of stranding unused DIMM capacity on whichever server happened to be provisioned with it.
  • Multi-host, hardware-coherent sharing (CXL 3.0+) — multiple hosts can coherently access the same memory region simultaneously (not just be assigned disjoint regions from a shared pool), which matters for workloads that want a shared coherent memory region across nodes without going through a network-based protocol.
  • Non-tree fabric topologies (CXL 3.0+) — earlier switching was strictly tree-shaped (one root, branching down); CXL 3.0 allows mesh-like fabrics with peer-to-peer device communication, which is a step toward CXL functioning as a general-purpose memory/resource fabric across a rack rather than a point-to-point expansion link.

Switch and controller vendors (XConn, Astera Labs, Microchip, Panmnesia, among others) are the companies building the pieces — switch ASICs, retimers, fabric managers — that this depends on; on the memory-device side, Samsung, SK hynix, and Micron all ship CXL 2.0-class Type 3 memory-expansion modules as of 2025–2026 (128–256 GB class capacities), with CXL 3.x/4.0-class parts following the spec's own rollout.

7. Memory Tiering

A second, related use case beyond pure capacity expansion: CXL memory is slower than local DRAM (extra hop through the CXL controller/switch adds real latency), so operating systems and hypervisors increasingly treat CXL-attached memory as a distinct, slower tier and place data accordingly — hot, frequently-touched pages on local DDR, colder pages on CXL memory, migrating pages between tiers as access patterns change. CXL 3.2 formalized hardware support for exactly this with the CXL Hot-Page Monitoring Unit (CHMU), which lets a CXL memory device track its own page access patterns in hardware and report hot/cold statistics back to the host OS, instead of the OS having to infer tiering decisions purely from software-side page-fault sampling.

This tiering framing is the more realistic near-term picture of how CXL memory gets used in practice: not a drop-in DDR replacement, but a second, larger, somewhat slower tier that a tiering-aware OS/hypervisor actively manages — conceptually similar to how an OS manages page cache between DRAM and an NVMe SSD, just one level higher in the hierarchy and with much smaller latency gap than the DRAM-to-SSD jump.

These three solve related but distinct problems, and the trade press increasingly discusses all three together because a large AI cluster typically uses more than one of them simultaneously:

Interconnect Primary job Topology Backing
CXL CPU ↔ memory expansion/pooling; device ↔ host coherent memory sharing Switched/fabric, host-centric Open standard — CXL Consortium (Intel, AMD, and the broader industry)
NVLink GPU ↔ GPU high-bandwidth interconnect within a tightly coupled cluster Dense, homogeneous GPU mesh/switch Proprietary — NVIDIA
UALink GPU/accelerator ↔ accelerator interconnect, explicitly positioned as an open NVLink alternative Scale-up accelerator pods, larger multi-vendor clusters Open standard — UALink Consortium (AMD, Intel, Astera Labs, others), 1.0 spec released 2025

The short version: NVLink and UALink both move data between accelerators at the highest bandwidth possible; CXL moves and shares memory capacity, primarily relative to a host CPU, though CXL 3.x's peer-to-peer device support blurs this at the edges. NVLink currently leads in shipped, production hardware and bandwidth-per-GPU; UALink is the open-standard answer to NVLink but was only ratified in 2025 with silicon expected through 2026–2027; CXL occupies a different part of the stack entirely and isn't in direct competition with either for GPU-to-GPU traffic, though all three now get planned together in AI data center architecture discussions.

9. Where CXL Sits in the Broader Memory/Storage Picture

   Compute die (GPU/XPU)
        │  (multi-TB/s, ns-scale)         ← HBM stacks: [hbm](https://globaltrendanalysis.com/hbm/)
   HBM stacks
        │
   CPU socket
        │  (tens of GB/s, tens-of-ns)     ← DDR: HBM Memory Explanation.pdf
   DDR5 local memory
        │
        ├──────────────────────────────┐
        │                               │  (extra latency hop via
   (local DRAM, hot tier)          CXL switch/fabric — CXL 2.0+)
                                         │
                                    CXL-attached memory pool   ← this doc
                                    (DDR-class BW, higher latency,
                                     shared/pooled, cold tier)
        │
   NAND-based SSD (persistent)      ← [nand](https://globaltrendanalysis.com/nand/), [ssd-controller-architecture](https://globaltrendanalysis.com/ssd-controller-architecture/)

CXL doesn't replace any tier already covered in this notebook — it changes how the DDR tier is attached and shared. Where HBM answers "how do I feed a compute array enough bandwidth" and NAND answers "how do I store data cheaply and durably," CXL answers "how do I get more memory capacity onto a system, and share it across hosts, without wiring it directly to one CPU's DDR channels." As of 2026 that's mostly playing out as capacity expansion and cold-tier pooling; the multi-host coherent-sharing and fabric features (CXL 3.0+) are the part of the spec still earliest in real-world deployment.

Sources

Contact

Have an Idea?
Let's Build It.

Whether you're a startup with a hardware concept, an established company needing deep design help, or an enthusiast with a wild project — we want to hear about it.

Emailaglucas@globaltrendanalysis.com
Phone(303) 921-4492
LocationBoulder, Colorado