How Nvidia learned to embrace the light in its quest for scale
The GPU king's move to optical scale-up was inevitable
If you thought Nvidia's GB200 rack systems were big, CEO Jensen Huang is just getting started. At GTC last month, the world's most valuable company revealed plans to use photonic interconnects to pack more than a thousand GPUs into a single mammoth system by 2028.
The company isn't waiting to secure supply chains either. Over the past month, the GPU giant has invested billions in companies specializing in optics and interconnects, like Marvell, Coherent, and Lumentum, in preparation for the widespread deployment of these systems.
"For everyone who is in our ecosystem, we need a lot more capacity," Huang said during his GTC keynote speech. "We need a lot more capacity for copper; we need a lot more capacity for optics; we need a lot more capacity for CPO; and that's why we've been working with all of you to lay the foundation for this level of growth."
However, Nvidia's journey to this point began much earlier. In fact, by the time OpenAI revealed ChatGPT to the world in late 2022, Nvidia already knew it had a problem.
At the time, the GPU giant's most potent systems only featured eight GPUs, and the models driving the AI boom required thousands to train. Nvidia needed a bigger box, or at least a faster network that could effectively distribute work across dozens of chips.
We caught our first glimpse of this with Nvidia's Grace Hopper superchips in 2023, but it wasn’t until early 2024 that the full picture came into view. Unveiled at GTC that year, the Grace Blackwell NVL72, a monstrous 120 kilowatt machine, uses a copper backplane containing miles of cables to make 36 nodes and 72 GPUs behave like one enormous AI accelerator.
Copper was the natural choice for this, Gilad Shainer, senior VP of networking at Nvidia, told El Reg.
"Copper is the best connectivity, if you can use it," he said. "It's very cost effective, very cheap, and consumes zero power. It's very reliable. There are no active components."
But copper isn't perfect. At 1.8 TB/s, the cables could only stretch a few feet before the signal degraded as GPUs communicated with one another. If you ever wondered why the NVL72's NVSwitches are all in the center of the rack, it's because the runs were that short. Copper's limited reach also meant Nvidia had to cram as many GPUs into a single rack as possible.
Two years later, Nvidia is rapidly approaching the limits of copper and will need to embrace optics if it wants to assemble an even bigger GPU system.
The pluggable problem
When Huang first showed off the NVL72 rack, codenamed Oberon, the only commercially viable way to connect two accelerators optically would have been to use pluggable optics.
These modules are about the size of a pack of gum and contain all the lasers, retimers, and digital signal processing required to turn electrical signals into light and back again.
Pluggables are nothing new in datacenter networks, but using them for scale-up compute fabrics, like Nvidia's NVLink, presents certain problems.
To reach the 1.8 TB/s of bandwidth, each Blackwell GPU would have required eighteen 800 Gbps pluggables: nine for the accelerator, and another nine for the switch. On their own, these pluggables don't use that much power – around 10-15 watts – but multiplied across 72 GPUs, that adds up pretty quickly.
As Huang noted in his 2024 GTC keynote speech, optics would have required an additional 20,000 watts of power.
However, a lot has changed since the Oberon rack was first revealed. Advancements in co-packaged optics (CPO), which integrates optical engines directly alongside the switch ASIC, have helped drive down power consumption.
In 2025, Nvidia became one of the first AI infrastructure providers to embrace CPO by integrating it directly into its Spectrum Ethernet and Quantum InfiniBand switches. (Broadcom-based Micas Networks was making similar moves.)
This dramatically reduced the number of pluggables required to build an AI training cluster. However, it was only more recently that the company began discussing the use of optics and CPO for its NVSwitch fabrics.
NVLink goes optical
After pooh-poohing optical interconnects as too power-hungry two years earlier, Huang revisited the topic at GTC this spring by unveiling the Vera Rubin NVL576 and Rosa Feynman NVL1152, two multi-rack systems that would use photonics to expand their compute domains by a factor of eight.
If NVL576 sounds familiar, that's because the number has come up before. In fact, alongside the original NVL72 rack, Nvidia teased a configuration with exactly that many GPUs, though to our knowledge no such system was ever deployed in the wild.
Nvidia also briefly marketed its Vera Rubin Ultra Kyber racks under the NVL576 branding before deciding that it didn't actually want to count each individual GPU die as a standalone accelerator.
Unless Nvidia's marketing or roadmap changes again, the actual Vera Rubin NVL576 will use a combination of copper and optical interconnects.
"There's a lot of conversation about 'Is Nvidia going to copper scale up or optical scale up?' We're going to do both," Huang said during this GTC keynote.
According to Ian Buck, VP of Hyperscale and HPC at Nvidia, the first layer of the network will use copper interconnects in the rack, which means no changes to the GPUs. The second spine layer will use pluggable modules.
We don't know exactly what topology Nvidia plans to use for this, but a two-tier fat tree would certainly fit the bill, and would only require a single rack's worth of switches (72 ASICs in total) for the spine layer.
For the modules themselves, pluggables would be the simplest option, but Nvidia could also opt for near-packaged optics (NPO), like what Lightmatter showed off last month.
For Vera Rubin, Nvidia is only talking about optical scale for its Oberon NVL72 racks and not its NVL144 Kyber systems.
We're not exactly sure why Nvidia made the decision, but it's worth noting that if you can scale up optically, you don't need to pack everything into one rack. So, it may have simply made more sense to support optical scale up across eight racks from a thermal and power standpoint.
Nvidia Feynman goes co-packaged
Where things really start to get interesting is with Nvidia's Feynman generation, which is supposed to start shipping in mid-to-late 2028. We're told these systems will be available with either copper or co-packaged optical NVLink interconnects.
Nvidia is being somewhat tight-lipped about how this will all work, but there are a couple of potential avenues.
The simplest option would be to integrate CPO into the NVLink switch ASIC and continue using copper interconnects in the rack.
This would require a two-tier NVSwitch fabric and two or three different switch ASICs: one that's half optical, one that's entirely optical, and likely one without CPO.
Going this route would enable Nvidia to support multiple configurations simply by swapping the NVLink switch trays or wheeling in a spine rack as needed.
The more interesting possibility would be to integrate the CPO into both the switch and the GPU package. This would almost certainly result in multiple Feynman GPU SKUs – one with and one without optics – but it would reduce the fabric to a single tier.
Speaking with El Reg at GTC last month, Shainer declined to comment on which approach the company planned to move forward with, but highlighted the advantages of a single-tiered compute fabric.
"Scale-up is something that you don't want to build multiple tiers if you don't have to, because you want to minimize latency between the compute engines," he said.
While possible to bake CPO into the GPU, a single tier NVL1152 system would require one helluva high-radix switch. But with Feynman unlikely to ship until mid to late 2028, we suppose it's possible.
- Ayar Labs taps Wiwynn to cram 1,024 GPUs into a photonic rack system
- Lightmatter says latest photonics will slash datacenter fiber bills in half
- Nvidia-backed photonics startup Ayar Labs fills its wallet to mass-produce CPO chiplets
- Nvidia burns $4B to light up American photonics manufacturing
Securing the means of production
Either option is going to need a healthy supply of laser modules. While CPO moves much of the optics and signal processing onto the package, lasers are usually kept separate for the purpose of serviceability. This helps to explain the $4 billion ($2 billion each) Nvidia plowed into Coherent and Lumentum, both companies specializing in optical lasers, last month. If it's going to embrace CPO in a meaningful way, the supply chain needs to be ready.
Further evidence to suggest Nvidia is moving to on-accelerator CPO is the company's $2 billion tie up with Marvell announced earlier this week.
As part of that investment, Nvidia will work with Marvell to integrate NVLink Fusion, a licensed version of its high-speed interconnect tech, into custom XPUs for use with the GPU giant's Vera CPUs. The work will also extend to the development of optical I/O technologies, though to what extent the companies didn't elaborate.
As our sibling site The Next Platform discussed earlier this week, Marvell's $3.25 billion acquisition of Celestial AI could come into play here.
The startup's photonic interconnect tech could be used to build a coherent memory network spanning multiple racks, which could be just as attractive to Nvidia as it would be to one of Marvell's biggest customers, including AWS. As you may recall, AWS is among Nvidia's biggest NVLink Fusion customers, with plans to use the tech in its next-gen Trainium4 compute clusters.
In any case, Nvidia has clearly seen the light on optical scale-up, and we can expect CPO to play a much bigger role in its system design moving forward. ®
