The hardware race in AI is also evident in next-generation memory technologies. Nvidia and AMD will not only use HBM4 (High Bandwidth Memory 4) memory in their new AI accelerators, which they plan to launch in 2026, but will also raise the performance bar even higher by customizing this memory.

HBM4 memory is coming soon

HBM is a memory structure consisting of vertically stacked DRAM layers. These layers are connected by TSV (Through-Silicon Via) connections. This increases bandwidth and improves energy efficiency. This structure, which typically has up to 12 layers, allows for the addition of custom logic circuitry at the bottom layer.

Nvidia and AMD are actively leveraging this feature to transform memory from mere data storage structures to modules capable of intelligent data routing.

The primary goal of these customizations is to exceed current performance limits and gain an advantage over specialized ASIC solutions. To maintain their competitive edge, especially against the ASIC accelerators favored by companies like Google, Amazon, and Meta in their internal hardware, Nvidia and AMD need to move beyond standard versions of HBM memory.

According to reports, these specialized HBM4 solutions will first appear in Nvidia’s “Rubin” architecture and AMD’s Instinct MI400 series. Both architectures are expected to significantly outperform current GPU accelerator designs.

Rubin is the successor to Nvidia’s current Blackwell architecture, while MI400 will be AMD’s first line of AI accelerators implemented at the shelf level. Unlike standard HBM memory, typically sourced from manufacturers like Micron, Samsung, and SK Hynix, these customized versions are tailored directly to Nvidia and AMD’s specifications.