Sideways Chip Stacking Emerges as Key to Scaling AI Memory Amid Thermal Challenges

At the IEEE VLSI Symposium, researchers from South Korea and Japan unveiled sideways-stacked DRAM architectures that bypass thermal limits and TSV area overheads in next-generation HBM.

Chips & Climate Desk
Updated Nov 8, 2026 2 min read
Sideways Chip Stacking Emerges as Key to Scaling AI Memory Amid Thermal Challenges

To overcome severe thermal bottlenecks in conventional high-bandwidth memory, researchers are turning to lateral die integration. At last month's IEEE VLSI Symposium, research teams from South Korea and Japan presented two distinct volumetric DRAM architectures designed to stack memory dies on their sides rather than vertically. According to IEEE Spectrum, these sideways-stacked designs promise to deliver significantly higher bandwidth and capacity for AI accelerators while preventing high-density memory arrays from overheating under continuous data center workloads.

Current AI accelerators, such as Nvidia's B300, rely on vertical HBM stacks placed adjacent to the processor on a shared substrate. However, as memory vendors attempt to stack DRAM dies higher than today's typical 12-die stacks, heat dissipation becomes critical. The inter-die fill material exhibits thermal resistance roughly 100 times higher than silicon, trapping heat inside the stack. Furthermore, scaling vertical stacks requires additional through-silicon vias to maintain bandwidth, consuming valuable silicon real estate that would otherwise be allocated to storage cells.

Addressing these structural limits, researchers from South Korea's KAIST, Ulsan National Institute of Science and Technology, and Hanbat National University introduced an architecture called V-Die. According to IEEE Spectrum, V-Die eliminates vertical through-silicon vias and the standard base die entirely. Instead, each die features its own I/O array along its bottom edge, connecting directly to the package substrate at a 20-micrometer interconnect pitch. Integrated microfluidic cooling channels maintain operating temperatures at 45 degrees Celsius, well below the standard 80-degree peak.

Simulations conducted by the South Korean team evaluated a 16-die V-Die configuration paired with Nvidia H100 GPUs running Large Language Model workloads. According to IEEE Spectrum, the sideways architecture demonstrated an 82 percent speed increase over predicted HBM4 performance, delivering 540 tokens per second compared to 296 tokens per second for HBM4 at equivalent capacity. The design reduced initial token latency by 32 percent, saving about 24 milliseconds, while reducing memory read time by 37 percent by leveraging four times as many package-level interconnect lines.

However, sideways assembly introduces substantial packaging precision challenges. James Myers, program director at Belgium-based imec, told IEEE Spectrum that constructing volumetric DRAM requires flipping pre-stacked dies onto their sides. Minor thickness variations among individual DRAM layers accumulate across the stack, creating planar irregularities that risk missing interconnect bond pads on the underlying substrate.

To solve this mechanical alignment tolerance problem, a Japanese research collaboration comprising the University of Tokyo, Tohoku University, and Riken presented the MOSAIC system. According to IEEE Spectrum, MOSAIC replaces physical data micro-bumps with an inductive coupling transceiver system. The architecture places 80-by-240-micrometer oblong inductive coils on the die edge and substrate, transmitting data wirelessly via magnetic fields to accommodate positional misalignment. Designed for direct mounting atop GPUs, MOSAIC achieves passive heat dissipation through its vertical silicon fins, supporting up to 294 thinned dies to provide 882 gigabytes of capacity within a single package volume.

Sources

  1. 01 Stacking Chips Sideways Gives AI More Memory — IEEE Spectrum — Semiconductors
#high-bandwidth-memory #ai-accelerators #semiconductor-packaging #memory #thermal-management