- HBM 3D design on GPU achieves record compute density for demanding AI workloads
- Maximum GPU temperatures exceeded 140°C without thermal mitigation strategies
- Halving the GPU clock speed lowered temperatures but slowed AI training by 28%.
Imec presented a study on 3D HBM design on GPUs at the 2025 IEEE International Electronic Devices Meeting (IEDM) with the aim of increasing computational density for demanding AI workloads.
The thermal system engineering co-optimization approach places four high-bandwidth memory stacks directly on top of a GPU via microbump connections.
Each stack consists of twelve hybrid connected DRAM chips and cooling is provided by HBM.
Heat reduction efforts and performance trade-offs.
The solution applies performance maps derived from industry-relevant workloads to test how the configuration responds under realistic AI training conditions.
This 3D layout promises an increase in computing density and memory per GPU.
It also offers higher GPU memory bandwidth than 2.5D integration, where HBM stacks rely on a silicon spacer around the GPU.
However, thermal simulations reveal significant challenges for 3D HBM design on GPUs.
Without remediation, maximum GPU temperatures reached 141.7°C, well above operating limits, while the 2.5D benchmark peaked at 69.1°C under the same cooling conditions.
Imec has explored strategies at the technical level, such as fusion of HBM batteries and thermal optimization of silicon.
System-level strategies include double-sided cooling and GPU frequency scaling.
Reducing the GPU clock speed by 50% reduced the maximum temperatures to below 100°C, but this change slowed down the AI training load.
Despite these limitations, Imec claims that the 3D framework can provide higher computational density and performance than the 2.5D reference design.
“By halving the GPU core frequency, the maximum temperature was reduced from 120°C to less than 100°C, achieving an important goal for memory operations. However, this move comes with a 28% reduction in workload…” said James Myers, Systems Technology Program Manager at Imec.
“…the overall package outperforms base 2.5D performance due to the higher performance density offered by the 3D configuration. We are currently using this approach to explore other GPU and HBM configurations…”
The organization suggests that this approach could support thermally resilient hardware for AI tools in dense data centers.
Imec presents this work as part of a broader effort to link technical decisions to system behavior.
This includes the Cross-Technology Co-Optimization (XTCO) program, launched in 2025, which combines STCO and DTCO thinking to align technology roadmaps with system scaling challenges.
Imec said XTCO enables collaborative troubleshooting of critical bottlenecks in the semiconductor ecosystem, including systems and fabless.
However, these techniques are likely to be limited to specialized installations with controlled energy and heat balances.
IN TechPowerUp