The Growing Memory Tax on AI Infrastructure

The note argues that higher DRAM prices and growing HBM capacity and manufacturing complexity will raise server and AI infrastructure costs, making memory efficiency a major design constraint through 2030.

Market Overview

The report links rising memory costs to conventional DRAM demand across server types and increasing HBM use in AI accelerators such as GPUs. It also cites supply constraints that intensified pricing pressure for both general-purpose and AI-optimized servers.

Key Findings on DRAM Pricing

Dell’Oro Group projects server DRAM average selling prices to reach about $10/GB in 2026, then moderate toward the $5/GB range by 2030. The report frames the spending impact as potentially large over the decade as prices remain high near the forecast peak.

In a sensitivity analysis, if DRAM ASPs were held at about $3.50/GB from 2026 through 2030, higher pricing could add nearly 10% to cumulative server spending through 2030. The note expects pricing to moderate more meaningfully from 2028 onward as additional supply enters and the industry responds.

Forecast and Demand Drivers

The report says its overall DRAM outlook has increased, not only due to pricing but also because of higher projected DRAM bit demand tied to server unit shipments. It points to agentic AI and storage-related workloads increasing demand for general-purpose servers alongside AI-optimized systems.

HBM Cost Challenge and System Design Implications

The note projects an average high-end accelerator to contain more than 500 GB of HBM by 2030, with an example that AMD’s MI455X already incorporates 432 GB. It says HBM may not follow the same long-term cost-per-bit declines as DRAM because increasing stack heights, density, bandwidth, and packaging complexity could keep cost per bit elevated.

The report identifies manufacturing and advanced packaging as important to offset rising costs, citing technologies such as hybrid bonding, higher-density HBM generations, improved yields, and potentially lower HBM ASPs. It also says the industry may need to reconsider how much HBM sits directly alongside each accelerator, noting that NVIDIA’s Rubin Ultra could incorporate less HBM than about 1 TB originally envisioned and that model efficiency and memory management could reduce HBM needs.

Memory Efficiency and Capacity Allocation

The report says memory economics may constrain future AI system design alongside compute performance. It estimates elevated DRAM pricing could add hundreds of billions of dollars to infrastructure spending over the next several years and expects HBM capacity growth and complexity to add pressure on system costs for accelerated servers.

It adds that memory suppliers must balance production across conventional DRAM, HBM, and NAND flash as demand rises across servers, AI accelerators, and storage. The note characterizes capacity decisions as harder if DRAM and NAND pricing remains elevated, and it argues that cost outcomes depend on both more supply and lower manufacturing costs plus more efficient memory use through architecture, packaging, software, and additional memory tiers.

Overall, the report links near-term and longer-term cost pressure to DRAM price levels and HBM scaling complexity, and it frames memory efficiency and memory supply allocation as central to sustaining AI infrastructure spending through the end of the decade. This Analyst Signals brief reflects a neutral, fact-based summary of the original research note.