Memory Now Makes Up Nearly Two-Thirds of AI Chip Costs

Memory costs in AI chips have climbed to nearly two-thirds of total component expenses, and that single fact is quietly rewriting the power dynamics of the entire artificial intelligence hardware industry.

How We Got Here

For most of the past decade, the AI chip conversation was dominated by compute: GPUs, TPUs, custom accelerators. Companies like NVIDIA, AMD, and Intel owned the headlines. But as large language models (LLMs) and other AI systems scaled in size and complexity, the demand for high-bandwidth memory (HBM) grew at a pace that caught much of the industry off guard. Memory went from a supporting role to the starring one.

The Numbers That Actually Matter

HBM memory now accounts for roughly 65% of the total cost of a high-end AI chip. To put that in concrete terms: in a system like the NVIDIA H100, the HBM stack costs significantly more than the compute die itself. Only three companies can produce HBM at meaningful scale — SK Hynix, Samsung, and Micron — making this trio the most strategically critical suppliers in the AI ecosystem right now. Demand has been so aggressive that HBM lead times have stretched well into 2025 and 2026 with no signs of easing.

What This Really Means

This cost shift has a sharp implication: NVIDIA gets all the brand recognition, but a massive slice of the value in every chip it sells flows to South Korea, into SK Hynix and Samsung's fabs. Memory manufacturers — historically stuck in a volatile, commodity-driven cycle — now hold negotiating leverage they've never had before. Whoever controls the memory controls the bottleneck, and right now that's not who most people think.

What Happens Next

Expect three significant moves over the next 18 months:

• Massive capacity investment in HBM production from SK Hynix, Samsung, and Micron, each racing to capture more of this margin.
• Growing pressure from hyperscalers like Google, Microsoft, and Meta to diversify memory suppliers or develop in-house memory solutions.
• Higher prices for complete AI systems, which will eventually pass through to cloud inference costs and, down the line, to end users.

The geopolitical angle here is also worth watching. If memory is the most expensive component and only three companies — two of them South Korean — can make it at scale, Western governments have a genuine strategic vulnerability sitting in plain sight. Expect chip sovereignty debates to get louder.

The real open question is whether anyone can break the HBM oligopoly before it ends up setting the terms for the entire AI industry.

Source: Hacker News