Matilda Bailey is a distinguished semiconductor specialist whose work sits at the intersection of high-performance computing and next-generation thermal management. With years of experience navigating the complexities of cellular and wireless infrastructure, she has become a leading voice on how hardware must evolve to sustain the relentless growth of artificial intelligence. In this conversation, we explore the shifting paradigms of datacenter architecture, focusing on the transition from external cooling to integrated thermal solutions within memory stacks. We delve into the economic restructuring of AI hardware spending, the technical hurdles of vertical chip stacking, and the competitive race to redefine memory density before the end of the decade.
How does the integration of cooling elements directly into the physical interface change the way we think about thermal resistance in high-bandwidth memory?
The shift toward placing cooling elements directly into the Die-to-Die Physical Layer, or the D2D PHY, is a fundamental departure from how we’ve handled heat for decades. Traditionally, we relied on external cooling where heat had to migrate out of the package before it could be dissipated, but iHBM changes that by creating an internal heat dissipation path. By embedding these Integrated Cooling Elements right where the memory and GPU interface, we are seeing a significant 30% reduction in thermal resistance, which is a massive leap for system stability. You can almost feel the relief this provides to the silicon; instead of the chips choking under their own thermal output, the heat is intercepted at the source. This allows processors to maintain their peak speeds without hitting those frustrating temperature ceilings that act as a constant drag on performance.
In light of recent spending trends where HBM has climbed to over 60% of AI chip component costs, what does this tell us about the changing hierarchy of datacenter hardware?
The financial landscape of the datacenter has been turned upside down, moving away from a logic-first mentality to a memory-centric one. Between the first quarter of 2024 and the end of 2025, we saw HBM spending surge from 52% to 63%, while logic dies—the traditional heavy hitters like GPUs—actually saw their share of spending dip from 14.2% to 12.9%. This tells us that the sheer volume of data being moved is now a more critical bottleneck than the speed of the processing itself. Architects are no longer treating memory as an afterthought; it is the primary concern because if you can’t feed the processor data fast enough, the most expensive GPU in the world is just sitting idle. This structural change signifies that the industry is willing to pay a premium for memory that can keep up with the overwhelming demands of modern AI workloads.
With the industry moving toward vertical stacking to improve latency, what specific challenges does this create for engineers trying to manage heat within the package?
Vertical stacking is a double-edged sword because while it gives us incredible density and low latency, it effectively creates a high-rise building of heat with no windows. When you stack memory chips on top of one another, the heat generated in the middle layers becomes trapped, creating a concentrated core of thermal energy that is incredibly difficult to exhaust. For years, this extra heat has been a major design constraint, forcing manufacturers to throttle performance to keep the hardware from degrading. The introduction of internal cooling layers is the industry’s way of finally breaking that “heat wall” by providing a dedicated path for energy to escape from the center of the stack. It’s a sophisticated dance of advanced packaging technology and custom silicon design that ensures the memory modules don’t become the weakest link in a high-performance system.
Looking at the competitive landscape with emerging technologies like Z-Angle Memory, how critical is the delivery window for these next-generation thermal solutions?
The clock is ticking loudly for every major player, as the demand for hardware has completely overwhelmed current supply chains in a way that feels permanent rather than cyclical. SK Hynix is targeting 2029 for its HBM5 products, but they aren’t alone in this race; Intel and Softbank are already working on Z-Angle Memory with a projected 2030 arrival. These dates are crucial because the market is currently facing structural shortages that have forced manufacturers to prioritize HBM over standard DDR5 memory. If a company can deliver a solution that offers better thermal management even a few months ahead of the competition, they could capture a massive share of the 2026 market where supply is expected to remain extremely tight. For datacenter designers, these developments represent a lifeline at a time when the expectations for rising performance are putting immense pressure on every component.
What is your forecast for the future of thermal management in the memory sector?
I expect that by the end of the decade, the concept of “external-only” cooling will be seen as an antique approach for high-performance AI environments. As HBM supply remains tight and prices continue to climb, the value of each module increases, making integrated cooling a standard requirement rather than a luxury feature. We will likely see HBM account for an even larger share of total component spending in 2026 and beyond, which will drive further innovations in custom silicon interfaces designed specifically to bleed off heat. The competition between iHBM and alternatives like Z-Angle Memory will push us toward a future where the memory stack itself is a sophisticated, self-cooling machine. Ultimately, the winners in the AI era won’t just be the companies with the fastest chips, but those who can most efficiently manage the laws of thermodynamics at the microscopic level.
