Lead
A single 400G stream hits a switch, fans into APIs and storage calls, and suddenly the fabric groans under multi-terabit east-west traffic that no steady-state forecast ever predicted and no tidy oversubscription ratio can fully contain. That is the daily reality of AI training and data prep storms now battering data centers, where the problem is less about raw ingress and more about the shockwaves unleashed inside. Operators describe networks that hold fine at the edge yet falter a few hops in, where microbursts stack in queues and the weak link reveals itself in seconds, not hours.
This shift has turned proximity from a convenience into a mandate. As GPU clusters densify in Virginia and California, storage gravitates toward those gravity wells, chasing latency budgets measured in milliseconds and throughput ceilings that punish every extra hop. Add to this a fast-rising neocloud market, rich in GPUs but spotty in network maturity, and the center of gravity for AI infrastructure has started to slide away from old assumptions.
Nut Graph
The story matters because AI traffic breaks the playbook that built modern clouds. Traditional workloads shaped predictable patterns; AI floods arrive in bursts, amplify inside the fabric, and resist tidy capacity models. In response, enterprises are blending neocloud GPU fleets with third-party storage, designing for peaks rather than averages, and moving data closer to compute to tame latency and egress costs.
Backblaze sits at the intersection of these changes. By upgrading 100G links to 400G, adding parallel 400G lanes, and boosting Arista port density, the company is rebuilding for a world where Tuesday’s traffic does not predict Thursday’s. Its expansion in US East underscores a larger trend: geography has become architecture. Meanwhile, neoclouds promise speed and price advantages on GPUs, but networking gaps—limited peering, single-transit dependencies, and lean SLAs—force customers to test harder and design with failure in mind.
The Mechanics: When Bursts Rewrite the Network
Inside a modern data center, AI jobs do not behave like steady tenants; they act like flash crowds. A single 400G ingress can fan out into terabits as orchestration layers hit load balancers, metadata stores, object storage, and caches in rapid succession. “The multiplier kills you,” one operator said. “Ingress is the tip; the east-west fan-out is the iceberg.” That fan-out exposes hidden choke points: shallow buffers, hash imbalance across ECMP paths, and oversubscribed tiers that looked fine on paper.
Day-to-day volatility compounds the challenge. Capacity that cruised yesterday can peg today due to a fresh dataset, a reshuffled training schedule, or a short-notice fine-tune job. The implication is strategic: networks must be built for peaks, not means. Larger fabrics, nonblocking spines, and microburst-aware telemetry turn into safety valves. As another engineer put it, “You need headroom you hope to waste—until the minute you don’t.”
Proximity then becomes a first-order control knob. Moving storage to sit near GPU clusters shortens paths, lowers jitter, and trims the east-west fan-out footprint. The effect stacks with cost: fewer cross-region and egress charges mean budget can shift to ports and lanes that actually blunt the bursts. Geography, once abstracted by the cloud, returns as a design constraint with material financial weight.
Market Dynamics: Neocloud Momentum and Its Weak Link
Neocloud providers are riding a surge as organizations hunt for fast, affordable GPU capacity. Flexible contracts, quick turn-up, and specialized racks make them attractive, and many buyers report meaningfully lower GPU prices than hyperscalers. Even large platforms have sourced capacity from neoclouds when demand spiked, signaling a broader recalibration of where compute gets done.
Economics around storage and data movement push the shift further. Hyperscalers often attach meaningful costs to egress and cross-region traffic; in aggregate, those line items can eclipse nominal storage rates. By contrast, third-party storage vendors commonly offer lower raw storage prices and friendlier egress, tipping total cost of ownership toward a mixed stack: neocloud GPUs paired with external storage placed near compute. Comparative analyses have highlighted double-digit percentage savings in such designs, especially for training pipelines with heavy data churn.
Yet networking remains the soft underbelly of many neoclouds. Reports and customer accounts cite limited public peering, small IPv4 allocations, and single-transit dependency as recurring themes, alongside SLAs that minimize liability. “Compute scaled like a rocket; the network is still taxiing,” one buyer said. The message is clear: performance can be great, until a route flap or maintenance window reveals a lack of diversity. Due diligence now includes questions once reserved for carriers: peering maps, transit diversity, prefix controls, and maintenance discipline.
Inside Backblaze: Building Shock Absorbers for AI Traffic
Backblaze has treated the shift as a structural reset rather than a tuning exercise. The company has been upgrading 100G links to 400G, lighting multiple 400G lanes inside facilities, and stepping up to higher-density Arista switches to raise aggregate throughput and device fan-in. These moves expand the blast radius a fabric can absorb before queues clip and flows back off. In practice, the added port density acts like a shock absorber: more parallel paths, more bandwidth headroom, and fewer unlucky collisions.
Planning has matched the hardware. Teams modeled scenarios where traffic doubles or quadruples, not as edge cases but as planning baselines. That mindset reflects lived reality: AI jobs can shift overnight due to model tweaks or dataset swaps. “Design for bursts you have not yet seen,” a Backblaze engineer noted, “or be ready to shape and shed when they show up.” Microburst-aware telemetry, ECMP tuning, and buffer policy reviews rounded out the toolkit, turning visibility into early warning rather than postmortem analysis.
Geography, again, played a starring role. Backblaze expanded its US East footprint to meet AI demand near major compute clusters and ecosystem partners—hyperscalers and neoclouds alike. By cutting physical distance, the company reduced tail latency and improved throughput floors for training pipelines, while helping customers avoid costly hairpins across regions. That co-location created a flywheel: more nearby compute pulled in more storage; more storage made the region stickier for new workloads.
Conclusion: The Next Moves for an AI-Heavy Network
The next phase called for choices that prioritized resilience over neatness. Teams needed to uplift to 400G where feasible, favor nonblocking spines, and preserve upgrade paths that allowed rapid lane turn-up without forklift swaps. They also needed to harden for fan-out: spread flows with ECMP, tune buffers against microbursts, and instrument fabrics with telemetry that caught hotspots before users did. Proximity would remain a lever; mapping GPU clusters and storage hot spots, then enforcing latency and throughput SLOs, reduced both jitter and spend.
Procurement shifted in parallel. Buyers tested neoclouds under load, verified peering and transit diversity, and read SLAs with an operator’s eye. Storage placement followed data life cycle: hot near GPUs, warm and cold in lower-cost tiers, with caching and prefetch to quiet cross-region chatter. Finally, operations adopted continuous capacity modeling and runbooks for surge events—autoscaling triggers, traffic shaping, and graceful backpressure—so that the network bent rather than broke. Taken together, those moves turned AI’s volatility from a chronic risk into a manageable design parameter.
