In March 2026, we tried to run a 24-hour AI inference workload for a new feature. Our cloud provider throttled us after 11 hours. The reason wasn't chips. It was power. --- That same week, Nscale announced it had acquired the Monarch Compute Campus in West Virginia. America's first state-certified AI microgrid, with up to 8 gigawatts of potential onsite power. Eight nuclear plants' worth of electricity, dedicated to AI. Jensen Huang has been calling these facilities "AI factories" for over a year. The Monarch acquisition is the first time one of those factories owns its own grid. The announcement wasn't about making AI cheaper for everyone. It was about securing supply before the squeeze hits. --- ## Microsoft's 1.35GW Bet on Vera Rubin Nscale signed a letter of intent with Microsoft for 1.35 gigawatts of NVIDIA Vera Rubin NVL72 GPUs. One of the largest GPU commitments in history. ### Vera Rubin, Briefly NVIDIA's next-generation AI chip architecture, named after the astronomer who discovered dark matter. The NVL72 configuration packs 72 GPUs into a single system connected by NVLink, delivering 1.4 exaflops of AI performance. One of these systems can train models in days that would have taken months on previous hardware. ### Microsoft Isn't Buying GPUs for Fun They're securing capacity because they see what's coming: demand for AI compute that current infrastructure cannot satisfy. The deployment starts in late 2027. The companies that control compute capacity in 2028 will define what's possible in AI. --- ## The Memory Numbers Worth Tracking While Nscale was announcing their acquisition, Micron entered high-volume production of HBM4. The RAM that powers these AI supercomputers. | Metric | HBM3E (Previous) | HBM4 (New) | Improvement | |--------|------------------|------------|-------------| | Bandwidth | ~1.2 TB/s | 2.8+ TB/s | **2.3x faster** | | Power Efficiency | Baseline | 20% better | Less heat, less cost | | Stack Capacity | 24GB | 36GB | 50% more memory | Memory bandwidth is the hidden bottleneck in AI training. You can have the world's fastest GPU, but if you can't feed it data fast enough, it sits idle. HBM4 changes that. Training runs that took weeks now take days. Models can be larger. Inference becomes cheaper at scale, for the companies that have access. For SaaS founders, the relevant question isn't "when will this be cheap?" It's "when will this be available to me, at any price?" --- ## Energy Is the Real Constraint Nscale didn't just buy a data center. They bought a power plant. The Monarch Compute Campus is certified as an AI microgrid. It can generate its own power, up to 8GW. In an era where data center power constraints are the single biggest bottleneck to AI expansion, owning your own electrons is the only way to guarantee supply. ### The Power Crunch Is Already Here Northern Virginia, the world's largest data center market, is running out of electricity. Major AI training facilities are being delayed because there isn't enough power available. Some regions have multi-year waitlists for new capacity. Power plants take 5-10 years to build. AI demand is growing faster than the grid can expand. The companies that secured energy supply now (Nscale, Microsoft, Meta) will have compute to sell. Everyone else will be bidding for scraps. ### More Infrastructure Doesn't Mean Cheaper Compute **Demand is inelastic.** AI training runs consume whatever power is available. When 50+ gigawatts of new data center demand hits a grid that can't expand fast enough, prices spike, not drop. **Regional competition.** Nscale's West Virginia facility has cheap power today. But as data centers cluster near any affordable energy source, they bid up local prices. That "cheap" power gets expensive fast when everyone competes for it. **Opportunity cost.** That 8GW could power roughly 6 million homes. As data centers consume more of the grid, either residential/commercial prices rise to clear the market, or governments impose restrictions. The era of cheap, abundant compute is unlikely to arrive on schedule. Nscale didn't just side-step a logistics problem. They secured a resource that will get scarcer and more expensive, then locked it up for Microsoft. --- ## What We're Doing About It We learned this the hard way. When our inference workload got throttled in March, we had to move the feature behind a queue and cap usage per customer. The cost didn't break us. The unavailability did. Our budget was fine. The capacity simply wasn't there. That experience changed how we plan. Three things we're doing now: 1. **Cost-modeling for scarcity, not abundance.** We stopped assuming inference costs will drop 10x. Our 2026 projections use $0.15 per query as a floor, not $0.10. If energy prices spike, we have headroom. When they drop, we pass savings to customers. 2. **Building for graceful degradation.** When GPU capacity runs short, our AI features fall back to cached or pre-computed responses instead of failing. The March throttle taught us that availability matters more than latency. 3. **Auditing our dependencies.** We mapped every AI feature to its provider and region. Two of them route through the same Northern Virginia cluster that's already capacity-constrained. We're moving one to a different region before it becomes a problem. The mistake we made was treating compute like electricity. We assumed it would always be there when we needed it. Compute is closer to water in a drought. You don't notice the constraint until it's your tap that runs dry. --- ## The Tiered Market Ahead The era of "cheap AI for everyone" may not arrive. Instead, we're heading toward a tiered market: hyperscalers with secured energy supply offering reliable (if not cheap) compute, while everyone else deals with volatility, rationing, and rising costs. If you run a SaaS business, don't assume AI compute will get cheaper. Building a business model that requires 10x cost reductions to work is a bet on abundance that may not pay off. Energy security, not GPU access, is becoming the defining constraint. We're still figuring out what this means for our product roadmap. One open question we haven't resolved: should we pre-pay for reserved GPU capacity 18 months out, or stay flexible and risk being shut out during peak demand? I don't have a clean answer. Neither does anyone I've asked. --- *If this was useful, we publish one post a week on building software in an era where the constraints are real. No "10 AI tools that will transform your workflow." Just what we're learning, the numbers we're seeing, and the mistakes we're making. Subscribe here.*

Posts tagged with "infrastructure"

The Compute Squeeze: What Nscale's 8GW AI Factory Actually Means for Your Business