Nico Johnson  00:00

For years, electricity demand across much of the country barely grew. There wasn't a phrase that we referred to as load growth. That's something that's relatively recent. Now we're hearing about data center projects asking for hundreds of megawatts, sometimes gigawatts, very quickly. What's changed enough that we need a different way of thinking about electricity demand.

Peter Kelly-Detwiler  00:22

Yeah, for about 20 years, we were in the period that I call or like to call flat, dumb, and happy. And during that period, especially say 2010 on, the

Peter Kelly-Detwiler  00:32

goal was how do we decarbonize the grid. So the focus was on how do we bring in renewables, and then we got enough solar in there. We started to have to add storage, and you know everything continues to evolve. Then, November 2022, Chat GPT jumps out of the box, and within a very short period of time, there were millions of subscribers, and and the data center community started to realize those are there. There, I mean, AI had been perforating for a long time, but suddenly it became real, and within a year or two, you started to see the forecast for future demand pop up year after year after year. So let's put that number in perspective. Last year in July, the peak across the United States, if memory service was 759,000 megawatts, so 759 gigs. I think that's the right number. Texas alone, in its June large load working group reported 474,000 megawatts, 474 gigs of large load that wanted to interconnect to ERCOT. Okay, so that's what we're talking about. And ERCOT, by the way, it peaked out at 91.3 gigawatts this year, so it has five times as much load, wanting to interconnect, 90% of which is data centers. Then there is some Bitcoin mine and some large industrial customers as well. And there is two things going on there: there is magnitude and there is speed, right? Because the the digital community is used to moving fast and breaking things. The electricity community is rightly cautious because if frequency goes bad, things break, people die. Right? I mean, the grid is the heartbeat of the world, and you have to keep it at the 50 or 60 hertz that we have across the across the planet. And so, these two cultures and sort of technological things collide, and we're right at the epicenter of that right now, and it's being reflected in markets and social conversations. And you can't go anywhere without hearing about AI and right behind it, electricity, because without power, it goes nowhere.

Nico Johnson  02:33

Hey guys, and welcome back to SunCast. That was Peter Kelly-Detwiler, also better known as PKD. I'll have him back later. He's gonna help us make sense of all this, but he just put his finger on the thing that this whole episode really is about. All of AI, every model, every data center, every dollar of it goes nowhere without one thing: electrons. Here's the question that I just don't hear that often: What is a megawatt hour actually worth to the thing that's consuming it? More specifically, what are the creators of this technology-the metas, OpenAI's, and anthropics of the world-willing to pay to generate these tokens, because if you know that number, then you know how this is going to play out, how power prices move, whether data centers will ever be flexible, who gets connected first, and what it all costs for the rest of us. I went to three people I feel are veritable experts on this topic. You'll hear from Hans Royal and David Siap, who've each built metrics of their own that get at half, more or less, of this equation, and they've both co-authored a paper together on exactly this topic. And then Peter is going to come back in to tell us whether any of it really survives contact with the real power market, Hans has spent 18 years in clean energy, a large part of it at a company that Schneider Electric acquired called Renewable Choice, where he's helped build corporate PPA advisory around renewable deals that number more than 25,000 megawatts of wind and solar, and we went looking for that number. That number that helps us really explain how much someone is willing to pay per token or per megawatt hour of compute. He he couldn't find it; just doesn't exist. So he built it. He calls it the compute heat rate, CHR. And if you know what a gas heat rate is, well, we've already got a bit of the shape of it. A gas heat rate tells you the price a generator needs to run. The compute heat rate asks the opposite question: What's the highest price an AI workload can pay or is willing to pay before it stops being worth computing at all? What he found is a gap between what a megawatt hour costs today and what AI can afford to pay for one, and that gap is roughly 100 to one. Now, just about everything else in this episode is a consequence of that metric, and that gap-a 100 to one-of what we currently pay in the broad, let's call it CNI or utility scale market. AI is willing to pay 100 times more. So I start where he started. Hans, you started down this path by asking a simple question. At least on the surface, the energy industry wasn't really asking, and that is, what is a megawatt hour actually worth to the thing that's consuming that megawatt hour, I'm curious how you sort of came about the realization that that concept mattered as we consider adapting to and addressing load growth.

Hans Royal  05:53

I never really set out to create a metric, but I was doing research in the intersection of of AI and data centers and and and energy and coming from an energy background, I was sort of searching for the metric, and I realized the metric didn't really exist, or I couldn't find it. And so instead of you know just giving up, I decided to create it. And so and the metric really, I think the simplest way to think about it and why it matters is that it really measures so the compute heat rate effectively the innermost loop of what's happening in you know in creation in these AI factories right is we're converting electricity into intelligence or we're converting electrons into the unit of measure on the other side which are tokens or intelligence on the other end and so the question is well what's the true economic conversion of of electricity into intelligence, and is it the same for all AI data centers, or is it different? And like, what's the number? And so I started that research really with that question. I didn't know the answer, right? It was just I want to understand that because I think of things through kind of a. This is obviously a borrowed term from the gas heat rate, which is converting effectively gas molecules into electricity. And so there's a heat rate, there's a conversion rate there, and it tells you about the efficiency of that process. And then there's all kinds of markets that develop around that over time. So I asked the sort of same question of, well, what what about the other side of the ledger? What about converting electricity into intelligence? And the math was eye watering when I first did it. Effectively, the compute heat rate is orders of magnitude higher than than the price of electricity in the country. So, for context, if I may, just sort of share a few stats. So,

Nico Johnson  07:31

yeah, the

Hans Royal  07:32

compute heat rate is in the 1000s and 1000s of dollars per megawatt hour. Okay, what does that what does that mean? Let's say the average price of electricity is you know $50 per megawatt hour, or maybe 90, $100 per megawatt hour. Typically, when you convert electricity into other economic things of value, like a steel mill creates steel, or aluminum smelter creates creates aluminum, at some point it's no longer worth it to use electricity and convert into that other thing, and let's say if what if energy prices doubled or tripled to $200 a megawatt hour, $300 a megawatt hour? Well, all of a sudden, a lot of other factories may decide, you know what, we're losing money on this. We're gonna we're gonna turn down. And the staggering thing about AI, when you do the math, which I've done through the compute heat rate analytics, is the conversion to tokens or to intelligence is something like on average $5,000 to $10,000 per megawatt hour. So that's maybe 100 times, 200 times higher than the current price of energy. And the spread is what matters. And once you can understand the spread between current electricity prices and the willingness to pay, essentially, that AI data centers have, then it tells you a lot of interesting things about how the power markets may evolve in the future, about how flexibility might be valued when you know data centers may become flexible resources, speed to power, etc. So there's a lot of things that the metric can help you, you know, help me at least understand, and excited to continue sort of measuring that over time.

Nico Johnson  09:06

Have you distilled down maybe the simplest useful explanation of compute heat rate? If it was in a Webster dictionary, it

Hans Royal  09:14

distills really down to a question, which is how expensive would electricity have to become before converting electricity into AI intelligence is no longer worth it. What's the number? That's what the compute heat rate fundamentally is. When would it no longer be worth it to build the next data center? And the answer currently, with current token pricing, current economics of the costs of building a data center and the interconnection and all the things is about $5,630 per megawatt hour. So the price of energy would need to get that high for its from an economic perspective. Not to speak of you know permitting and everything else, but from an economic perspective, electricity would have to be above $5,630 to make it. No longer worth the conversion of electricity into tokens,

Nico Johnson  10:03

which is to say, like if we think about this as like a data center is a token factory. It's an intelligence factory, right? At the the point at which they would say, "Ah, enough! Shut the factory down. We can't afford to produce another unit of intelligence, is about $5,600 a megawatt hour,

Hans Royal  10:18

which is about 100 times higher a willingness to pay than pretty much any other industrial factory of other types, so it's a completely different ball game.

Nico Johnson  10:28

I'm curious which differences among AI workloads actually matter if we're trying to understand the loads, which loads can and which loads cannot move in terms of compute heat rate or the or their willingness to pay.

Hans Royal  10:41

Quickly realized when I first you know wrote the research paper that sort of announced the compute heat rate and how it's calculated that these numbers are moving around all the time, and so it made sense for me to really create an index to track the changes over time because almost every day now, almost every week, a new model is coming out from OpenAI, from Anthropic, or from SpaceX, XAI, etc. And so I'm now creating it and publishing an index each quarter. I'm sure in the future that'll have to be updated even more regularly because this market is changing so rapidly. But number one, the tracking of the the compute heat rate index over time helps sort of in general show you where the trends are going you know is the willingness to pay for energy actually going up over time is the spread shrinking is it going to become come down to sort of reality into sort of normal industrial pricing but so far it's quite robustly staying in the you know 100x plus times the willingness to pay for anyone else. So that that that is very stable. Seemingly, the next thing in the q3 2026 CHR index that I attempted to do is be a lot more granular of the different types of workloads a data center may be running. From the most expensive or the highest CHR would be Frontier Inference. So if you do the math on, say, you know, the new Astro model from OpenAI that just came out, GPT six, right, came out just last week, or Claude Fable five, or that tier, that quality and cost of tokens, the compute heat rate of that is actually in the 10s of 1000s of dollars per megawatt hour equivalent. Versus then on the other end of the spectrum, the sort of commodity inference, so to speak, it's when you're running you know mini models that are super efficient that don't use that much electricity that you get a lot of tokens out of a GPU, etc. They're very cheap prices on pennies per million input and output. Then the CHR is actually much lower, but interestingly, still in the hundreds of dollars per megawatt hour plural at the very bottom of the spectrum. So it's it's both stratify the average price right now, or the average CHR is 5630. The very lowest sort of quote cheapest or lowest CHR is in the several $100 per megawatt hour range, and then up to the you know several dozen $1,000 per megawatt hour range on the high end, so it's quite stratifying because we are seeing the commoditization of mini models, cheaper models, sort of Chinese open weights, etc. models that are sort of pulling the bottom down, while the frontier continues to be robust and quite expensive from a token perspective.

Nico Johnson  13:20

I'm curious, what did you

Nico Johnson  13:21

learn that caused you to change the model, sort of the the underlying parts of it, and what part of the original idea maybe even got strengthened after you did that?

Hans Royal  13:33

In something like this, I started out with a lot of assumptions, and I had to make judgment calls on okay, which GPU should I use as a standard? You know how closely does that reflect the installed base of GPUs that actually exist? And so right now we're standardizing across the h1 100, which is the most common GPU that's installed. But of course, there's other GPUs that are coming out right from NVIDIA and even other TPUs and other technology. So really, what I've done is really kind of just enhance the research over time. I have a pretty long research agenda, and then each quarter I will add a new feature of okay, we're actually getting more specific and more granular in this area. And so, a couple examples from q2 to q3. Instead of using say a average token throughput assumption for a GPU between models, which was you know well sourced, but it was an average figure, I'm now differentiating how many tokens do you get out per GPU cluster when you run a Frontier model, which takes a lot of calculations, versus a MIDI model, which takes a lot fewer calculations. You can get way more tokens out of the same hardware running a small mini model than you do the Frontier, and so the differentiation, for example, on actual token output between the model classes that I have is a big change from q2 to q3, and just helps each number and each tier be more accurate going forward as to what the compute heat rate is for those tiers. As as one example, and this was again based on feedback and you know great collaboration. From sort of the wider market, who have engaged with the work, I think there's a lot of talk in the marketplace about flexibility. There's legislation in key states like Texas requiring flexibility, however however that ends up being defined. So I'd like to connect this to flexibility, if if you can. If delaying or moving computation can destroy a lot of economic value, then why would a data center ever agree to flexibility? And what does the grid have to give them in return? You know, there's famous studies about well, if if new AI data centers could be just this much flex, this these many hours, 100 hours per year, or some amount of hours per year, the small percentage that would unlock a lot of available capacity on the grid. Grid and can't we all just do that? And and I think yes, that would be great if everyone just sort of did that. But the economics and what the CHR can help us understand is that the economics don't suggest that that's a rational economic decision for a data center who's running Fable Five or the frontier models, like that's just not a rational decision. If they're forced to or whatever, sure. But from a just raw economics perspective, the incentives are not aligned for a data center to be flexible. So what do you do? Well, number one, there are different types of flexibility, right? Number one, flexibility could mean you just turn off the GPUs,

Nico Johnson  16:20

yeah,

Hans Royal  16:21

which may not be rational, like I'm saying, depending on what workload you're running. But there's other types of flexibility from a grid perspective. Maybe during those hours of the year where the grid kind of needs you to be flexible, flexible, can you fire on distributed generation resources or batteries to power your GPUs so the grid sees you being flexible? But from a workload perspective, you continue running your GPUs at full capacity. There's work. There's load shifting. There's workload shifting between facilities. If you have a data center footprint in on the East Coast and in Texas and on the West Coast, maybe you can actually shift workloads between your GPUs in order to be flexible in that way, and there's some great new startup companies that are working on just that problem, which I'm really excited to see. And there's a lot of other solutions in between. What the CHR can do is measure the economics of those decisions to differentiate between hey, at what price does it make sense? Maybe it makes sense at a certain energy price to actually move or curtail or shift, you know, the commodity workload tiers that are in the 200 to $500 megawatt hour equivalent. If prices get to there, okay, well then maybe it actually does make rational sense to curtail or move those workloads. Versus again, if you're running the frontier, and almost no scenario would it make economic sense. And so, my view there is that first, if you can measure the economics of this, then can you provide rational and scalable solutions that both the utilities and the grid and the RTOs and ISOs will will want, but also the data centers will want to participate in because it makes fair and rational sense on both sides, and the CHR can help do that across these different workload tiers.

Nico Johnson  18:09

So this is the part of the conversation where I put that flexibility question back to Hans. It's what our entire industry right now, frankly, is arguing whether these data centers can actually be made flexible and whether we can build them as such,

Hans Royal  18:22

there is a real economic trade-off between flexibility and speed to power. And certainly, one of the instruments to address this is sort of forced flexibility, and that would be: hey, if you want access to the grid, you have to agree to be flexible in these and these conditions. My view is that there could also be proactive flexibility where you look at your workloads that you're planning to run within the data center and you say, look, actually, economically, I'm willing to curtail a certain percentage of the hours of the year because it makes economic sense for me to not run my data center when energy prices go above a certain threshold, and so it's almost like there's a demand curve of hey, for every 1% or 5% or 10% of flexibility I can offer economically to the utility, how does that influence how fast I can get connected onto the grid? And that's actually the subject of some upcoming research that I have in a paper I'm co-writing with another organization that actually is measuring the cost of delay and has a whole metric around that. Well, we're saying, well, for how much speed to power will you offer? What amount of flexibility and where is the break-even point that makes sense economically for both parties of that of that trade? And the compute heat rate is one metric that helps you understand the flexibility economics, and again, the more that can make rational sense to all counterparties, I think the more scalable those types of solutions become over time.

Nico Johnson  19:52

Hans, I'm curious where you see your work having the greatest influence. Is it investor? Decisions, utilities, market sentiment. Who are you trying to reach with the work? Ultimately, who do you want to help or influence with this with this metric? And I'm curious what feedback you're getting from those that you are reaching.

Hans Royal  20:14

Look from the investor perspective, if you're investing in either data centers or energy infrastructure, understanding kind of the end-to-end economics of that picture and where your investment fits in along that value creation stack is first of all, I think, important to know, and people want to know that. But there's also different counterparties. You know, not everyone is vertically integrated like Google is, owning all the infrastructure, the TPUs up to actually selling the AI through Gemini, whereas others, you know, may be taking one slice of that pie-powered land to building the shell to you know being a Neo Cloud to a colo provider to a hyperscaler to the ultimate token you know foundation model, and so understanding kind of where you are in that value creation, and then you know potentially innovating and capturing some of that value along that journey that that makes rational sense for where you sit as an investor is is a huge area that I've been very you know surprised and and you know excited to collaborate with you know private equity, public equities, investors, etc. That's one one area. Yeah, data center developers themselves, right? Wanting to know the compute heat of their own facilities is is another interesting area that I was sort of like interested in in hearing about. I've also been very impressed with regulators and sort of the RTO picking up, you know, RTO sort of picking up the compute heat rate as well in thinking about how to structure large load tariffs and how to actually, you know, scale those to make to make economic sense. And so that's also another big area where I think the compute heat rate work can can really add some value. Or at least I've been hearing that from those types of stakeholders, and I'm excited to continue collaborating because if you can set up a system that makes rational economic sense to all counterparties, I think there is a lot of opportunity to provide community benefits, help you know use some of the spread, frankly, between the current power price and the willingness to pay to upgrade the grid, to modernize our grid, to do the next level of you know demand response and virtual power plants, et cetera. Right? I think there's a lot of opportunity. That's that's why we part of the thesis of the clean energy's AI moment that I co-wrote with David Siap was really saying this is a wonderful opportunity for the clean energy economy to take advantage of these economics to be able to build the next generation of technology solutions in the clean energy space.

Nico Johnson  22:35

So this is where David Siap comes in because he has the other half of this story, Hans has told us what AI is willing to pay. David's work asks the question on, frankly, the other side of the meter: What does it actually cost to serve that load, and who ends up carrying that cost? David's the senior manager for energy at Creo, the nonprofit that works with a couple of 100 families and family offices trying to move serious private capital into climate and climate tech. Before that, McKinsey, then X, then Google's Moonshot Factory, and then techno-economic research at Berkeley Lab held David's attention. Earlier this year, he was the lead author on a CREO paper called "Affordability is All You Need. We'll link to it if you want to go read a little bit more, and it introduced a metric that we're about to get into: the marginal cost of firm capacity, or MCC. It doesn't ask what does a megawatt hour from this technology cost, but rather what happens to somebody's power bill when the system adds one more unit of reliable peak capacity by using it. Jacob Shaw picked it up, and so did Jennifer Granholm, fair warning, it produces some uncomfortable answers about resources a lot of us in solar are rather fond of. I started by asking him what he was trying to fix.

David Siap  23:50

LCOE, levelized cost of energy, is typical metric that we use to compare energy resources, and it's great on a project basis. But think once you zoom out, it tends to fall short when we want to talk about you know reliability. When we want to talk about firming, right? It typically needs there's extra steps there, right? And so the marginal cost firm cost capacity, the MCC takes that into account. It incorporates the this concept called the effective marginal effective load carrying capacity of a resource, which is essentially how much a resource will contribute to when when the grid is stressed during peak hours. Maybe that's the hottest summer day, or these weird winter peaks that we're seeing, and it takes that into account. And I think that's that's what it was formulated for, and then it reflects it back into household energy bills during this time where energy affordability is is super important. And and just maybe last point, why the ELCC is relevant is because we are in an area of low growth after so. A couple decades of flat energy demand, and so as as load grows, the peak grows, and that's how the build the the grid rather is is built and planned and operated. And so I think the MCC better reflects that in combination with LTOE, you get a pretty good picture, a better picture of of resources, true cost. Is

Nico Johnson  25:23

there an example that might be useful around what is a decision that I might make one way using LCOE that I would look at differently using MCC in creating this metric? What is it that what what choices or decisions are you trying to help shape or influence?

David Siap  25:39

I think there's a couple, and they mostly show up at the tail ends of the spectrum, right? So I think starting with the positive one on the far left side, the cheapest cost, resources, demand flexibility, so energy efficiency, demand response, batteries as well. Those look really good through this lens, because it takes into account the energy savings and the avoided infrastructure spending, that's a feature of these resources, right? And so, one might prioritize a different resource without this view of of the of the of how much they they may save via energy savings or or offset infrastructure, the other side, and this is where I've had a lot of conversations, is resources that may be cheap from a nominal capacity perspective or have low LCOEs, but they don't contribute much to peaks. Right, so in particular, residential rooftop solar in a high solar penetration area, like say UGM or Kaiso, the a resource there might look cheap from a nominal capacity perspective. But then when you integrate how it impacts reliability, then all of a sudden you know other resources might look better.

Nico Johnson  27:03

You recently wrote an article that, in many ways, was the genesis of this dialog that that I wanted to have with you today, in concert with Hans Royal, around sort of the moment that we're in with clean energy and AI, and how sort of the energy sector as a whole is rising to meet this new load growth challenge. When you put the work that you've done on the cost of adding capacity next to Hans's work on what AI workloads can afford to pay for power, I'm curious what does that help an investor perhaps understand or see that they might otherwise miss.

David Siap  27:41

One thing we've seen in the venture space in particular is it's just kind of a distortion on caused by this very high willingness to pay from from the hyperscale, right? And when I say that, I'm particularly thinking about the distributed energy resource space. Maybe building energy in particular is a good example. I was speaking with an investor just the other day, and he said, "I thought I made four or five building investments. I actually made four or five data center investments because these products. All of a sudden, you know, I thought I was investing in a water heater, a furnace, an induction stove. They're sprouting batteries, whether electrochemical or thermal, and control strategies where they can say credibly that they are grid resources and provide great services. And by the way, they are they are seeing valuation upticks because of that. Right, it's highly economic because a hyperscaler will come in and say, "Hey, you know, my willingness to pay for energy is in the $6,000 a megawatt hour range, where everybody else is, you know, 80, 100, 200 at most dollars per megawatt hour, and so so that massive new customer with this high willingness to pay is distorting the market, and people are rejiggering their products to, to to meet it, and I think if investors have a view of what's coming, that really helps them position and make their bets.

Nico Johnson  29:08

When you're speaking with investors about AI and power and the expectations at marketplace, I wonder what what variables are you encouraging them to more carefully inspect before they put their money to work.

David Siap  29:19

At this point, there's a lot of investors that are wondering where we are in the supply demand cycle, right? And I think the the CHR speaks to that directly. I think one thing is is the obvious thing to point out is the durability of the pipeline, right? There's there's this giant data center pipeline, and folks are discounting it. The highest I've heard is 90% I've heard 70% I've heard 30% It's such a wide range that you know nobody knows really how much data centers will be, how many data centers will be built, and if you're stacking bets bets on top of that, right? Let let's say you're you're building turbines or or or something downstream or. Stream really of data centers. Well, you're beholden to that that that pipeline, which is so chaotic, which makes it really hard. I think to deploy capital at least from a traditional manner without some some of the shenanigans. I think we see out there. I

Nico Johnson  30:15

want to follow that thread on the bear case. If AI demand ends up being let's just say half of what today's forecast suggests. Which investments do still look smart versus which ones become harder to justify? If

David Siap  30:27

we get half of the pipeline, that's still a massive increase in load. So I think at 50% I think a lot of the investments that we see, you know, advanced clean energy, say geothermal, those kinds of bets, solid-state transformers, those kind of bets. I think those will most likely look relatively smart. I think one question that I got a lot, I did a lot, is on rooftop solar position on this curve, and given this is a solar podcast, I feel like perhaps it's it's worth addressing, and and really it it what it reflects is that rooftop solar is expensive, particularly in America. I I get this question enough that I ran the numbers for Australia, and if you run it for Australia, the number comes in about a third of what it does for America, and it's right in there with all of the other sort of affordable resources. And I'm talking specifically residential rooftop solar, where in the U.S. you have high soft costs, and you don't have necessarily the the best, you know, usually single axis fix or not sorry, not single axis tracking fix. Yeah, so they don't necessarily contribute much to to that to that peak load, which makes it look particularly bad in this context. However, it's still you know a reasonable source of energy, and so you know I I'm not saying there's no use use case for rooftop solar, but that's a conversation I've had a few times, especially inhabiting you know this clean energy invested landscape. It

Nico Johnson  32:02

makes sense. So you're advising investors. Investors are asking you where's a smart place to put their money, and they want to see that it has a both an economic and an ecological impact. If we look at the next sort of $10 billion of energy investment aimed at serving this load growth, where do we put it, and why? That's the broad question investors are asking, right? Is it storage, flexibility, utilization, clean power, gas transmission? There's, there's what's fast today that could turn into expensive years from now. How do you think about that?

David Siap  32:36

Yeah, I think fast today, expensive years for now. From now, I think gas, and I think that that's a lot of folks are signing up for particularly behind the meter, say error derivative, which is very much in loop these days. What we should do instead, I think we can scale demand flexibility in meaningful way to unlock gigawatts of capacity on the grid. I think there have been plenty of studies out there that show, to your point about utilization, there is headroom on the grid. We just have to find it. We just have to integrate, and we just need to have the structures in place to to where these large loads can integrate in a way that's that that's out beneficial for society. I mean, plenty of folks have talked about how an expanding load base could lower energy bills, and maybe that plays out, but also maybe it doesn't. And so I think the more we emphasize these resources that are lower cost of capacity, lower lower cost pathways to expanding the grid, that the the more likelihood that we get this future where AI actually lowers energy costs for.

Nico Johnson  33:45

Okay. Later on, I put Hans's argument to David that the economics just don't support flexibility, so don't expect it. And he doesn't disagree with the math. He just doesn't think the math is what decides the outcome.

David Siap  33:58

Let me first start. I agree with everything Hans said. You know that with the energy prices, it is only economically rational for maybe this lower tier resources at the most stressed times and for the grid. However, there it is the case where it is a condition to getting connected to the grid, right? And so in that case, it doesn't matter how economic your compute is if you do not have access to the grid. If it is a condition of your access, then it does make economic. It is economically rational to accept that because the rest of the GPU load that you get access to is so economically beneficial, right? So maybe you can't use it during this hour of this day or this hour of that day, but for the 8740 other hours of the year, you are printing money at a massive rate, and and we are seeing the hyper. Let's do this. By the way, Google like six months ago said they hit a one gigawatt of of internally flexible demand data center capacity rather, and I think the large low tariffs as they're being rolled out have this sort of flexibility built in. I think across the country, so we are seeing

Nico Johnson  35:21

so AI can pay 100 times what anyone else will pay. Serving it costs more than we're admitting, and getting connected might end up mattering more than either. This is where I wanted a translator of sorts. You heard him at the top of the episode. PKD is something of an icon, in my opinion, in this industry. 25 years plus, he's been involved at the utility level and consulting with the largest players in the industry. He helped build the demand response group at Constellation of Energy up to about 1,700 megawatts of dispatchable load, and patented the platform that it ran on. So when somebody tells you data centers can just become flexible loads, well, Peter is the one who knows exactly what has to be true for that sentence actually carry water. Co-founded Northbridge Energy Partners, and he wrote the Energy Switch, which I hope many of you have read or cue up. He also writes for the RTO Insider. In fact, his piece on the compute heat rate is how many of us even heard about Hans' work in the first place. He's got a way of making complicated things really sound simple. Starting with this one,

Peter Kelly-Detwiler  36:27

this does come down fundamentally to the capacity issue and the energy issue. And to use kind of a poor and sloppy analogy, I think of it as like Jurassic Park. SB six, for example, Senate Bill six says you can't connect. You can't get into the park unless you commit to letting us interrupt you. Like the everybody is anxious to get into the park. When Governor Abbott announced that all the data centers were going to have to be studied before any of them could interconnect, there was an estimate that the lost revenue on a monthly basis for a gigawatt scale data center was around 1.75 billion dollars per month. Russo talks about a gigawatt data center that they can, as a service, lease for 15 billion dollars a year. Okay, so we're talking, you know, 10s of millions of dollars per day. Once you let those dinosaurs into the park, the Velociraptors. Like, if you don't set up the rules ahead of time, once they're in there, they're going to eat everything because, to Hans's point, their appetite is voracious. Right? That they don't care what the price of the food is; they'll eat as much as they can get. And so, as as regulators, we really have to think about how do we set the rules of the game so nobody can get into the park unless they agree to behave in a certain way. And if they do start eating other animals, we kick them out. Now, a lot of the lot of the animals are now saying the Velociraptors. Well, I'm not going to go into the park because I can't connect to it anyway. So they're building outside the park with you know their own gas fire generators. GE, 20% of the turbines they've sold, you know, on a you know forward basis is data center direct load. Now there's going to be all kinds of problems there, but this is the kind of thing that we're dealing with, and it has inevitably it has impacts on everybody else in the system, and that's why I think what David and Hans are working on is so critically important right now.

Nico Johnson  38:19

What's the basic difference between training an AI model and using that model for inference, and why might those two kinds of load behave differently on the grid?

Peter Kelly-Detwiler  38:27

Your training models, your large language models, the LLMs, what they do is ingest tons of data, raw data, tokens. Mary had a little lamb. Period is Mary had a little lamb five plus the period six tokens, and what the large language models do. Let's just talk about language first. We won't talk about math and all these other things. But what they do is they take all the all the different combinations of words that create sentences and books and paragraphs and so on, and they try and figure out. It's like you know when you text right now, and the next word pops up on your text that's the one that you can choose from. That's because models have been trained to figure out well what's the next likely output going to be. So you start to create this generative AI where it can begin to quote sort of think for itself. Okay, it's why you can now have a digital movie where it makes the movie, or you have digital novels that nobody's written. The machines have because they've figured out they've built the neural networks like our brains are, and they can now do it on their own. And they're now at the point, by the way, they can start to program and create their own code, and so they become more and more complex over time. So the LLMs, the large language models, you basically set them and run them for sometimes months on end, and they use trillions of tokens, and they also use trillions of parameters, which are the weights that essentially are the the way that you teach it to think about the data that it's ingesting. Okay, so then you create a model. All right, so now. Let's assume we'll shift to self-driving cars. We've created a model for Waymo or Tesla where it where it looks at all these things and it can recognize, oh, that's a cat, that's a woman, huh? That's a woman pushing a stroller. Now, if I see a woman pushing a stroller at the same time a cat comes down a tree and starts to cross the road, and they're both in front of me. Now I'm in inference mode. My car takes that information and immediately decides to run over the cat and not the woman with the stroller. I've taken the model and now I've used it in the real world to make a decision. Okay, does that make sense?

Nico Johnson  40:35

Yeah, it does.

Peter Kelly-Detwiler  40:37

So inference can affect high speed trading, or it can help us make movies, or you know, do all kinds of different stuff. But it's essentially the application of what we learned in the models we created, and so the models, therefore, they can run flat for long periods of time, like 80% load factor, somewhere around that, and they stop for certain periods of time, and then you load them up again and you start. Inference tends to be more clustered around human behavior because it follows human activity, so it tends to have a lower load factor and is more peaky, and generally has to be closer to people because it has a lower latency because you don't want to run over the woman with the stroller instead of the cat.

Nico Johnson  41:19

I wonder how we help folks understand the relevant economic model or economic value associated with both.

Peter Kelly-Detwiler  41:28

Yep. Okay. So, if you're sort of feeding that animal and teaching it to learn, it has a little bit more plasticity in it. Okay. It can run 24/7, and it doesn't need to make an immediate decision. So the training, yes, it creates economic value, but it doesn't generate the same level of economic value that a high-level inference type of activity would, where it it can affect some economic activity or something related to medicine or not running over the woman with the stroller, right? So, so one of them's what the thing does in the classroom, essentially, and then once that thing graduates and goes out into the world and starts doing stuff, that in theory can can create a lot more economic value per kilowatt hour consumed.

Nico Johnson  42:12

I've mentioned the brief episode that you did on compute heat rate. You did a great article in RTO Insider on it as well, and you asked a simple question that really helped me frame this whole discussion, which is how price sensitive our data center loads, and at what price would they decide not to consume the power? But I wanted to get your perspective on why this captured your attention. Why is this such an important question?

Peter Kelly-Detwiler  42:36

I'd met Hans before. One day he reached out to me on LinkedIn and he said, "Hey, I have this concept for you, and to be fair, a number of people reach out to me on LinkedIn with sort of perpetual motion machines or other types of technology they want me to get interested in, and a lot of times I have to do the Heisman because I only have so many hours in the day. But the concept intrigued me because I'd met him before. I knew he was really intelligent, and he and just in that brief description, he explained to me what he was trying to do. So I was like, "Oh, okay, I'll take the call. Still reluctantly, because you know opportunity costs.

Nico Johnson  43:08

Yeah, I got a bit.

Peter Kelly-Detwiler  43:09

And then we got on the call, and he started to explain it, and kind of like the hairs went up on the back of my neck because because I recognized we all already could see the massive juggernaut of data load coming into the grid, and here's a guy who's explaining these elasticities, these price sensitivities, and how it could affect price formation in various grids, ISOs and RTOs, and maybe even based upon where in the grid and how much consumption there is relative to transmission capability. So at some point, it actually could create the price formation function that shifts the locational marginal price. And so as we started talking about it, my mind began to whir, and I thought, oh, there's a there's a there there I think. And so then I had my editor from RTO Insider say, hey, what's the next piece you're going to write about? I said, I just talked to this guy Hans Royal, and he explained this concept. And if I'm wrong, I'm going to come across looking like an idiot here. But it felt risky to me. But it also felt like there was really something there. Yeah. So I thought, all right, I will take the risk and put this thing out into the world. And with a little bit of trepidation, and and with Hans's help, you know, he read it, made started look like an idiot, which was very helpful. And now it turns out it's one of their top read pieces throughout the whole year. It stimulated a really high level of curiosity, and apparently it was one of you know was PGM that read that piece and then started to integrate into some of their thinking style. So it was. I was fortunate that I recognized an idea that was way smarter than anything I would have come up with. But I was I was smart enough to recognize how good it was, and then and then just sort of explain it and throw it out into the world.

Nico Johnson  44:54

You built demand response long before data center flexibility became a big topic. What's the difference between showing that a data center can reduce its load and giving the grid something that it can actually depend on?

Peter Kelly-Detwiler  45:07

Yeah, so back in the day when I was doing it at Constellation, we had the second largest virtual power plant in the world, but it was fairly limited in some ways, and you only had to do you know x number of hours per year, x number of events, and so on, and it wasn't a precondition to entry, which you have to make for because these lows are so big, they fundamentally change the supply-demand dynamics. Now, what's really interesting about this whole topic is that we are all dragged intellectually, kicking and screaming in some instances, to have to understand how data centers work because there's we can ignore how aluminum smelters work or Howard Johnson's or you know a Chick Fil A or something because it's the law of large numbers. This is a large number, so it it really changes what we have to have to do and how we have to think about it. You know, and so with with with response with with the question of demand response, we have to understand okay how much of this is real. Emerald AI, for example, they just what raised a billion dollars or something, and their whole software function is predicated on okay, what are the loads doing in the data centers, and then what's the grid doing, and when do we get our signals? And then what do we do? Do we shift to batteries? Do we change the compute function in the facility? If so, what's the facility doing? And which of the things can we terminate right now or time shift? And which ones can we locationally shift to someplace else if there is another location? So there's multiple activities, for example, that Emerald AI is in theory doing it once, and they're they've now done this five places around the world, and they did a three-hour, 30% reduction in Phoenix, and they have a 96 megawatt data center they're building with digital realty in Dominion Service territory with EPRI and others, but nobody's proven this yet empirically that we can rely on it and let and sort of lather, rinse, repeat it.

Nico Johnson  47:04

Yeah, and

Peter Kelly-Detwiler  47:04

so that's the next step for me. Is okay. I'm in Missouri. I'm in the Show Me State. Like, yes, theoretically we can do all this, and my implication is hold them to a very high standard and make sure that they actually can do this, not just say they can do it, or do what SB six says in Texas, which is, hey, I don't care what your economics are, we're going to cut you off before we do any rotating blackouts to anybody else in the

Nico Johnson  47:31

grid. Yeah,

Peter Kelly-Detwiler  47:33

or chills in in SPP self plus power pool. They basically say, hey, we can interconnect you really quickly, but you have to commit to certain levels of interruptibility, I think ultimately this could become a very, very powerful tool and make the grid better. And this willingness to pay that data centers have means that in theory they could turn around and buy a whole heck of a lot of demand response from everybody else across the system at prices I could never pay for at Constellation when I was running my DR group. If we have a grid that's like 45% capacity utilization in the distribution grid and maybe mid-50s in the bulk power system, there is all kinds of waste built into that system. If it was an airline, we'd go everywhere we wanted two days before Thanksgiving, walk up to the counter, get a ticket, get on a plane, and not pay more than 250 bucks. Yeah. Then the rest of the year, I'd be psyched because at six foot four, I could lie across three seats. But most of the airlines would follow Spirit very, very quickly. Right. That's how we operate our grid.

Nico Johnson  48:31

Yeah.

Peter Kelly-Detwiler  48:31

Plus, yeah. If it was airlines that have another 15% planes parked in hangars someplace, never used in case a few of them broke. Right. Yes. So there's plenty of waste inherent in the system or inefficiencies that, in theory, if we could pull down or keep the peak where it is and pull up the shoulders, would create goodness for everybody. If we can do that,

Nico Johnson  48:53

kind of going back to to Hans, if AI loads really are willing to pay many times what the traditional system or industrial customer will pay. When does that actually start to affect the price everybody sees? Right. That's. I think that's. That is an overarching question. That I don't think we have an answer for yet. It's one that David Siap and his work around MCC is trying to contemplate. It's something that, on the other end of the spectrum, the the compute heat rate tries to help us think through PKD. Where do you fall on that?

Peter Kelly-Detwiler  49:25

So first of all, we should differentiate between infrastructure capacity-related stuff and then energy. Okay, so with capacity, you can fill up what existing capacity you have, and then you have to start pushing the flexibility thing pretty big. And then in theory, like for example, Indiana Michigan Power says because of data centers and increased utilization, they're going to be able to offer $53 million back to customers in rate, you know, reduce tariffs and freeze rates for three years. That's fine until you hit the point at which you need to build new infrastructure, and then. Because of all this data center demand, switchgear, transformers, turbines, everything costs 50% to 100% more than it did three years ago. Because supply is pretty inflexible and inelastic, so that's one piece of it that maybe you could lower distribution costs by pushing more megawatt hours across the system. But you know, we were having a conversation the other day. Hans and I and one of his colleagues saying, "Well, what happens if you're in a place like Texas or someplace else, and you add 10 gigs of demand into the system? Now your dispatch curve shifts. You know, you've already burned through your renewables, you burn through your lower cost gas, maybe, and some of things. Now, on every day, you're higher up the curve every single day. So, what do the costs do in the bulk power system? And this is another thing that you know these folks at CHR are leading the thought process on of like what happens when you when an 800 pound gorilla jumps into the pool? How does that impact power prices for everybody in competitive wholesale markets? Where if supply doesn't keep up with the demand function, this is what happens. A

Nico Johnson  51:04

lot of folks are looking at this, David and others, through the investor lens. We're talking about infrastructure that could last 30 or 40 years, built around demand forecasts for an industry that is changing faster than any of us can keep track of it. So, how do you make a responsible long-term infrastructure debt or decision when you don't really understand or trust the short-term forecast.

Peter Kelly-Detwiler  51:27

Let's first of all look at PJM's three base residual auctions for capacity as Exhibit A of what not to do. So there have been many conversations around phantom load, and the estimates are they could be 3x to 10x what actually gets built. Among other things, there's finite amounts of chips. There's certainly finite amount of of of equipment in the supply chain. Now, what happened to PJM was besides the fact that they reaccredited capacity of existing generation, so there was less supply in that auction to start with, you had these forecasted demand of data center loads coming in, and nothing on the supply side because the interconnection queue was all clogged up as well. So over the three auctions, the price went from like say it went from in the 30 to $40 range to 269 90, and then Governor Shapiro, the technical term is he freaked out, and he put in a cap for two years of 325, a ceiling of 325 per megawatt day, a floor of what 175 a megawatt day, and over the next three auctions, because it got extended, the price hit the cap. And PGM in a white paper said, "Well, we still didn't get the capacity we need to meet this forecasted demand. We are six gigawatts short, right? And and and and therefore they think the price is probably north of $555. They ran the simulations, and that's what it looked like. Now the market monitor comes in and runs their own simulations and said, "Over the $26 billion or whatever, no, 40 something billion that was spent in these capacity auctions. Close to 50% of it was because of these forecasted data center loads, many of which won't materialize in the time frame they were projected. Okay, so everybody in PGM society at large in those 13 states, 21% of U.S. GDP is having 10s of billions of dollars added to their power bill for something that may be a ghost, or at least a lot of it. Right? That that gives you a sense of why it's so critically important to get the forecast right. Now, what can you do? You can start to do what Texas is doing with Bad Zero and say, "Look, we if you want to play this game? It ain't free. You put serious money on the table, $50,000 per megawatt, to get involved in the studies, and then you have to do the studies showing what's going to happen with power flows, etc. Up until now, it's been free optionality. So if someone has $50 billion to spend, they throw some here, some here, some here, some error, and if these ones succeed, they pull these chips off the table, and no one charges them anything for it. So the first thing, again, with people with that kind of willingness to pay, they're rational economic actors. They want the greatest return for their investment. They're neither moral or immoral; they're amoral. Treat them that way. Make this $1 and cents thing. But then the other risk is, who knows? With the chips changing all the time, we don't know what society's appetite for data is. And the longest one of these tariffs so far that I've seen is 20 years. So that means you're still rate basing a whole bunch of equipment for 30 or 40 years. Now, since go back to the point, ChatGPT only jumped out of that jack in the box in November of 2022, and now we're making all these forecasts for an industry that's only existed for three years at scale for 30 or 40 years from now. Who knows what the world is going to look like then? So, the risk issue is a huge one, and the longer the contracts can be with reputable. Players with capital and with you know lines of credit, something you can attach to if they fail, is going to be critically important to protect ratepayers.

Nico Johnson  55:09

What is one thing that you could see happen over the next 12 to 24 months that would materially change your view of what AI is going to do to the electric grid? The

Peter Kelly-Detwiler  55:17

one I see right now is they're going to lose their social license to operate very very quickly, 70% of people who were surveyed in the most recent survey don't want data centers within three miles of their homes. The fact that Anthropic had three people, one quit and said he sees a serious risk to humanity from artificial intelligence, and then another person who's still an employee said he thinks by the end of this decade, there's a 10% chance it could lead to human extinction. That is social license to operate with a capital S. Yeah. Oh, right. And so I think we're entering into a situation where really, even though people are saying it's the data center, I don't want three miles from my house. What they're really saying is, I don't want to lose my job. I don't want my kids' world to be inhabited by robots, where they have a relationship with other human beings that we can't fathom right now. Where I don't know where my job's going to be. The level of existential uncertainty and and fear right now is growing on a daily basis. Just read the headlines. Look at the war in Ukraine that's now AI driven, and so I think the biggest thing that's going to change this is the social license to operate. I mean, when Governor Abbott is saying, "Whoa, whoa, whoa, slow down, it's one thing if it's Governor Hochul in a blue state, but when the red states start going, or Mike Rogers in Michigan, and when the Republican Party says, hey, we need to think about this differently because if we are gung ho on this, it could go sideways on us real fast. That to me is the single biggest thing. Or the other one is if China dumps a whole bunch of tokens into the U.S. in an effort to sort of disrupt our markets. This is this is a bipolar competition right now. There's a huge national security element involved in it, and I think we're also going to see at some point one of these big companies doesn't make the earnings they projected they were going to, and that's going to cause some perturbations too. But that'll still probably, like in the.com age, the soft puppet companies went defunct, but the grid, yeah, but I'm sorry, but the internet continued to prevail, so you know who knows. But I think that social license to operate becomes really huge within the next few months.

Nico Johnson  57:27

Okay, I asked David and Hans the same question. Here's what they had to say.

David Siap  57:32

What I'm looking to see is the the green shoots. So for me, what that looks like is adoption of advanced clean energy, increased adoption of of demand response, and increase this sort of business model and technological innovation. So when when I think of what AI can do on the positive end, it is fund those sort of things, fund this new paradigm where flexibility all of a sudden is incorporated in utility processes and IRBs in a more meaningful way, and and also develop, say, you know, next generation geothermal, and and and maybe even advance nuclear in in the far in the longer horizon. So I take the optimist angle to that question, and I think that that that's that's what I would would like to see happen rather than that a 10% you know humanitarian humanity destruction that they deep credit.

Hans Royal  58:31

Look, I see both a a risk and a tremendous opportunity. So, what could happen in the next 12 or 18 months to change view? I'm always an optimist, so I always think markets will figure things out, and I'm frankly an optimist about the future of AI. But the social license to operate is super important right now, and I think the economics suggest that there is a big opportunity for data centers to invest in new technologies to pay their own ways, so to speak. Do it in a smart way that actually does result into reduced costs for everyone else on the grid. In fact, why not build tremendous amounts of clean energy to offset the costs of the data center build out for ratepayers across the entire country? The economics justify it, and so I think that is one thing that is an opportunity that would change my my view of of the risk. The risk, obviously, is that you know we don't go in that direction, and the not in my backyard sentiment becomes more and more stark, and this becomes more and more of a political football, which inhibits not only the development of data centers, but also the development of new energy technologies and the advancement of the grid, which we need regardless of AI. We need our grid to be modernized anyway. So I think that's the risk. But the opportunity is is to actually take advantage of this and look. If in 12 months or 18 months, data. Centers actually operated at scale as flexible grid resources. There's tremendous value in that, and I just think the economics and the the tools and the toolkit need to be there to make that a rational economic decision for all parties involved.

Nico Johnson  1:00:13

I've been trying to get my head around this topic for months before we get a chance to sit down and really dig into it. I understand a great deal better now than I did before. This whole conversation came from a paper that Hans and David wrote together for Creo called "Clean Energy's AI Moment. I've borrowed the title, frankly. We'll of course link to it in the show notes. I hope you'll go and read it further if you're so inclined. But I hope that the last hour has helped unpack this for those of you who, like me, perhaps are a little too busy to do all that reading. As we've discussed, you can find Hans Research at computeheatrate.com and on his Substack. David's earlier Creo paper, Affordability is all you need, which is where MCC came from, and Peter's writing his book and his podcast, which I highly recommend, Energy Future, will all be down in the show notes. I hope you'll go follow all three of them. And if this conversation was useful to you, please help support their ongoing work and ours. Subscribe and share this with the rest of your community. Here's what I'm taking away from this conversation: We spent two decades in an industry where demand didn't grow, and we are very good at optimizing for that world. That world, frankly, has crumbled under our feet, and the thing replacing it is willing to pay roughly 100 times what anyone else will pay for the same electron. That's either the best thing that's ever happened to clean energy, meaning the capital that finally modernizes the grid, funds the flexibility and builds the firm generation that we've been saying we need for 20 years will flow in, or it's 10s of billions of dollars landing on ratepayers' bills for capacity that never gets used. Both of those are up in the air right now. Which one we get depends on decisions being made right this very moment in rooms most of us don't participate in, but we could. So pay attention to this ongoing conversation. If you're building in this space, the economics are more on your side than they ever had before. Remember, you are what you listen to. So thanks again for showing up, Solar Warrior. It's half the battle.

‍