THEWHITEBOX
AI Is Profitable: The Real Question is How Much

You’ve been told AI can’t make money. I’ve said it myself, and that thinking AI will make money one day was more a matter of faith than numbers.

But now, I can confidently say I would be wrong to keep thinking that, because something has changed.

Over the next months, we’re going to see markets adjusting to this new reality: AI is no longer about whether it can make money, but how much and by whom.

Let me explain to you why my previous ideas about AI not being able to make money were wrong (or no longer true under new evidence). We’ll first review the industry, and then we’ll obviously look at what recently changed to fuel my newfound optimism.

However, problems remain. Ironically though, the problem most investors worry about is largely solved, while the real problem lies in the area they already consider solved.

What is the ‘AI industry’?

The AI industry is divided into four layers with considerably different business models and constraints: hardware, infrastructure, models, and applications.

  • Hardware companies are the set of companies that design and build the accelerators. Companies here include both designers (NVIDIA, AMD, or Google) and manufacturers (TSMC, SK Hynix, Lumentum, and basically the entire semiconductor industry).

  • Infrastructure companies build the data centers to host those servers and offer them to other companies. These companies can also host models and serve them. These include the Hyperscalers (Microsoft, Google, Meta, Amazon), neoclouds like Coreweave, etc.

  • Model companies use that hardware to train models and serve them. This includes OpenAI, Anthropic, Mistral…

  • Application companies build solutions on top of models and charge for the product itself, not the AI.

I don’t need to explain how NVIDIA makes money; it’s obvious. For the others, however, things are trickier than it seems.

The token and margins

Besides strictly hardware companies, the rest on that list have a fundamental relationship between money and tokens. Tokens are just the basic semantic units models break data into. Text into words and syllables, images into patches of pixels.

This is not only necessary for the AI to process and generate data, but it’s also key to meter the AI service. In other words, how much AI costs or charges is based on the number of tokens.

If you send a text that gets broken down into 100 tokens and the AI responds to you with 500 additional tokens, you are charged for 600 tokens; the 100 input tokens get charged at one price, the 500 output tokens at another price, which is always several times higher (this is because output tokens are more expensive to generate relative to how costly it is to process input tokens).

For example, if the input price is $10/million tokens and the output token price is $50/million tokens, that query resulted in a charge of $0.026 for you, or two cents.

You do get charged more because the AI is handling additional input tokens in the background, because for every request you send, the provider adds a lot of additional text to the model to handle behavior, personality quirks, and so on, a part of the overall prompt called the system prompt.

Knowing this, it’s very easy to see what the business models in each case look like.

For infrastructure companies, their business is literally selling tokens, which is why Jensen calls AI data centers “token factories”. Therefore, profits are measured by the price per sold token minus the cost per token.

If they sell tokens at a fixed margin relative to costs, there’s no way around it: they make money. But here’s where we get to one of the main confusions people have with AI. AI does not have a margin problem; it’s very easy for them to add a margin on top of every token they process and guarantee profits. Their issue is cash flow.

This is because you can be profitable and still lose money. For example, if you buy a lemonade stand for $100k and you have revenues of $80k and costs of $79k, you’re profitable; you’ve made $1k, but it will take you 100 years to recover the investment.

Is this a good business?

But AI businesses look even worse, because it’s not about buying a lemonade stand and calling it a day; they need to buy 10 lemonade stands every year, and inside each lemonade stand they have lemonade machines that are absurdly expensive (around 70% of overall cost), break down like paper on water, and have a very short life (around six years).

In case you’re wondering, the ‘lemonade machine’ is the GPUs and interconnect fabric in data centers.

So, for every lemonade stand you buy, every six years, you’re going to have to spend an additional $70k, while you’re still making only $1k profit for every lemonade stand.

So, yeah, you’re profitable, but you’re also burning cash like there’s no tomorrow. There’s a reason public companies have to publish both an income statement and a cash flow statement!

This is why, for the life of me, I can’t understand the industry’s fixation with margins when we’re talking about an incredibly capital-intensive technology. For such business models, only cash flows should matter!

Therefore, for the infrastructure companies, the question is not: can they make money? They can. The question instead: Can they make enough money?

Alternatively, infrastructure companies can also sign GPU rentals charged on the hour. For example, Anthropic signed a $15 billion/year deal for 300 MW of Blackwell GPU compute.

A Blackwell GPU, considering all the other components of the server, not just the GPU, requires around 1,400 W, so that’s roughly 220,000 GPUs. Therefore, Anthropic is paying around $8/GPU per hour.

This is a fixed price, so SpaceX isn’t charging a price based on how many tokens Anthropic manages to sell; it’s a fixed $8 per GPU per hour.

For the model Labs like OpenAI and Anthropic, the business is similar, but has to consider a key difference: much larger operating costs.

AI Labs make money the exact same way as infrastructure companies (when they aren’t signing rental deals like the one above). However, unlike the latter, they have to first train models.

Training a model is very expensive, not just because of the training run and the experiments associated with it, which can grow into the hundreds of millions or even billions, but because they have even larger expenses on three fronts:

  • Data: They have to spend enormous amounts of money securing new data for training. This data is usually “manufactured” by companies like Mercor that gather experts on given areas and pay them to generate it. These experts can cost hundreds an hour each. I won’t get into the details today, but you also have to pay for synthetic environments for RL, which can cost millions too.

  • Workforce: The workforce in these labs is some of the scarcest, priciest human beings on the planet, with some making dozens of millions and even hundreds of millions of dollars a year in salaries. Even if most of that salary is paid in equity, that’s still an expense that gets reflected on income statements.

  • Marketing: AI Labs don’t have the distribution networks companies like Google have, so they have to invest heavily in marketing.

Combined, these companies are spending way more than they are making. Remember that these three big cost fronts are added to the compute costs, which can be rent if they are renting data centers from others, or depreciation costs if they own the data centers. Still, unless enough proof shows otherwise, AI Labs are still incapable of making money.

And for the application layer companies, companies building software on top of AI models, their business is the most straightforward of them all but simultaneously probably the hardest: you’re essentially playing a low-margin game.

AI is not only a big portion of your product that you largely don’t own, but also the tool for others, including your own customers, to compete with you.

To me, this is an inevitable sign that software is going from a high-margin business to a low-margin one for most segments (of course there’ll be exceptions in areas like embedded software).

The biggest reason, besides competition, is that this software will have to be tailor-made for agents to use, not humans. That is, lightweight and cheap. That means that a software company can no longer spend 70% of total costs on operating costs like salaries.

The cost structure I’m ranting about is ones like these below:

You can’t spend 72% of your costs on salaries and marketing. Because if you do that, you’re going to soon see a competitor offering your product, same features and everything, for 100 times fewer costs because they run the same low production costs plus a hundredth of your operating costs because their team is 20 people, not 7,000.

And yes, 20-strong startups with impressive software offerings, something completely impossible before coding agents, now exist.

I know this is an unpopular thing to say because I’m basically predicting massive layoffs in the software industry, but I hope you understand my goal here is to tell you what things will happen, not things I would want to happen.

My view on SaaS is even more dire when you factor in moats and what I believe is the industry’s most likely future, agent tooling, but I went deep into that here in case you want to better understand my views about the future of software.

Again, as with all things, there will be exceptions, and I think there’s room for high-margin software in key areas where execution is not only about lines of code but other stuff (regulation via certification, distribution effects, systems of record, and such), but my overall view of the software industry is that it’ll be one of low-margin products.

Overall, however, at first glance, AI simply looks like a bad business. Let me explain.

The factors working against profits

There are multiple reasons why AI can be seen as a bad, even terrible, business.

  • Despite AI software being basically commoditized, everything is built on top of extremely expensive, non-commoditized hardware. NVIDIA and Micron command +75% gross margins, meaning they charge at least four times the production costs for every product they sell

  • The product being built on top of this hardware is being heavily pressured to drop prices because the recipe to train models is accessible to anyone with the money to pay for it. There are differences, naturally, but the technical bar to produce a good model isn’t as high as most people think, explaining why there are dozens of AI Labs launching good models.

  • Open models are released for free, meaning those labs charging for closed models are competing with what’s basically “free” software (it’s not actually free, though; you still have to pay to run it, and not a little).

Reading all this, one has to feel terrible about the prospects. However, I’m starting to change my mind, and if you look closely, some AI companies are poised to make a lot of money.

But to do so, it requires a gargantuan effort to redefine the very principles of hardware and software, and clarify where AI’s real issue persists today. Luckily, however, that change is finally taking place.

Ironically, the problem most investors worry about is likely to be solved, while the real problem lies in the area they already consider solved.

logo

Subscribe to Full Premium package to read the rest.

Become a paying subscriber of Full Premium package to get access to this post and other subscriber-only content.

Upgrade

A subscription gets you:

  • NO ADS
  • An additional insights email on Tuesdays
  • Gain access to TheWhiteBox's knowledge base to access four times more content than the free version on markets, cutting-edge research, company deep dives, AI engineering tips, & more

Keep Reading