TL;DR — Generative AI gives Google Search something it never had: a real marginal cost per answer. Alphabet’s defense is custom TPUs and aggressive model efficiency. The operating margin held at 32% for 2025 and hit 34% in Q2 2026, but the bill moved to depreciation and free cash flow, which turned negative in Q2 2026.
For two decades, Google Search ran on economics few businesses ever get to see. The expensive work of crawling and indexing the web was done in advance, so each extra query cost almost nothing to serve while advertisers paid for the clicks it produced. Generative AI changes that. When Gemini writes an answer instead of retrieving a list of links, every response burns fresh compute, and Alphabet’s most profitable product picks up a real marginal cost for the first time.
Generative answers turn Search into a business with variable costs
The old model rewarded scale almost without limit. In 2024, Google Services earned $121.3 billion of operating income on $304.9 billion of revenue, a segment margin of roughly 40%, and Alphabet as a whole posted a 32% operating margin. Most of the cost base was infrastructure that every additional query used at almost no extra cost.
Inference breaks that pattern. A large language model runs a new computation for each response, and the cost rises with model size and answer length. More AI usage means more accelerators, more electricity and more depreciation, which is why investors started treating capital expenditure as a margin variable rather than a pure growth signal.
Alphabet’s own spending shows the scale of the shift. Capital expenditures were $52.5 billion in 2024, and in October 2025 management raised its 2025 guidance to a range of $91 billion to $93 billion. That money reaches the income statement over several years as depreciation, which is exactly why margin pressure from AI tends to arrive quietly and late.
TPUs and model efficiency are Alphabet’s main margin defense
Alphabet’s answer is vertical integration. Instead of buying all of its compute at merchant prices, it designs its own Tensor Processing Units (TPUs), now in their seventh generation with Ironwood, and runs workloads on them alongside Nvidia GPUs. Owning the chip lets Google keep the margin a supplier would otherwise capture and tune hardware and models together.
The same chips are rented to Google Cloud customers, so part of the investment earns revenue directly instead of only lowering internal costs. We cover the chip roadmap in detail in our analysis of Alphabet’s TPU hardware moat.
The efficiency data available when this piece first ran was striking. In August 2025, Google reported that the median Gemini Apps text prompt used 0.24 watt-hours of energy, and that energy per median prompt had fallen 33-fold over the prior 12 months. Energy is only one component of serving cost, but a curve that steep suggests the per-query penalty is shrinking fast.
That is the real test for the stock. Much of Wall Street’s attention goes to ranking whose model is smartest. The more useful question for a shareholder is whose model is cheapest to serve at a given quality, because Alphabet has to disrupt its most profitable product with a more expensive one before a competitor does it first.
The compute tax is real. So far, Alphabet has paid it out of the cash flow statement rather than the operating margin.
Update: margins held through mid-2026 while free cash flow took the hit
Updated October 2026. Alphabet’s full-year 2025 results, reported on February 4, 2026, showed an operating margin of 32.0%, flat with 2024, despite a $3.5 billion European Commission fine and a $2.1 billion Waymo compensation charge. The Google Services margin rose to about 40.7%. On the earnings call, Sundar Pichai said Gemini serving unit costs fell 78% over 2025.
Demand is pushing the other way, which is why efficiency matters so much. Pichai also said queries in AI Mode run about three times longer than traditional searches, and Google Search & other revenue grew 17% in Q4 2025 to $63.1 billion. Longer, more complex queries cost more to answer, but they also appear to be monetizing, at least so far.
The cost did not disappear; it moved. Alphabet’s 2025 annual report says serving AI offerings will “require more compute power than our historical consumer and enterprise offerings.” The filings show where the bill lands:
- Depreciation: $21.1 billion in 2025, up 38% from $15.3 billion in 2024.
- Capital expenditures: $91.4 billion in 2025, with 2026 guided to $175 billion to $185 billion.
- Free cash flow: $73.3 billion for 2025, then negative $5.9 billion in Q2 2026.
- Operating margin: 34% in Q2 2026, up 2 percentage points year over year.
The second quarter of 2026 made the split plain. Operating margin expanded and Google Cloud earned a 35.6% segment margin, a business we track in our Google Cloud coverage. Yet quarterly capital spending reached $44.9 billion and free cash flow went negative. The original thesis holds in modified form: the threat is still quiet on the income statement and loud on the cash flow statement.
What to watch next: three numbers will show whether the defense keeps working. First, depreciation growth relative to revenue growth as the 2026 build-out lands on the income statement. Second, the Google Services operating margin, which is where any per-query cost creep in Search would surface. Third, any new disclosure on serving cost per token; if management stops reporting efficiency gains while depreciation compounds, the margin pressure this piece flagged will stop being silent.
Sources and disclosure: Alphabet’s Q4 2025 earnings release, 2025 Form 10-K, Q3 2025 release and Q2 2026 release, all filed on SEC EDGAR; Alphabet Investor Relations; Sundar Pichai’s Q4 2025 earnings remarks; and Google Cloud’s August 2025 inference energy paper. The author holds a long position in Alphabet (GOOGL/GOOG). Nothing here is investment advice. Our full disclosure policy is on the About Third Pole Markets page.
