---
title: "AI consumption KPIs: cost per conversation, tokens per resolved intent, and margin per seat"
canonical_url: https://ampl.webclat.com/guides/ai-consumption-kpis
description: "The consumption-KPI rail for usage-priced AI features: cost per conversation, tokens per resolved intent, and margin per seat - what each measures and how to instrument it in your product analytics."
source: Webclat | Amplitude Solutions (official Amplitude partner, independent consultancy)
---

# AI consumption KPIs: cost per conversation, tokens per resolved intent, and margin per seat

**In short:** When an AI feature is priced by usage - tokens, API calls, compute - the KPIs that matter shift from "how many people used it" to "what did it cost to deliver, and does the value it created cover that cost." Cost per conversation, tokens per resolved intent, and margin per seat are the three consumption KPIs that connect AI spend to the same funnels and cohorts a product team already tracks, instead of leaving it in a finance spreadsheet nobody in product ever opens.

## What it is

Most product KPIs - DAU, activation, retention - measure whether people show up and keep coming back. Consumption KPIs measure something classic product analytics was never built to answer: what did serving this specific interaction actually cost, in tokens and dollars, and is that cost sustainable at the usage level you're actually seeing. This matters specifically because AI features are usually priced by usage - tokens, API calls, compute minutes - in a way a normal app feature (a button, a page view) simply isn't. A feature that's popular by every engagement chart you own can also be the one quietly losing money per use.

Three consumption KPIs cover the rail. Cost per conversation is the total token and API cost of a full agent session, divided by session count - the direct read on unit economics per interaction. Tokens per resolved intent is tokens spent divided by successfully resolved user intents, not just total messages sent - it ties spend to whether the agent actually did its job, not just how much it talked. Margin per seat (or per user) is revenue attributable to the AI feature, minus its consumption cost, per paying seat - the number that answers whether a usage-priced AI feature is profitable at your current pricing, not just popular.

None of this works from a billing dashboard alone. A provider's own billing page tells you total spend, not spend per outcome - you need the token and cost figures joined to the same session and user_id your product events already use, so a spend spike traces back to a specific cohort, feature, or failure mode instead of showing up as one lump number in finance's monthly report.

## When you need it

- Your AI feature is priced - to you by a vendor, or by you to your customers - on usage rather than a flat seat fee, and nobody on the product team can currently answer "what does this cost us per interaction" without asking finance.
- A feature is popular by every engagement metric you track, but leadership suspects (or finance has flagged) that it isn't economically sustainable at that usage level.
- You're about to price or re-price a usage-based AI feature for customers and need your own unit economics before you can set a margin-safe price.

## Implementation, done properly

1. Capture cost and token counts as event properties on the same agent-session events you're already instrumenting - `input_tokens`, `output_tokens`, `estimated_cost_usd` on the turn or session event - rather than reconciling a separate provider billing export against your analytics data after the fact.
2. Define "resolved intent" explicitly and durably before computing tokens-per-resolved-intent, using the same task-completion or quality signal your agent-quality measurement already produces. An undefined or shifting definition of "resolved" makes the ratio meaningless as a trend line the moment anyone changes how it's computed.
3. Build cost per conversation as a computed metric - total session cost divided by session count - segmented the same ways you'd segment any other product KPI: by cohort, by intent cluster, by plan tier. That way a rising average isn't hiding one expensive intent cluster dragging up an otherwise healthy number.
4. Attribute revenue at the same grain before computing margin per seat. If the AI feature is bundled into a subscription rather than billed separately, decide - with finance, explicitly - what revenue share it's credited with. An unattributed or arbitrarily-attributed revenue number makes margin per seat a figure nobody will trust in the room where it actually matters.
5. Put all three metrics on the same dashboard as the engagement metrics they sit next to - DAU, activation, retention - so a stakeholder sees usage and unit economics together, not split across two tools maintained by two different teams.

## How to verify it worked

1. Reconcile a day's total estimated cost, summed from your event properties, against the actual provider billing statement for the same day. A persistent, growing gap usually means a token-counting or pricing-tier assumption baked into your estimate has drifted from the provider's current rates.
2. Pick one real, known-expensive session - a long conversation, a heavy tool-call chain - and confirm its cost-per-conversation figure is plausible by hand-checking token counts against the raw trace, not just trusting the aggregate.
3. Confirm tokens-per-resolved-intent moves in the expected direction after a genuine efficiency fix - a shorter system prompt, a caching layer, a smaller model for a simple intent. A metric that doesn't respond to a known improvement is measuring something other than what you think it is.
4. Re-run the margin-per-seat calculation on a standing cadence, not once at build time. Both sides of that ratio - your price, the provider's rate - move independently of your product, and a KPI computed once at launch goes stale exactly when a rate hike or a promo pricing change makes it matter most.

## FAQ

### What are the most important AI agent KPIs?

A short list spanning three rails, not one: adoption (usage of the AI feature, activation specifically for it), quality (task completion, agent success rate, user friction), and consumption (cost per conversation, tokens per resolved intent, margin per seat). The common mistake is tracking only the adoption rail - a heavily-used, unprofitable, low-quality agent looks great on an adoption-only dashboard.

### What is agent success rate?

The share of agent sessions that resolve what the user actually came to do - distinct from a raw completion rate or an absence-of-errors rate, either of which can look healthy while the agent still fails the user's actual intent. Amplitude's own Agent Analytics frames one version of this as a task-completion signal; verify the exact definition against current documentation before adopting a vendor's number as your own success-rate metric.

### What are LLM product metrics?

The metrics that connect an LLM-powered feature's technical behavior - tokens, latency, evaluator scores - to product outcomes: adoption, retention, revenue. That's a different category from pure model-evaluation metrics that live entirely inside an ML team's own tooling and never reach a product dashboard; this page is about the product-facing half.

### What are AI assistant metrics?

The same category as LLM product metrics, applied to an assistant specifically: adoption (who uses it, how often), resolution or success rate (did it actually help), friction (did the user have to fight it), and - the axis this page adds - consumption (what it cost to deliver).

### What are AI adoption metrics?

The subset of the above concerned only with usage: who uses the AI feature, how often, and what share of eligible users ever try it. Adoption metrics alone can't tell you whether the feature is good or economically sustainable - that's exactly why they need to sit next to quality and consumption metrics, not stand in for them.

### What are chatbot KPIs?

The same three-rail list - adoption, quality, consumption - applied to a chatbot specifically. The framing is identical whether the AI surface is a chatbot, an in-product copilot, or an agent calling tools; what changes is only which events you instrument to measure each rail.

Related: [Guide: Amplitude Agent Analytics](https://ampl.webclat.com/guides/amplitude-agent-analytics), [Pendo Agent Analytics vs Amplitude Agent Analytics](https://ampl.webclat.com/compare/pendo-agent-analytics-vs-amplitude), [Measuring AI-agent usage inside your product](https://ampl.webclat.com/guides/measuring-product-agent-usage), [Amplitude AI agents: what they do and when to use them](https://ampl.webclat.com/guides/amplitude-ai-agents-what-they-do)