Measure AI Cost per Conversation, Not per Call

§1 Step 1: Define the Conversation You Will Measure

A cheap API call is not necessarily a cheap customer interaction. Consider a hypothetical assistant that makes six requests to resolve one question. Measuring each request separately hides the total expense of delivering that outcome.

Start by defining a conversation boundary. For a support assistant, use a ticket or an explicitly closed session. For a coding assistant, use a task identifier. If you choose an inactivity timeout, document it and apply it consistently. Avoid comparing products that use different boundaries.

Your primary metric should be total attributed cost divided by the number of conversations in the same reporting cohort. Keep cost per call as a diagnostic metric, not the headline.

§2 Step 2: Attach Every Attempt to a Conversation

Create a conversation_id before the first model request and propagate it through your backend. Give each request attempt its own identifier, including retries and fallback attempts. Keep these records even when the user sees no answer.

• – Record conversation_id, request_id, attempt_id, and the selected model.
• – Capture timestamps, completion status, and the reason for any retry.
• – Store reported token usage, applicable prices, and calculated cost.
• – Associate separately billed tools and retrieval operations with the same conversation.

Make ingestion idempotent: receiving the same usage event twice must not double the recorded expense. Keep your ledger focused on identifiers and usage metadata rather than storing conversation text unnecessarily.

§3 Step 3: Calculate Costs from Billable Token Categories

Do not multiply every token by a single blended rate. Input, cached input, and output tokens can have different prices. Preserve those distinctions when the API exposes them. ([help.openai.com](https://help.openai.com/en/articles/4936856-understanding-and-counting-tokens?utm_source=openai))

For a simple pricing schedule, calculate request cost as uncached input tokens multiplied by their rate, plus cached input tokens multiplied by their rate, plus output tokens multiplied by their rate. Divide by one million when prices are quoted per million tokens. Add other applicable charges separately. This calculation follows the distinct pricing categories rather than assuming one universal token price. ([help.openai.com](https://help.openai.com/en/articles/4936856-understanding-and-counting-tokens?utm_source=openai))

Check category definitions before adding counts together. For example, OpenAI documents usage fields where cached tokens are already included in input tokens and reasoning tokens are included in output tokens. Adding those subsets again would double-count them. ([developers.openai.com](https://developers.openai.com/api/docs/guides/agents-api/observability?utm_source=openai))

§4 Step 4: Capture Usage After Streaming Completes

For OpenAI Chat Completions streaming, request usage with stream_options.include_usage set to true. The final usage-bearing chunk provides accounting information for the request; do not estimate the bill by counting visible text fragments. Verify equivalent behavior separately for any relay or alternative backend. ([help.openai.com](https://help.openai.com/en/articles/10478918-reviewing-api-usage-and-costs?utm_source=openai))

If a stream is interrupted or canceled, that final usage chunk may never arrive. Mark the attempt as usage pending rather than assigning zero cost. Reconcile it against billing records when available, and keep unresolved estimates visibly separate from confirmed charges. ([developers.openai.com](https://developers.openai.com/api/reference/resources/chat/subresources/completions/streaming-events?utm_source=openai))

§5 Step 5: Aggregate the Full Conversation Ledger

Sum all attributed request attempts and separately billed services within each conversation. Then calculate average conversation cost across a defined cohort. Include abandoned and failed conversations in an all-started-conversations view; report successful conversations separately.

For illustration, suppose 100 conversations generate $12 in model charges and $3 in tool charges. The average is $0.15 per conversation. If 75 reach the defined success condition, total cohort spend per successful outcome is $0.20. These are hypothetical amounts, not provider prices.

§6 Step 6: Compare Routing Strategies by Outcome

Test routing changes against the same task set. You could compare Gemini 2.5 Flash with Claude Haiku 4.5 without assuming either is cheaper overall. Measure total conversation spend, success rate, follow-up turns, and latency together.

Fluxio fits this workflow when you want a streaming-first, pay-per-token relay with one key across models. Use its shared integration surface to simplify routing experiments, while retaining your application-owned conversation ledger. Do not assume a common interface makes usage fields or billing categories identical.

§7 Step 7: Build a Conversation Cost Review

Track average, median, and ninety-fifth-percentile conversation cost alongside success rate and reconciliation coverage. Segment by task type and routing strategy. Investigate expensive conversations before imposing blanket token limits, and accept optimizations only when the resulting outcomes remain acceptable.

Sign up for Fluxio.