How to Read Usage Logs and Find Expensive Requests

Start an AI cost investigation with one question: which requests deserve attention first? This walkthrough uses a hypothetical usage log to move from individual charges to actionable fixes. Keep two views throughout: the cost of each API attempt and the total cost of completing a user task.

§1 Step 1: Define the Usage Log Window

Select one complete day, use a consistent time zone, and separate production traffic from development tests. Export whatever request-level records your service provides, or inspect your application logs. Avoid assuming that any particular dashboard includes every field you need.

Build a working dataset containing these fields where available:

• – Request timestamp, request identifier, and application trace identifier.
• – Requested model, returned model identifier, and application feature.
• – Input tokens, output tokens, and separately reported cache categories.
• – Charged amount, currency, response status, and retry attempt number.
• – Total duration, time to first token, and stream completion status.

Preserve provider request identifiers alongside your own trace identifiers. OpenAI exposes an x-request-id header and recommends logging request IDs for production troubleshooting. Other services may use different identifiers. ([platform.openai.com](https://platform.openai.com/docs/api-reference/debugging-requests))

§2 Step 2: Rank Requests by Charged Amount

Sort the charged amount numerically from highest to lowest. Then calculate each row's share of the selected period's total spend. Keep estimated costs in a separate column so they cannot be mistaken for settled charges.

Illustrative example: suppose 1,000 requests cost $20 altogether, while the ten most expensive requests account for $8. Those ten requests represent 1% of request volume but 40% of spending. Investigate that group before optimizing the remaining 990 requests.

Next, group by feature and sum the charges. A feature with modest individual requests may still deserve priority because of its frequency. Retain both rankings: expensive individual attempts and expensive aggregate workloads.

§3 Step 3: Separate Token Categories Before Calculating Cost

For each expensive row, inspect the token breakdown rather than relying on total tokens alone. Published API pricing can distinguish input, output, cached input, and cache-write categories, so equal token totals do not necessarily imply equal costs. ([platform.openai.com](https://platform.openai.com/docs/pricing))

Where a request has only uncached text input and text output, estimate its token cost as input tokens multiplied by the applicable input rate, plus output tokens multiplied by the applicable output rate. Divide by one million when rates are quoted per million tokens. Add any other applicable charges separately. ([platform.openai.com](https://platform.openai.com/docs/pricing))

For a purely hypothetical calculation, 40,000 input tokens at $2 per million cost $0.08, while 4,000 output tokens at $10 per million cost $0.04. The combined estimate is $0.12. These are teaching rates, not Fluxio prices. Verify whether cached tokens are included within the reported input count before subtracting or adding categories.

§4 Step 4: Check Streaming Usage Before Marking Requests Complete

Inspect how your client records final usage. For OpenAI Chat Completions, setting stream_options.include_usage requests an additional usage-bearing chunk before the stream finishes. Confirm equivalent behavior with your relay rather than assuming compatibility covers every detail. ([platform.openai.com](https://platform.openai.com/docs/api-reference/chat/create))

Also distinguish missing usage from zero usage. OpenAI documents that an interrupted stream may prevent receipt of the final usage chunk. Mark such records as incomplete and reconcile them against available billing records instead of assigning a zero cost. ([platform.openai.com](https://platform.openai.com/docs/api-reference/chat/create))

§5 Step 5: Trace Expensive Attempts Back to User Tasks

Open the trace for each high-cost request. Look for repeated conversation history, oversized retrieved documents, unnecessarily long answers, and repeated tool results. Group retry attempts under the same user task without deleting their individual charges.

In another hypothetical example, a successful task with three charged attempts of $0.12 each costs $0.36, not $0.12. Deduplicate repeated log entries for the same attempt, but preserve genuinely separate attempts.

§6 Step 6: Change One Driver and Compare Results

Choose one intervention: shorten retrieved context, request a more concise answer, or test another model against the same evaluation set. Compare cost per successful task, answer quality, and latency over equivalent workloads.

Fluxio is worth considering when you want a streaming-first, pay-per-token relay for testing multiple models through one API key. Keep application-side logging consistent across those tests, and verify actual charges rather than assuming a model switch saves money.

Sign up for Fluxio.