LLM Cost Monitoring Without a Proxy (No Latency, No SDK)

The Problem With Proxy-Based LLM Monitoring

When you search for "LLM cost monitoring" or "OpenAI usage tracking," almost every tool you find — Helicone, Portkey, LangSmith, Lunary — works the same way: they act as a proxy between your code and the AI provider. You change your base_url to point at their servers, and they relay your requests while logging token counts and costs.

This approach has real downsides:

Added latency: Every request travels through an extra hop. For a production chatbot already hitting 1–2 second response times, this matters.
Prompt exposure: The proxy sees every prompt and completion. Most LLM applications contain sensitive user data — you're trusting that vendor with it.
Single point of failure: If the proxy goes down or has a bug, your AI features break. This happened to Helicone users during their acquisition period.
Tight coupling: Changing your base URL means touching every API call in your codebase. Migrating away is painful.
Vendor lock-in: Your usage history lives in their database, not yours.

The proxy sees your prompts. Not "might see" — they literally process every request to build usage metrics.

The Alternative: Read the Provider Billing API Directly

Every major AI provider exposes a billing or usage API. OpenAI has /v1/usage. Anthropic has a usage endpoint under their billing API. DeepSeek exposes usage statistics. These endpoints return the exact data you need — token counts, costs, model breakdowns — without you having to intercept a single API call.

Instead of sitting between your code and the AI provider, this approach polls the provider's billing API on a schedule (every hour, for example) and stores the results in your own database. Your production requests go directly to OpenAI, Anthropic, or whoever — zero latency impact, zero prompt exposure.

The trade-off: you get cost data with a slight delay (up to 1 hour), not real-time per-request. For cost monitoring, this is almost always fine. You don't need to know a request cost $0.003 the moment it completes — you need to know when your monthly spend is trending toward $500 when you expected $50.

How to Pull Usage Data From Each Provider

OpenAI

OpenAI's usage API requires an admin key (a separate key type from your regular API keys — create one at platform.openai.com/api-keys under "Service accounts"):

// Fetch daily usage for a date range
const response = await fetch(
  `https://api.openai.com/v1/usage?date=${date}`,
  {
    headers: {
      Authorization: `Bearer ${process.env.OPENAI_ADMIN_KEY}`,
    },
  }
);

const data = await response.json();
// data.data[] contains per-model usage with n_context_tokens_total,
// n_generated_tokens_total, and cost breakdowns

The response includes per-model breakdowns with token counts. To calculate cost, you need the pricing table for each model (which changes periodically — maintain a local copy or use a library that tracks it).

Anthropic

Anthropic's usage API returns spend aggregated by model:

const response = await fetch(
  `https://api.anthropic.com/v1/usage`,
  {
    headers: {
      'x-api-key': process.env.ANTHROPIC_API_KEY,
      'anthropic-version': '2023-06-01',
    },
  }
);

// Returns usage_data[] with model, input_tokens, output_tokens
// Cache tokens have separate pricing (cache_read vs cache_creation)

Important: Anthropic counts cache-read tokens and cache-creation tokens separately, and they have different prices. Make sure your cost calculation accounts for both.

DeepSeek

DeepSeek exposes usage through their dashboard API. The auth is standard bearer token:

const response = await fetch(
  'https://api.deepseek.com/user/balance',
  {
    headers: {
      Authorization: `Bearer ${process.env.DEEPSEEK_API_KEY}`,
    },
  }
);

// Returns balance_infos[] with currency and total_balance
// For detailed usage, use the /v1/usage endpoint

OpenRouter

OpenRouter is particularly useful because it aggregates 500+ models from multiple providers under a single API. Their /api/v1/usage endpoint returns spend broken down by model and date range:

const response = await fetch(
  'https://openrouter.ai/api/v1/usage',
  {
    headers: {
      Authorization: `Bearer ${process.env.OPENROUTER_API_KEY}`,
    },
  }
);

// Returns usage[] with model, tokens, and cost in USD

Building a Polling Architecture

The simplest architecture is a cron job that runs every hour and fetches the last 24 hours of usage from each configured provider. Store results in a time-series table with (provider, model, date, input_tokens, output_tokens, cost_usd) columns.

-- Minimal schema for cost tracking
CREATE TABLE usage_records (
  id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
  provider TEXT NOT NULL,           -- 'openai', 'anthropic', 'deepseek'
  model TEXT NOT NULL,              -- 'gpt-4o', 'claude-3-5-sonnet', etc.
  usage_date DATE NOT NULL,
  input_tokens BIGINT NOT NULL DEFAULT 0,
  output_tokens BIGINT NOT NULL DEFAULT 0,
  cost_usd NUMERIC(10, 6) NOT NULL DEFAULT 0,
  fetched_at TIMESTAMPTZ NOT NULL DEFAULT now(),
  UNIQUE(provider, model, usage_date)
);

CREATE INDEX idx_usage_provider_date
  ON usage_records(provider, usage_date DESC);

The UNIQUE constraint on (provider, model, usage_date) lets you use INSERT ... ON CONFLICT DO UPDATE for idempotent upserts — safe to run the same poll multiple times without duplicating data.

Anomaly Detection Without ML

Once you have a week of historical data, you can detect cost spikes using a simple z-score calculation — no ML required:

// Get 7-day rolling average and std dev for a provider
const stats = await db.query(`
  SELECT
    AVG(cost_usd) as mean,
    STDDEV(cost_usd) as stddev
  FROM usage_records
  WHERE provider = $1
    AND usage_date >= CURRENT_DATE - INTERVAL '7 days'
`, [provider]);

const { mean, stddev } = stats.rows[0];
const zScore = (todayCost - mean) / stddev;

// Alert if today's cost is 2+ standard deviations above normal
if (zScore > 2.0) {
  await sendAlert({ provider, todayCost, mean, zScore });
}

A z-score of 2.0 means today's spend is 2 standard deviations above the 7-day average — statistically unusual enough to warrant a look. Adjust the threshold based on how noisy your workload is. For predictable batch jobs, 1.5 is fine. For variable workloads, use 2.5 to avoid alert fatigue.

The Read-Only API Key Principle

Most providers let you create read-only API keys — keys that can only access billing and usage data, not make API calls. Always use these for your monitoring system:

OpenAI: Create a Service Account key with restricted scopes at platform.openai.com/api-keys
Anthropic: Regular API keys work, but scope them to read-only if available
DeepSeek/OpenRouter: Use separate keys from your production keys

If your monitoring system is compromised, a read-only key can't make API calls and can't generate cost on your behalf. This is especially important if you're building a multi-tenant monitoring service.

Encrypting API Keys at Rest

Don't store provider API keys in plain text, even in a database with row-level security. Use envelope encryption: generate a random AES-256-GCM key per record, encrypt the API key with it, and store the encrypted key alongside the initialization vector and auth tag.

import { randomBytes, createCipheriv } from 'crypto';

function encryptApiKey(plaintext: string): string {
  const key = Buffer.from(process.env.ENCRYPTION_KEY!, 'base64'); // 32-byte key
  const iv = randomBytes(16);
  const cipher = createCipheriv('aes-256-gcm', key, iv);

  const encrypted = Buffer.concat([
    cipher.update(plaintext, 'utf8'),
    cipher.final(),
  ]);
  const authTag = cipher.getAuthTag();

  // Store as: iv:authTag:ciphertext (all base64)
  return [
    iv.toString('base64'),
    authTag.toString('base64'),
    encrypted.toString('base64'),
  ].join(':');
}

The encryption key lives in your environment variables (not the database), so a database dump doesn't expose provider API keys.

What You Can't Do With This Approach

To be fair, billing-API polling has real limitations compared to proxy-based monitoring:

No per-request tracing: You can't correlate a specific API call with a specific cost. You see daily/hourly aggregates per model, not per-call breakdowns.
No prompt/completion logging: If you need to debug which prompts are expensive, you need to instrument this yourself (e.g., log token counts from API responses).
1-hour delay: Some providers update usage data with a delay. Real-time alerts on a per-request basis aren't possible.
No latency tracking: The billing API doesn't tell you how long requests took. You'd need to measure this in your application code.

For most use cases — catching billing spikes, understanding model cost breakdown, budget enforcement, chargeback reporting — these limitations don't matter. If you need per-request tracing for debugging or compliance, a proxy model is genuinely the better fit.

LLMeter: An Open-Source Implementation

We built LLMeter using exactly this architecture. It's open-source (AGPL-3.0) and handles the polling, storage, anomaly detection, and alerting for OpenAI, Anthropic, DeepSeek, OpenRouter, and Mistral.

Setup is about 30 seconds: paste a read-only provider key, and the dashboard populates with your historical cost data immediately. The free tier covers one provider forever with 30-day retention and one budget alert.

The GitHub repo is at github.com/amedinat/LLMeter if you want to see the billing adapter implementations for each provider, the anomaly detection logic, or the Supabase schema.

How to Monitor LLM API Costs Without Routing Traffic Through a Proxy