Token Usage Dashboard¶

Overview¶

The Token Usage page provides real-time visibility into LLM token consumption across every cognitive process in the project. Every call through the LLM Proxy Bridge is logged with provider, model, process attribution, token counts, latency, and a prompt preview. The page makes it possible to spot runaway consumers, see provider mix at a glance, and force a specific service to a specific provider+model when auto-routing picks wrong.

Dashboard URL: http://localhost:3032/token-usage

Page layout¶

The page is organized into three tabs sharing a single header bar:

• Summary cards (always visible): Total Tokens, Total Calls, Avg Latency per LLM call, and a Subscription summary (Claude Max + GitHub Copilot quotas). Every card respects the active time window (see header actions below).
• Tabs: Overview · Evolution · Recent Calls. (The earlier dedicated By Process and Timeline tabs were folded into Overview's Treemap and Evolution's By Tokens (in/out) toggle respectively — see below.)
• Header actions: Time window dropdown (Last 1h · 24h · 48h · 7 days · 30 days · All time) · ⚙ Settings (opens the LLM Routing dialog) · ⟳ Refresh (re-fetches summary + recent; shows a busy spinner while the request is in flight).

Time window selector¶

The window dropdown drives ?hours= on the /api/token-usage/summary request. Every metric on the page — summary cards, treemap, By-Process table, Timeline, and the Evolution stacked chart — re-aggregates for the chosen window. The default is Last 24h so the page matches the historical "trailing 24 h" behavior on first load; switching to All time aggregates the full retained DB without changing any other UI.

Bucket size for the stacked chart adapts to the window so the timeline never balloons past ~500 data points: 2-min buckets for Last 24h, 10-min for 48h, 30-min for 7 days, 2-hour for 30 days, 6-hour for All time. The active bucket size is shown beneath the Evolution chart's title (e.g. "Last 7 days · 30-minute buckets").

Overview tab¶

The header cards plus two side-by-side panels:

• Token Consumption by Process — a treemap where larger rectangles mean more tokens. Top of the page in the screenshot above shows observation-writer (≈ 2.6 M tokens) dominating, with consolidator-digest and consolidator-insight as distant runners-up. Hover any box for a tooltip with process, total tokens, input/output split, call count, and avg latency — including the small boxes that don't fit an inline label. The same payload is also rendered as an SVG <title> element so screen readers and native-browser hover work even when the recharts tooltip is unavailable.
• By Provider — a donut chart split by provider (claude-code 67 % / copilot 33 % under normal conditions on this host).
• By Model — the same totals broken down by canonical model name (claude-sonnet-4.6, claude-haiku-4.5, claude-opus-4.6). The proxy canonicalizes whatever spelling each upstream returns — claude-sonnet-4-6 (Claude CLI dash-version), claude-sonnet-4.6 (Copilot dot-version), Claude Sonnet 4.6 (Anthropic title-case), bare sonnet (CLI fallback when modelUsage is empty), claude-haiku-4-5-20251001 (Copilot dated snapshot) all collapse to the same row. The raw upstream identifier is preserved per call in the model_raw column.

Evolution tab¶

The Evolution tab answers "how does my consumption evolve over time, and who are the main consumers?" It renders a stacked area chart across the selected window with three independently-toggled axes — By Process, By Model, and By Tokens (in/out) — plus a Stacked / Overlapping render mode.

Stack axis toggle¶

The dropdown at top-right of the chart card switches what each band represents. Same window, same data, three lenses:

Design rules¶

• Main-consumer threshold. The chart only renders series contributing ≥ 0.5 % of the window's total tokens. Test/diagnostic processes (reap-test, reap-final, fake-process, export-test, …) routinely satisfy > 0 but contribute fractions of a percent — they're dropped here to keep the legend readable. The full unfiltered breakdown is still available via the summary.process_keys / summary.model_keys API fields for callers that want it.
• Stable colors per process / model. Canonical processes (observation-writer, consolidator-digest, etc.) keep the same color across every chart on the page via PROCESS_COLORS. Unmapped processes (e.g. wave-analysis-wave1/wave2/wave3) fall through to a deterministic hash → SAFE_EVOLUTION_PALETTE mapping — every key gets a stable slot across re-renders, never the all-gray fallback that used to make competing stacks indistinguishable. The same hash logic now drives the Overview Treemap and Recent Calls left-border, so the legend you learn here matches across every panel.
• Adaptive bucket size. Driven by the header time window. The bucket-minutes value is echoed in the card subtitle so the reader can sanity-check the granularity.

Below the chart, Top Consumers is a compact table showing each visible series's total tokens in the window plus its share bar — same data the chart stacks, but in tabular form so you can read exact totals without hovering through the chart.

Recent Calls tab¶

Latest 50 calls across all processes. Columns: Time · Process · Provider · Model · In · Out · Latency · Preview. The Preview column shows the leading characters of the prompt with any XML wrapper tags (<system-prompt>, <task>, <inputs>, etc.) stripped — those wrappers swallowed most of the visible width before the fix and made the column unreadable.

LLM Routing Settings (the ⚙ dialog)¶

The Settings button opens a modal that pins individual services (cognitive processes) to a specific provider + model. The dialog is the dashboard-side of GET / PUT /api/llm/settings exposed by the proxy.

Available providers row at the top reflects the proxy's current /health snapshot — claude-code and copilot typically show online; openai, groq, and anthropic show "(offline)" when their API keys are unset or when the host is on a corporate network where the firewall blocks them.

Service rows list every process value the proxy has ever logged (read from the live token-usage DB). Each row has two dropdowns:

• Provider — any of the five providers, plus "(auto-route)" to clear the pin.
• Model — auto-populated from the chosen provider's model alias list (so picking copilot shows claude-haiku-4-5, claude-sonnet-4-6, claude-opus-4-6).

Hard-pin semantics. A pin is a hard override in routing — it wins over any body.provider the caller passes. If the pinned provider becomes unreachable mid-flight, the proxy still falls through to auto-route (so a Copilot outage doesn't take down observation-writer). Unpinned services use the auto-route logic — Claude Max for Claude Code, Copilot for OpenCode / corporate sessions, falling back to Groq → OpenAI → Anthropic on public networks. Save writes the new pin map to the proxy via PUT /api/llm/settings; the proxy persists it to .data/llm-proxy/settings.json.

Storage and the JSON-export pattern¶

The Token Usage page reads from a two-store setup that mirrors LSL's filesystem convention — git-trackable per-hour JSON files alongside an untracked SQLite WAL DB. Phase 36 moved the export away from a single monolithic JSON to a per-(date, time-window, user-hash) layout so multiple users sharing the project via git push their own hourly snapshots without merge conflicts:

Why both? SQLite WAL files don't merge cleanly across machines; per-hour JSON files do. Teammates share token-usage history via git pull. Filename anatomy: the HHMM-HHMM time-window is the same one LSL uses (sourced from the health coordinator's /health/state.lsl_meta.current_window with a local fallback); the 6-char hex <hash6> is the deterministic per-user identifier exported by the proxy wrapper as LLM_PROXY_USER_HASH before exec node.

Cross-user merge contract. On every proxy boot, hydrateFromExports() walks <baseDir>/**/*.json and inserts every row with INSERT … ON CONFLICT(user_hash, id) DO NOTHING. After git pull brings down a peer's ..._<other-hash>.json file, the next proxy kickstart ingests it and the peer's rows appear in your dashboard alongside yours. Cold-start hydration is always-on (no count > 0 → return early exit) — the composite unique index supplies idempotency, not skipping.

The schema is one token_usage table:

Manual queries¶

# Total tokens today
sqlite3 .data/llm-proxy/token-usage.db \
  "SELECT SUM(input_tokens), SUM(output_tokens) FROM token_usage \
   WHERE timestamp > datetime('now', '-24 hours')"

# Top processes by token usage
sqlite3 .data/llm-proxy/token-usage.db \
  "SELECT process, SUM(total_tokens) AS total FROM token_usage \
   GROUP BY process ORDER BY total DESC LIMIT 10"

# Provider distribution
sqlite3 .data/llm-proxy/token-usage.db \
  "SELECT provider, COUNT(*), SUM(total_tokens) FROM token_usage GROUP BY provider"

Architecture¶

Data flow¶

1. Cognitive processes (observation-writer, consolidator-digest/insight, wave-analysis agents, health-coordinator probes) send completion requests to the LLM Proxy Bridge (:12435).
2. Each request includes a process identifier. If a row in /api/llm/settings has a pin for that process, the proxy honors it; otherwise the auto-route runs.
3. The proxy routes to one of the five providers.
4. After completion, the proxy logs the call to the SQLite DB and schedules the debounced JSON export.
5. The Health Dashboard server (server.js) reverse-proxies /api/token-usage/* and /api/llm/settings to the proxy.
6. The frontend (token-usage.tsx) renders the four tabs from the aggregated data; the Settings dialog renders the LLM Routing dropdowns from the same source.

Key files¶

API endpoints¶

These are served by the proxy bridge; the dashboard's server.js proxies them through :3033.

Summary¶

GET /api/token-usage/summary?hours=24[&bucketMinutes=N]

Aggregates the trailing hours window. hours accepts an integer (e.g. 24, 168, 720) or the literal sentinel all — the latter falls back to the full retained DB and clamps the timeline start to the earliest persisted row so wide windows don't emit thousands of empty buckets. bucketMinutes is optional; when omitted, the bucket size scales with the window (2 / 10 / 30 / 120 / 360 minutes for 24h / 72h / 7d / 30d / All).

Response shape (snake_case throughout, dashboard reads it directly):

{
  "hours": 168,
  "bucket_minutes": 30,
  "total_calls": 2615,
  "total_input": 3920000,
  "total_output": 336000,
  "total_tokens": 4256000,
  "avg_latency_ms": 5830,
  "by_provider":     [{ "provider": "copilot",     "calls": 2250, "input_tokens": …, "output_tokens": …, "total_tokens": … }],
  "by_process":      [{ "process":  "observation-writer", "calls": 1280, "input_tokens": …, "output_tokens": …, "total_tokens": …, "avg_latency": … }],
  "by_model":        [{ "model":    "claude-sonnet-4.6", "calls": …, "total_tokens": … }],
  "by_subscription": [{ "subscription": "claude-max",    "calls": …, "total_tokens": … }],
  "by_hour":         [{ "hour": "2026-05-15T18:00:00.000Z", "input_tokens": 12400, "output_tokens": 860, "calls": 4 }],

  // Stacked series for the Evolution tab — pivoted so recharts can stack
  // columns directly (one row per bucket, one column per process / model).
  // `process_keys` / `model_keys` give the column order ranked by total
  // tokens descending.
  "process_keys":    ["observation-writer", "consolidator-insight", …],
  "model_keys":      ["claude-sonnet-4.6", "claude-haiku-4.5", …],
  "by_process_hour": [{ "hour": "2026-05-15T18:00:00.000Z", "observation-writer": 12400, "consolidator-insight": 0, … }],
  "by_model_hour":   [{ "hour": "2026-05-15T18:00:00.000Z", "claude-sonnet-4.6": 12400, "claude-haiku-4.5": 0, … }]
}

Recent¶

GET /api/token-usage/recent?limit=50

[
  {
    "id": 18342,
    "timestamp": "2026-05-15T19:24:01.412Z",
    "process": "observation-writer",
    "provider": "copilot",
    "model": "claude-sonnet-4.6",
    "input_tokens": 2700,
    "output_tokens": 142,
    "latency_ms": 5100,
    "prompt_preview": "coding # Session Logs (/sl) — Session Continuity Command Load and …"
  }
]

LLM routing settings¶

GET /api/llm/settings

Returns current pins plus reference data so the dialog can populate dropdowns without a second request:

{
  "settings": {
    "observation-writer": { "provider": "copilot", "model": "claude-sonnet-4.6" }
  },
  "processes": ["observation-writer", "consolidator-digest", "health-coordinator", "..."],
  "availableProviders": ["claude-code", "copilot"],
  "allProviders": ["claude-code", "copilot", "openai", "groq", "anthropic"]
}

PUT /api/llm/settings   (Content-Type: application/json)

Replaces the entire pin map atomically; the proxy persists the document to .data/llm-proxy/settings.json.

Cognitive process reference¶

A process of unknown means the caller did not send a process field. These rows are kept (they still consume tokens) but they should be eliminated by patching the caller — unknown is not a useful name in the Settings dialog.

Troubleshooting¶

No data showing¶

1. Verify the proxy bridge is up: curl http://localhost:12435/health | jq
2. Check the DB exists and has rows: sqlite3 .data/llm-proxy/token-usage.db 'SELECT COUNT(*) FROM token_usage'
3. Verify the proxy summary endpoint responds (the frontend hits it directly): curl 'http://localhost:12435/api/token-usage/summary?hours=1' | jq '{total_calls, total_tokens, bucket_minutes}'

Refresh button stays busy¶

Watch the dashboard server logs: docker logs coding-services 2>&1 | grep token-usage. The button uses a fetch promise — if the proxy hangs (long-running CLI subprocess, etc.), the button stays in its busy state until the request times out or the proxy returns.

"Unknown" rows in the table¶

Calls showing process: "unknown" come from callers that haven't been updated to pass a process identifier. The proxy bridge health checks used to show as unknown too — those have been retagged as health-coordinator. If you still see unknown rows after a fresh window, trace the call site with: grep -r '"process"' integrations/ scripts/ observations/ | grep -v '\.test'.

Token counts showing 0¶

Some providers (particularly Copilot) don't always return token counts in the response body. The proxy estimates tokens from prompt/response text length (~4 chars per token) and sets tokens_estimated = 1 so you can tell estimated from authoritative rows in a manual query.

Related documentation¶

• LLM Architecture — provider routing, subscriptions, fallback chains
• Health Monitoring — system health dashboard overview
• LLM CLI Proxy — the consumer-side view of the proxy
• Observational Memory — online learning pipeline