Live Session Logging¶

Where your sessions go¶

.coding/history/YYYY/MM/, one file per time window, named YYYY-MM-DD_HHMM-HHMM-<hash>.jsonl.

There is nothing to run. Logging starts with the session and stops with it.

What it handles for you¶

Reading them back¶

Use /sl in a session to load recent logs for continuity. Files are ordered by filename, not modification time — a checkout or submodule update rewrites mtimes wholesale, and only the date-encoded name is reliable.

When logging looks stalled¶

The [LSL●] badge appears in the status bar only when this pane's logging is unhealthy — no badge is good news. If it does appear, check the per-project log under .logs/.

The classification problem¶

Work rarely stays in one repository. You fix something in a project while the fix really belongs to the infrastructure, or vice versa, and a naive logger files everything under wherever the shell happened to be.

Five layers decide, cheapest first: a session filter carrying conversation context, a path analyser matching file-operation patterns, and progressively more expensive checks after that. Most content is routed by the first layers in under a millisecond; only the genuinely ambiguous reaches anything costly.

That ordering is the design. Classification runs on every exchange, so it has to be nearly free in the common case.

Redaction happens before the write¶

Secrets are stripped on the way in, not cleaned up afterwards. A log that is sanitised later has already existed unsanitised on disk.

The trade-off is false positives — a redactor tuned to catch credentials will sometimes catch something that merely looks like one. That is the correct direction to err, but it means a redacted-looking value in a log is not proof that a real secret was there.

Supervision¶

Capture is monitored in layers, on the assumption that the layer below might have stopped without saying so. A logger that dies loudly is easy; one that keeps running while quietly writing nothing is the case the supervision exists for, which is why liveness is judged by whether output is still being produced rather than by whether the process exists.

The files¶

One file per time window under .coding/history/YYYY/MM/, named with its date, its window and a hash. Suffixed siblings (…-1_<hash>) are continuations of the same window.

Order by filename, never by mtime. A git checkout, a clone or a submodule update rewrites modification times across the tree, which would surface months-old files as the most recent. The name carries the truth.

Across agents¶

All four supported agents are captured. Agents with their own session transcripts are read directly; those that only print to a terminal are captured through the tmux pipe and turned back into discrete prompts by a per-agent pattern — which is what AGENT_PROMPT_REGEX in an agent's config file is for.

Reading logs back¶

/sl loads recent logs for continuity at the start of a session. Beyond that the files are plain JSONL: one entry per line, with the session header, prompt-set boundaries and each message as its own record.

Real-time conversation monitoring and intelligent classification with zero data loss.

What It Does¶

• Real-Time Monitoring - Captures every coding agent conversation as it happens (Claude Code, Copilot CLI, OpenCode)
• Intelligent Classification - 5-layer system routes content (LOCAL vs CODING)
• Zero Data Loss - 4-layer monitoring architecture ensures reliability
• Multi-Project Support - Handles multiple projects with foreign session tracking
• Security Redaction - Automatic sanitization of secrets and credentials

5-Layer Classification¶

Early exit optimization: Classification stops at first confident decision.

Supervision & Recovery¶

The transcript monitor is managed by a priority-ordered supervisor chain:

Self-protection features:

• Periodic flush: Writes accumulated exchanges every 5 minutes during long agent runs
• Idle timeout with tmux guard: Stays alive while tmux session exists (prevents restart budget waste)
• Auto-recovery: Prompt hook detects LSL down and spawns coordinator (rate-limited 1/min)

Content Routing¶

LOCAL Content (Project-Specific):

• Stored in: project/.coding/history/YYYY/MM/
• Format: YYYY-MM-DD_HHMM-HHMM_<userhash>.jsonl

CODING Content (Infrastructure):

• Redirected to: coding/.coding/history/YYYY/MM/
• Format: YYYY-MM-DD_HHMM-HHMM_<userhash>_from-<project>.jsonl

Rotation appends a part index before the hash (…-1_<userhash>.jsonl). A part is a continuation, not a standalone file — see Session format.

Session format¶

Tranches are written as pi session JSONL — one JSON entry per line, the format the pi agent reads natively. Only the serialization changed: hourly files, the filename structure, the 5-layer classification, rotation and redirect handling all behave exactly as before.

Every prompt set parents off the tranche spine rather than the previous set, so removing one drops its whole subtree and nothing else — which is what lets the monitor re-flush a set in place as a turn grows.

A lsl.promptSet carrying synthesized: true had its boundaries inferred rather than read: the oldest layout has no prompt-set anchor, so consecutive blocks were grouped by their repeated **User Request:**. The flag marks structure the markdown never actually recorded.

Converting the legacy corpus¶

scripts/backfill-lsl-to-pi.mjs converts existing markdown. The unit of work is a chain — an hourly tranche plus its rotation parts — because rotation splits mid-token: a part can begin inside a JSON fence opened by its predecessor, so it cannot be parsed alone.

# Dry run is the DEFAULT — writing requires an explicit --write
node scripts/backfill-lsl-to-pi.mjs --all-history-repos
node scripts/backfill-lsl-to-pi.mjs --all-history-repos --write --commit

Safety properties, in the order they matter:

• Each history repo is tagged pre-pi-format before anything is touched, so git reset --hard pre-pi-format reverts a repo completely.
• Markdown is deleted only when git can restore it — a file that is untracked or locally modified is kept, whatever else happens.
• Every chain is verified before its markdown is removed. A chain that fails, or that converts to nothing, is quarantined with its markdown intact and a reason written to .coding/quarantine/.
• Not every .md yields a .jsonl. A part in which no block starts belongs to its predecessor's blocks; those are recorded in chain-map.json as absorbedInto, so a missing output file is provably accounted for rather than merely absent.

Stop the ETM's committer for the duration of a bulk run. It commits the whole history directory periodically, and will otherwise sweep the conversion into its own commits — harmless, but it costs the per-month commit granularity that makes a bad batch revert cleanly.

Security Redaction¶

13 pattern types automatically sanitized:

• API keys and tokens
• Passwords and credentials
• URLs with embedded passwords
• Email addresses
• Corporate user IDs

Performance: <5ms overhead per exchange.

Redaction runs on every exchange, on the monitor's main thread, and rewrites content before it reaches disk. Two consequences worth knowing:

• A pathological pattern stops the monitor, without killing it. A regex whose leading character class contains the literal separating it from a following class of the same alphabet backtracks catastrophically. aws_secret_truncated did exactly this: 200 KB redacted in 239 ms, 400 KB never finished. The process stayed alive at ~100% CPU, so ps and the service health checks looked fine — only the heartbeat stopped, and the coordinator marked LSL down after 15 s. See Troubleshooting.
• A pattern that is too broad destroys data irreversibly. Redaction happens at write time, so the original never reaches the file and no reader can recover it. aws_secret_standalone matched any 40-character token, which is exactly the shape of a git SHA — commit ids across the corpus read <AWS_SECRET_REDACTED>.

Both are fixed. When adding a pattern, bound its quantifiers and check it against a large input; tests/live-logging/ConfigurableRedactor.redos.test.js guards this with a time bound rather than a pattern assertion, so it keeps covering patterns added later. Note the patterns live in two places — the JSON config and a fallback table inside ConfigurableRedactor.js — and fixing one leaves the other live.

Configuration¶

File: config/live-logging-config.json

{
  "session_filter": {
    "enabled": true,
    "bias_threshold": 0.65,
    "window_size": 5
  },
  "embedding_classifier": {
    "enabled": true,
    "similarity_threshold": 0.65,
    "model": "Xenova/all-MiniLM-L6-v2"
  },
  "semantic_analyzer": {
    "enabled": true,
    "provider": "groq",
    "model": "llama-3.3-70b"
  }
}

Transcript Sources¶

The monitor supports three transcript sources, auto-detected per project:

All sources are normalized to a common exchange format before processing.

Key Files¶

Troubleshooting¶

Monitor not starting¶

# Check if running
ps aux | grep enhanced-transcript-monitor

# Check ETM heartbeat via coordinator (Phase 33+: .health/*.json files are
# no longer written; coordinator's lsl slice is the source of truth)
curl -fs http://localhost:3034/health/state \
  | jq '.lsl | to_entries | map(select(.key | endswith(":coding")))'

# Restart via coding command
coding --restart-monitor

LSL red while the monitor is running¶

The statusline shows [LSL🔴] and the project letter turns yellow, but the process is alive and the Health API is green. The monitor is wedged, not crashed — most often in a redaction regex, since that runs on the main thread for every exchange.

The distinguishing signal is CPU plus a stale heartbeat:

# Wedged looks like ~100% CPU in state R, with CPU time tracking elapsed time
/bin/ps -eo pid,%cpu,state,etime,time | grep enhanced-transcript-monitor

# The coordinator infers 'stopped' after >15s of heartbeat silence, so a wedged
# monitor reports stopped while ps still shows it running
curl -s localhost:3034/health/state | python3 -c \
  "import json,sys; print(json.load(sys.stdin)['lsl_by_project'])"

# Settle it: look for ArrayForEach -> StringPrototypeReplace -> RegExpReplace
sample <pid> 3

Recovery is kill -9 on the wedged PID followed by launchctl kickstart -k gui/$(id -u)/com.coding.etm; the monitor replays the backlog it was stuck on. If it re-wedges on the same session, the trigger is content-dependent (a long base64-like run) — fix the pattern rather than restarting again.

Contrast with a crash loop, where the process is absent and launchctl list shows a non-zero exit — most commonly a missing node_modules/@fwornle/km-core symlink.

LSL files not generated¶

# Verify monitor is processing
tail -50 .logs/transcript-monitor-test.log

# Check today's files
ls -la .coding/history/$(date +%Y/%m)/ | grep "$(date +%Y-%m-%d)"

# Recover from transcripts
PROJECT_PATH=/path/to/project CODING_REPO=/path/to/coding \
  node scripts/batch-lsl-processor.js from-transcripts ~/.claude/projects/-path-to-project

Classification issues¶

# Check classification logs
ls -la .coding/history/logs/classification/

# Verify config
cat config/live-logging-config.json | jq '.embedding_classifier'