Knowledge Workflows Guide¶

Complete guide to the knowledge capture, processing, and storage systems.

Overview¶

The coding infrastructure uses two complementary knowledge management systems:

UKB - Update Knowledge Base¶

Current System: MCP Integration¶

The UKB CLI tool has been replaced with MCP (Model Context Protocol) integration. Type "ukb" in Claude chat to trigger knowledge updates.

Quick Start¶

# In Claude chat (NOT terminal):
User: "ukb"

# Claude executes incremental analysis automatically
# Shows summary of entities created, relations, insights generated

Full Analysis¶

User: "ukb full"

This processes entire git history and all session logs instead of just changes since last checkpoint.

What Happens When You Type "ukb"¶

1. Claude detects knowledge update request
2. Decides: incremental or full analysis
3. Calls MCP semantic-analysis tool
4. Executes 14-agent workflow with SmartOrchestrator
5. QA agent provides semantic routing (proceed/retry/skip/escalate)
6. Stores to GraphDB to LevelDB to JSON export
7. Shows you a summary with confidence metrics

14-Agent Multi-Agent System¶

Smart Orchestrator Flow¶

Orchestration Layer¶

• SmartOrchestrator - Semantic coordination with confidence propagation and intelligent retry
• CoordinatorAgent - Executes workflow definitions, manages agent lifecycle

Data Extraction Agents¶

1. GitHistoryAgent - LLM-powered commit pattern analysis and evolution extraction
2. VibeHistoryAgent - Analyzes session logs with LLM context extraction
3. CodeGraphAgent - AST-based indexing via Memgraph (requires Docker)

Analysis & Enrichment Agents¶

1. SemanticAnalysisAgent - Deep semantic analysis with LLM fallback chain
2. OntologyClassificationAgent - Maps entities to project ontology
3. WebSearchAgent - Researches patterns with optional LLM result ranking

Knowledge Generation Agents¶

1. InsightGenerationAgent - Creates structured insights with PlantUML diagrams
2. ObservationGenerationAgent - Adds observations using LLM structuring
3. DocumentationLinkerAgent - LLM-powered semantic doc-to-code matching

Quality & Persistence Agents¶

1. QualityAssuranceAgent - Semantic Router: validates quality, generates routing decisions (proceed/retry/skip/escalate), provides confidence scoring
2. DeduplicationAgent - Semantic duplicate detection using OpenAI embeddings
3. ContentValidationAgent - Final validation and entity refresh
4. PersistenceAgent - Stores entities to GraphDB

SmartOrchestrator Features¶

• Semantic Retry: Not mechanical threshold tightening - provides specific guidance on what went wrong
• Confidence Propagation: Each step reports confidence; downstream agents are aware of upstream quality
• Routing Decisions: QA generates proceed/retry/skip/escalate based on semantic analysis
• LLM-Assisted Decisions: Uses AI to interpret failures and suggest remediation

Storage Architecture¶

The storage flows through three layers:

1. Graphology (in-memory) - Fast graph operations with 1-second auto-persist
2. LevelDB (persistent) - Durable storage at .data/knowledge-graph/
3. JSON Files (git-tracked) - Team sync via .data/knowledge-export/coding.json

Team Synchronization¶

Developer A workflow:

1. Types "ukb" in Claude
2. Workflow executes
3. Updates: .data/knowledge-export/coding.json
4. Updates: .data/ukb-last-run.json
5. Git commits both files
6. Git pushes to remote

Developer B workflow:

1. Git pulls from remote
2. Gets updated coding.json (knowledge)
3. Gets updated ukb-last-run.json (checkpoint)
4. Types "ukb" in Claude
5. Only processes commits/sessions since checkpoint (avoiding duplicate work)

Continuous Learning System¶

Key Features¶

• Agent-Agnostic Design: Currently optimized for Claude Code; designed to support other AI assistants in future
• Real-Time Extraction: Learns as you code, not after the fact
• Semantic Search: Find relevant knowledge using vector similarity
• Budget-Aware: Tracks LLM costs and enforces a configurable monthly limit
• Privacy-First: Automatically routes sensitive data to local models
• Cross-Session Learning: Share knowledge across different coding sessions

System Components¶

Inference Layer:

• UnifiedInferenceEngine - Central LLM inference with multi-provider support
• BudgetTracker - Cost tracking with configurable monthly limit enforcement
• SensitivityClassifier - 5-layer privacy detection
• CircuitBreaker - Failure detection and provider failover

Knowledge Management:

• StreamingKnowledgeExtractor - Real-time knowledge extraction with buffering
• KnowledgeRetriever - Semantic search with temporal decay
• ConceptAbstractionAgent - Pattern generalization (3+ instances)
• TemporalDecayTracker - Knowledge aging and freshness management

Knowledge Extraction Flow¶

1. Exchange Processing: Developer interacts with coding agent
2. Intent Classification: Classifier categorizes developer intent
3. Budget Check: Budget tracker verifies cost allowance
4. Sensitivity Detection: Classifier routes sensitive data to local models
5. Knowledge Extraction: Buffered exchanges are processed and stored
6. Budget Fallback: Automatic fallback to local models when budget exceeded

Knowledge Retrieval Flow¶

1. Search Request: Developer queries for knowledge patterns
2. Cache Check: System checks for cached results first
3. Embedding Generation: Query converted to vector embedding
4. Vector Search: Qdrant performs HNSW search with filters
5. Temporal Decay: Results adjusted based on knowledge age
6. Ranking & Filtering: Results ranked by relevance and filtered by threshold
7. Cache Storage: Results cached for future queries (5-minute TTL)

Performance: Cache hits return results in ~20ms vs ~300ms for vector search.

Dual-Database Strategy¶

Qdrant Collections¶

SQLite Tables¶

• budget_events - LLM cost tracking
• knowledge_extractions - Extraction metadata
• session_metrics - Session analytics
• embedding_cache - Cached embeddings

Budget Configuration¶

Monthly Limit¶

Configure a monthly USD cap that suits your usage; alerts fire at the configured threshold percentages.

const system = new KnowledgeLearningSystem({
  budgetLimit: 10,  // monthly USD cap (configurable)
  budgetAlerts: [
    { threshold: 50, action: 'log' },
    { threshold: 80, action: 'warn' },
    { threshold: 90, action: 'notify' }
  ],
  budgetAwareRouting: true  // Prefer cheaper providers when budget tight
});

Provider Costs¶

Fallback Chain¶

The system tries providers in order: groq (remote, fast) then openrouter (remote, accurate) then local (free, private)

Temporal Decay¶

Knowledge ages over time with configurable decay rates:

Exceptions (never decay):

• coding_principle
• architecture_pattern

Status Indicators¶

Knowledge System Status¶

Initialization¶

Automatic Initialization¶

Knowledge system is automatically initialized during fresh installations:

# Run the installer (automatically initializes knowledge system)
./install.sh

# Or initialize manually if needed
node scripts/initialize-knowledge-system.js

Verification¶

# Check knowledge-pipeline freshness via the coordinator (Phase 33+:
# .health/*-transcript-monitor-health.json is no longer written; the
# health-coordinator's knowledge_pipeline slice is the source of truth)
curl -fs http://localhost:3034/health/state | jq '.knowledge_pipeline'

# Check status line
CODING_REPO=/path/to/coding node scripts/combined-status-line.js

# Run E2E tests
node scripts/test-knowledge-extraction.js [--verbose]

Visualization¶

Start VKB Server¶

# In terminal:
vkb server start

# Open browser to http://localhost:8080

VKB Features¶

• Interactive graph visualization
• Entity browsing and search
• Relation exploration
• Real-time updates via WebSocket

See VKB Visualization Guide for complete documentation.

Migration Guide¶

What to Stop Doing¶

• ukb --interactive (command doesn't exist)
• ukb --auto (command doesn't exist)
• ukb entity add ... (command doesn't exist)
• Manual JSON file editing

What to Start Doing¶

• Type "ukb" in Claude chat
• Let Claude call MCP semantic-analysis tool
• Trust the 14-agent workflow with SmartOrchestrator
• Review auto-generated insights and confidence metrics
• Commit .data/knowledge-export/*.json and .data/ukb-last-run.json

Troubleshooting¶

Qdrant Connection Errors¶

# Check if Qdrant is running
curl http://localhost:6333/health

# Start Qdrant with Docker
docker run -p 6333:6333 qdrant/qdrant

Budget Exceeded¶

// Check budget status
const budget = await system.getBudgetStatus();
// Review budget.used, budget.remaining, budget.percentage

// Force heuristic mode (no LLM costs)
const system = new KnowledgeLearningSystem({
  forceHeuristic: true
});

Slow Vector Search¶

• Enable quantization for faster searches
• Reduce search limit
• Use 384-dim embeddings instead of 1536-dim

Key Files¶