--- name: "Des Doc V0.1" description: "Design document with full layer diagrams" depth: 1 links: ["../index.md", "./spec-v0.1.md"] --- harness design document version: 0.1 status: architectural draft 1. overview Harness is a local-first orchestration runtime for structured LLM execution. The system is designed around a strict separation of concerns: layer| responsibility Router| conversational UX Harness| orchestration + policy Agent Runtime| task execution Model Manager| inference lifecycle Context Processor| memory synthesis Event Store| persistence + replay Tool Runtime| external actions The architecture assumes: - LLMs are probabilistic - context is expensive - memory must be externalized - workflows require validation - inference is transient - orchestration owns reliability The system intentionally resembles operating system architecture more than chatbot architecture. --- 2. architectural goals primary goals - replayable execution - bounded autonomy - local-first operation - model/provider abstraction - deterministic-enough workflows - strict validation - structured artifacts - observability - hardware-aware scheduling --- secondary goals - distributed execution - remote provider failover - semantic memory - automatic evaluation - synthetic training data generation --- 3. system architecture ┌────────────────┐ │ User │ └───────┬────────┘ │ ▼ ┌────────────────────────┐ │ Router │ │ conversational facade │ └──────────┬─────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────┐ │ Harness │ │ │ │ ┌────────────┐ ┌────────────┐ ┌──────────────────┐ │ │ │ Session │ │ Transition │ │ Approval Engine │ │ │ │ Lifecycle │ │ Engine │ │ │ │ │ └─────┬──────┘ └─────┬──────┘ └────────┬─────────┘ │ │ │ │ │ │ │ ▼ ▼ ▼ │ │ ┌───────────────────────────────────────────────────┐ │ │ │ Event Bus │ │ │ └───────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────┐ │ Execution Layer │ │ │ │ ┌──────────────┐ ┌──────────────────────────────┐ │ │ │ StageRuntime │ │ ContextProcessor │ │ │ └──────┬───────┘ └──────────────┬───────────────┘ │ │ │ │ │ │ ▼ ▼ │ │ ┌──────────────────────────────────────────────┐ │ │ │ Agent Runtime │ │ │ └──────────────────────┬───────────────────────┘ │ └─────────────────────────┼────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────┐ │ Inference Layer │ │ │ │ ┌───────────────┐ ┌────────────────────────────┐ │ │ │ Model Manager │────▶│ Inference Providers │ │ │ └───────────────┘ │ local / remote / hybrid │ │ │ └────────────────────────────┘ │ └──────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────┐ │ Tool Runtime │ └────────────────┘ --- 4. execution lifecycle step 1 — user interaction User communicates only with Router. Router responsibilities: - intent interpretation - steering ingestion - summarization - conversational continuity Router never owns workflow state. --- step 2 — session initialization Harness: - creates session - initializes projections - emits SessionCreated event - resolves workflow entry stage --- step 3 — context synthesis ContextProcessor: - retrieves relevant events - compresses history - filters artifacts - injects policies - builds token budget Output: ContextPack --- step 4 — stage execution StageRuntime: - resolves agent role - resolves required capabilities - acquires suitable model - executes inference --- step 5 — artifact validation Pipeline: 1. routing validation 2. payload validation 3. semantic validation 4. approval validation Only validated artifacts emit success transitions. --- step 6 — transition evaluation TransitionEngine: - evaluates rule graph - emits transition event - schedules next stage --- step 7 — replay + persistence Every mutation: - emits immutable event - updates projections - persists session state --- 5. event sourcing architecture rationale Event sourcing is mandatory because: - LLM execution is nondeterministic - debugging requires replayability - context reconstruction must be deterministic - memory requires compression pipelines --- event categories DomainEvents InferenceEvents ToolEvents ApprovalEvents CompressionEvents LifecycleEvents SystemEvents --- event flow UserInputReceived ↓ StageScheduled ↓ ContextBuilt ↓ InferenceStarted ↓ ArtifactProduced ↓ ArtifactValidated ↓ ApprovalRequested ↓ TransitionExecuted --- 6. context processing design core assumption Raw conversational accumulation destroys smaller models. Context must be: - hierarchical - compressed - relevance-ranked - bounded --- context layers layer| meaning L0| live execution L1| active stage L2| compressed session memory L3| durable project memory L4| archival history --- context synthesis pipeline events ↓ deduplication ↓ relevance ranking ↓ semantic compression ↓ artifact extraction ↓ policy injection ↓ token budgeting ↓ ContextPack --- compression rules tool logs Raw: 4000 lines shell output Compressed: pytest failed: - auth_test.py - timeout in token refresh --- retention policy data| strategy artifacts| latest valid tool logs| summarize conversation| semantic transitions| retain approvals| retain --- 7. model management design goals - limited hardware optimization - hot-swapping - GPU residency control - provider abstraction - inference isolation --- model manager responsibilities - model lifecycle - loading/unloading - GPU scheduling - swap timeout enforcement - health monitoring - concurrency limits --- scheduling strategies residency modes mode| behavior persistent| never unload dynamic| unload after timeout ephemeral| unload immediately --- capability routing Stages do not request model names. Stages request capabilities: requirements: - coding - reasoning - tool_calling Registry resolves: - best local model - available GPU budget - fallback providers --- 8. agent runtime design assumptions Agents are: - stateless - ephemeral - replaceable Agents never: - persist memory - mutate workflow state directly - own permissions --- agent lifecycle spawn ↓ receive ContextPack ↓ execute ↓ emit Artifact ↓ terminate --- 9. approval system rationale Autonomous systems require bounded risk. Approval gates prevent: - destructive execution - runaway automation - hidden escalation --- approval tiers tier| meaning T0| inference only T1| read-only T2| reversible mutation T3| external/network T4| destructive --- steering-aware approvals Approvals may inject corrective context. Example: approved, but verify migrations against staging schema first Approval events become part of future context synthesis. --- 10. transition engine rule-based execution Transitions are declarative. No workflow logic is hardcoded. --- example transitions: - when: artifact.status == "success" goto: validation - when: retries > 3 goto: failed --- safeguards Required: - cycle detection - deadlock detection - transition tracing - graph visualization --- 11. validation system layered validation layer 1 — routing Checks: - stage compatibility - capability availability - policy alignment --- layer 2 — schema Pydantic validation: - structure - typing - required fields --- layer 3 — semantic Checks: - hallucinated paths - invalid references - unsafe commands - contradictory artifacts --- layer 4 — approval Checks: - policy thresholds - user permissions - escalation rules --- 12. persistence model storage backend Default: SQLite Future: - PostgreSQL - distributed event stores --- persisted entities entity| purpose events| source of truth projections| fast reads artifacts| outputs approvals| audit transitions| replay summaries| context synthesis --- 13. observability required telemetry - token usage - inference latency - stage duration - retries - approval frequency - tool failures - transition graphs --- debugging features - replay from cursor - event inspection - context inspection - transition trace - artifact lineage --- 14. security model principles - least privilege - explicit escalation - isolated execution - auditable actions --- recommendations - sandbox shell tools - filesystem allowlists - network policies - process isolation - execution timeouts - secret vault integration --- 15. scalability roadmap v1 single-node: - sqlite - local inference - sequential execution --- v2 multi-provider: - distributed workers - remote execution - shared event store --- v3 adaptive orchestration: - evaluator models - speculative execution - automatic routing optimization --- 16. architectural risks risk| mitigation context entropy| aggressive compression workflow spaghetti| transition tracing infinite retries| bounded retry policies GPU thrashing| residency scheduler hallucinated execution| semantic validators hidden state| event sourcing --- 17. philosophy Harness treats LLMs as bounded semantic processors embedded inside deterministic orchestration. Reliability emerges from: - validation - event sourcing - constrained execution - context synthesis - approval systems - replayability not from trusting model intelligence alone.