13 KiB
name, description, depth, links
| name | description | depth | links | ||
|---|---|---|---|---|---|
| Des Doc V0.1 | Design document with full layer diagrams | 1 |
|
harness design document
version: 0.1 status: architectural draft
- overview
Harness is a local-first orchestration runtime for structured LLM execution.
The system is designed around a strict separation of concerns:
layer| responsibility Router| conversational UX Harness| orchestration + policy Agent Runtime| task execution Model Manager| inference lifecycle Context Processor| memory synthesis Event Store| persistence + replay Tool Runtime| external actions
The architecture assumes:
- LLMs are probabilistic
- context is expensive
- memory must be externalized
- workflows require validation
- inference is transient
- orchestration owns reliability
The system intentionally resembles operating system architecture more than chatbot architecture.
- architectural goals
primary goals
- replayable execution
- bounded autonomy
- local-first operation
- model/provider abstraction
- deterministic-enough workflows
- strict validation
- structured artifacts
- observability
- hardware-aware scheduling
secondary goals
- distributed execution
- remote provider failover
- semantic memory
- automatic evaluation
- synthetic training data generation
-
system architecture
┌────────────────┐ │ User │ └───────┬────────┘ │ ▼ ┌────────────────────────┐ │ Router │ │ conversational facade │ └──────────┬─────────────┘ │ ▼
┌──────────────────────────────────────────────────────────┐ │ Harness │ │ │ │ ┌────────────┐ ┌────────────┐ ┌──────────────────┐ │ │ │ Session │ │ Transition │ │ Approval Engine │ │ │ │ Lifecycle │ │ Engine │ │ │ │ │ └─────┬──────┘ └─────┬──────┘ └────────┬─────────┘ │ │ │ │ │ │ │ ▼ ▼ ▼ │ │ ┌───────────────────────────────────────────────────┐ │ │ │ Event Bus │ │ │ └───────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────┐ │ Execution Layer │ │ │ │ ┌──────────────┐ ┌──────────────────────────────┐ │ │ │ StageRuntime │ │ ContextProcessor │ │ │ └──────┬───────┘ └──────────────┬───────────────┘ │ │ │ │ │ │ ▼ ▼ │ │ ┌──────────────────────────────────────────────┐ │ │ │ Agent Runtime │ │ │ └──────────────────────┬───────────────────────┘ │ └─────────────────────────┼────────────────────────────────┘ │ ▼ ┌──────────────────────────────────────────────────────────┐ │ Inference Layer │ │ │ │ ┌───────────────┐ ┌────────────────────────────┐ │ │ │ Model Manager │────▶│ Inference Providers │ │ │ └───────────────┘ │ local / remote / hybrid │ │ │ └────────────────────────────┘ │ └──────────────────────────────────────────────────────────┘ │ ▼ ┌────────────────┐ │ Tool Runtime │ └────────────────┘
- execution lifecycle
step 1 — user interaction
User communicates only with Router.
Router responsibilities:
- intent interpretation
- steering ingestion
- summarization
- conversational continuity
Router never owns workflow state.
step 2 — session initialization
Harness:
- creates session
- initializes projections
- emits SessionCreated event
- resolves workflow entry stage
step 3 — context synthesis
ContextProcessor:
- retrieves relevant events
- compresses history
- filters artifacts
- injects policies
- builds token budget
Output:
ContextPack
step 4 — stage execution
StageRuntime:
- resolves agent role
- resolves required capabilities
- acquires suitable model
- executes inference
step 5 — artifact validation
Pipeline:
- routing validation
- payload validation
- semantic validation
- approval validation
Only validated artifacts emit success transitions.
step 6 — transition evaluation
TransitionEngine:
- evaluates rule graph
- emits transition event
- schedules next stage
step 7 — replay + persistence
Every mutation:
- emits immutable event
- updates projections
- persists session state
- event sourcing architecture
rationale
Event sourcing is mandatory because:
- LLM execution is nondeterministic
- debugging requires replayability
- context reconstruction must be deterministic
- memory requires compression pipelines
event categories
DomainEvents InferenceEvents ToolEvents ApprovalEvents CompressionEvents LifecycleEvents SystemEvents
event flow
UserInputReceived ↓ StageScheduled ↓ ContextBuilt ↓ InferenceStarted ↓ ArtifactProduced ↓ ArtifactValidated ↓ ApprovalRequested ↓ TransitionExecuted
- context processing design
core assumption
Raw conversational accumulation destroys smaller models.
Context must be:
- hierarchical
- compressed
- relevance-ranked
- bounded
context layers
layer| meaning L0| live execution L1| active stage L2| compressed session memory L3| durable project memory L4| archival history
context synthesis pipeline
events ↓ deduplication ↓ relevance ranking ↓ semantic compression ↓ artifact extraction ↓ policy injection ↓ token budgeting ↓ ContextPack
compression rules
tool logs
Raw:
4000 lines shell output
Compressed:
pytest failed:
- auth_test.py
- timeout in token refresh
retention policy
data| strategy artifacts| latest valid tool logs| summarize conversation| semantic transitions| retain approvals| retain
- model management
design goals
- limited hardware optimization
- hot-swapping
- GPU residency control
- provider abstraction
- inference isolation
model manager responsibilities
- model lifecycle
- loading/unloading
- GPU scheduling
- swap timeout enforcement
- health monitoring
- concurrency limits
scheduling strategies
residency modes
mode| behavior persistent| never unload dynamic| unload after timeout ephemeral| unload immediately
capability routing
Stages do not request model names.
Stages request capabilities:
requirements:
- coding
- reasoning
- tool_calling
Registry resolves:
- best local model
- available GPU budget
- fallback providers
- agent runtime
design assumptions
Agents are:
- stateless
- ephemeral
- replaceable
Agents never:
- persist memory
- mutate workflow state directly
- own permissions
agent lifecycle
spawn ↓ receive ContextPack ↓ execute ↓ emit Artifact ↓ terminate
- approval system
rationale
Autonomous systems require bounded risk.
Approval gates prevent:
- destructive execution
- runaway automation
- hidden escalation
approval tiers
tier| meaning T0| inference only T1| read-only T2| reversible mutation T3| external/network T4| destructive
steering-aware approvals
Approvals may inject corrective context.
Example:
approved, but verify migrations against staging schema first
Approval events become part of future context synthesis.
- transition engine
rule-based execution
Transitions are declarative.
No workflow logic is hardcoded.
example
transitions:
-
when: artifact.status == "success" goto: validation
-
when: retries > 3 goto: failed
safeguards
Required:
- cycle detection
- deadlock detection
- transition tracing
- graph visualization
- validation system
layered validation
layer 1 — routing
Checks:
- stage compatibility
- capability availability
- policy alignment
layer 2 — schema
Pydantic validation:
- structure
- typing
- required fields
layer 3 — semantic
Checks:
- hallucinated paths
- invalid references
- unsafe commands
- contradictory artifacts
layer 4 — approval
Checks:
- policy thresholds
- user permissions
- escalation rules
- persistence model
storage backend
Default:
SQLite
Future:
- PostgreSQL
- distributed event stores
persisted entities
entity| purpose events| source of truth projections| fast reads artifacts| outputs approvals| audit transitions| replay summaries| context synthesis
- observability
required telemetry
- token usage
- inference latency
- stage duration
- retries
- approval frequency
- tool failures
- transition graphs
debugging features
- replay from cursor
- event inspection
- context inspection
- transition trace
- artifact lineage
- security model
principles
- least privilege
- explicit escalation
- isolated execution
- auditable actions
recommendations
- sandbox shell tools
- filesystem allowlists
- network policies
- process isolation
- execution timeouts
- secret vault integration
- scalability roadmap
v1
single-node:
- sqlite
- local inference
- sequential execution
v2
multi-provider:
- distributed workers
- remote execution
- shared event store
v3
adaptive orchestration:
- evaluator models
- speculative execution
- automatic routing optimization
- architectural risks
risk| mitigation context entropy| aggressive compression workflow spaghetti| transition tracing infinite retries| bounded retry policies GPU thrashing| residency scheduler hallucinated execution| semantic validators hidden state| event sourcing
- philosophy
Harness treats LLMs as bounded semantic processors embedded inside deterministic orchestration.
Reliability emerges from:
- validation
- event sourcing
- constrained execution
- context synthesis
- approval systems
- replayability
not from trusting model intelligence alone.