feat(qa): remote NIM provider + headless-QA robustness
Enable autonomous QA through a remote OpenAI-compatible provider (NVIDIA NIM) and harden the tool/approval path so unattended multi-stage runs complete. - inference: add openai_compat provider (Bearer chat-completions for NIM/OpenAI), dispatched by provider type "nim"/"openai"; key via api_key/api_key_env. - server: bind configured [server] host/port instead of a hardcoded 8080; POST /sessions accepts an optional `intent` (WS parity) for intent-driven workflows. - kernel: thread the bound operator profile's approval_mode into per-tool gating so auto/yolo enable unattended approval (engine still consulted; policy/plane-2 BLOCK stays terminal); on a recoverable tool failure feed the tool's arg-schema back into context so the model self-corrects instead of repeating a malformed call. - tools: split deletion out of file_write into a separate, explicitly-named file_delete tool — a model can no longer delete a file by getting a write-mode parameter wrong. - server: add GET /metrics/tool-reliability — per-model tool-call validity from the event log (measurement groundwork for capability-aware routing). - docs: update AGENTS.md across kernel, tools, server, inference. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -2,7 +2,7 @@
|
||||
|
||||
## Purpose
|
||||
|
||||
Inference adapter layer. The root module provides `DefaultProviderRegistry` (registers `InferenceProvider` instances) and `FirstAvailableRoutingStrategy`. Submodules cover shared HTTP client infrastructure (`commons/`) and the llama.cpp server adapter (`llama_cpp/`).
|
||||
Inference adapter layer. The root module provides `DefaultProviderRegistry` (registers `InferenceProvider` instances) and `FirstAvailableRoutingStrategy`. Submodules cover shared HTTP client infrastructure (`commons/`), the llama.cpp server adapter (`llama_cpp/`), and the remote OpenAI-compatible adapter (`openai_compat/`, e.g. NVIDIA NIM).
|
||||
|
||||
## Ownership
|
||||
|
||||
@@ -13,8 +13,9 @@ Adapter for LLM inference backends. Implements `core:inference` interfaces. No d
|
||||
- `DefaultProviderRegistry` implements `ProviderRegistry` from `core:inference`.
|
||||
- `FirstAvailableRoutingStrategy` is the default routing policy; extend only in `core:inference`, not here.
|
||||
- All LLM responses are proposals — they must be validated by the core before affecting state (invariant #7). Adapters return raw responses; they do not validate.
|
||||
- Network calls to the llama.cpp server are environment observations; results must be recorded as events by callers if replay must reproduce them (invariant #9).
|
||||
- Network calls to inference backends are environment observations; results must be recorded as events by callers if replay must reproduce them (invariant #9).
|
||||
- Model lifecycle (spawn/own the llama-server process) lives in `llama_cpp/`; the autonomous scheduler is intentionally unbuilt (see root CLAUDE.md).
|
||||
- `openai_compat/` is fully remote (no local process/GPU). It speaks `POST {baseUrl}/chat/completions` with `Authorization: Bearer`; `baseUrl` must include the version segment (e.g. `/v1`). It has no `/tokenize`, so it uses a heuristic tokenizer, and no GBNF — JSON artifacts rely on the core's validate-after-retry. Server dispatch keys `[[providers]] type = "nim" | "openai"`; the key comes from `api_key` or `api_key_env`.
|
||||
|
||||
## Work Guidance
|
||||
|
||||
@@ -26,9 +27,11 @@ Standard adapter rules apply (see parent `AGENTS.md`). HTTP client code uses Kto
|
||||
./gradlew :infrastructure:inference:test --rerun-tasks
|
||||
./gradlew :infrastructure:inference:commons:test --rerun-tasks
|
||||
./gradlew :infrastructure:inference:llama_cpp:test --rerun-tasks
|
||||
./gradlew :infrastructure:inference:openai_compat:test --rerun-tasks
|
||||
```
|
||||
|
||||
## Child DOX Index
|
||||
|
||||
- `commons/` — shared `ManagedInferenceProvider`, `ModelManager`, `ResourceProbe` (Nvidia/AMD), `ResidencyMode`; no separate AGENTS.md (sub-leaf, covered by this doc)
|
||||
- `llama_cpp/` — `LlamaCppInferenceProvider`, `LlamaProcess` (spawns/owns llama-server), `LlamaCppEmbedder`, `LlamaCppTokenizer`, `GbnfGrammarConverter`; no separate AGENTS.md (sub-leaf, covered by this doc)
|
||||
- `openai_compat/` — `OpenAiCompatInferenceProvider` (remote Bearer-auth chat completions for NVIDIA NIM/OpenAI), `HeuristicTokenizer`, `OpenAiApiModels`; no separate AGENTS.md (sub-leaf, covered by this doc)
|
||||
|
||||
Reference in New Issue
Block a user