kami/Maven
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Reconcile the deployed resident model documentation (V-407)

Browse Source
This commit is contained in:
claude
2026-08-13 01:27:13 +04:00
parent 8035a317d2
commit f957a3ad13
1 changed files with 20 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+20 -16
View File
@@ -135,37 +135,41 @@ Russian recall — you may see many "clarify" responses).
## Qwen3 resident model for router + phraser ## Qwen3 resident model for router + phraser
The target daemon uses the locally trained Qwen3-1.7B checkpoint for both The deployed resident model is stock **Qwen3-1.7B** (`UD-Q4_K_XL`), a Thinking
routing and phrasing. Training is Qwen3 Base → RU CPT → joint persona/router variant at `n_ctx` 4096. `CLAUDE.md` carries the rule on which models qualify.
SFT → merged GGUF, and is still in flight (#122) — until it lands, the deployed
resident model is stock **Qwen3.5-0.8B** (`Q4_K_M`), see `deploy/mavend.json`.
Without a configured model, `StubPhraser` plus the classifier remain the Without a configured model, `StubPhraser` plus the classifier remain the
deterministic floor. deterministic floor.
During training, use the runbook in A locally trained Qwen3-1.7B checkpoint is still in flight (V-122). Training
`docs/plans/2026-07-18-qwen3-resident-training-eval.md`. After the decision gate runs Qwen3 Base, then RU CPT, then joint persona and router SFT, then a merged
and SFT pass, copy the merged GGUF into the mounted model directory and set: GGUF. The
runbook is `docs/plans/2026-07-18-qwen3-resident-training-eval.md`. After the
decision gate and SFT pass, copy the merged GGUF into the mounted model
directory and point `model_path` at it.
**Configure in `deploy/mavend.json`.** This is the deployed `phraser` block:
```json ```json
"phraser": { "phraser": {
"model_path": "/opt/maven/models/llm/Qwen3-Maven-1.7B-Q8_0.gguf", "model_path": "/opt/maven/models/llm/qwen3/Qwen3-1.7B-UD-Q4_K_XL.gguf",
"bin_path": "llama-server", "bin_path": "llama-server",
"n_gpu_layers": 99, "n_gpu_layers": 99,
"n_ctx": 2048 "n_ctx": 4096,
"cache_ram_mib": 512,
"timeout": "60s"
} }
``` ```
**Configure in `deploy/mavend.json`** — the `phraser` block points at this The daemon spawns `llama-server` as a subprocess. The router and replier reach
model and the daemon spawns `llama-server` as a subprocess. The router and that one server through the shared `internal/llm` client. Model files live in
replier use the same llama-server via the shared `internal/llm` client. `/mnt/hdd1/llms`, bind-mounted over `models/llm/`, so a gguf sitting in the repo
is loaded by nothing.
Telegram tokens are read from `deploy/telegram.env` (gitignored), expanded Telegram tokens are read from `deploy/telegram.env` (gitignored), expanded
via `${VAR}` in the JSON config. via `${VAR}` in the JSON config.
**Routing is Qwen-first** with classifier fallback. The LLM router runs The cascade order, and which stage may decline to the next, is in
after stage-0 (exact-match grammar) and before the classifier cascade. On any `docs/routing.md`. It is not restated here.
error or parse failure, the classifier handles the utterance — the turn never
breaks on the model.
## Web UI conventions ## Web UI conventions