mavgpud: turn gemma's thinking off in the chat template (V-489)
Owner's call, 02-08-2026. Without it the 12B spends the reply budget on reasoning tokens and answers empty at low max_tokens. Verified on the box: "Столица Франции?" now answers "Париж" with no reasoning_content.
This commit is contained in:
@@ -15,6 +15,7 @@
|
|||||||
"--batch-size", "2048",
|
"--batch-size", "2048",
|
||||||
"--ubatch-size", "512",
|
"--ubatch-size", "512",
|
||||||
"--jinja",
|
"--jinja",
|
||||||
|
"--chat-template-kwargs", "{\"enable_thinking\":false}",
|
||||||
"--spec-type", "draft-mtp",
|
"--spec-type", "draft-mtp",
|
||||||
"--spec-draft-n-max", "2"
|
"--spec-draft-n-max", "2"
|
||||||
],
|
],
|
||||||
|
|||||||
Reference in New Issue
Block a user