Talk MCP: a client for external tool servers (#251)

docs/plans/06-mcp-support.md asks for the host direction — Maven connects OUT
to MCP servers and consumes what they offer. This is the client half: the
protocol, the transports, the connection manager, the config block. Nothing is
wired into a turn yet, and nothing here exposes Maven's own capabilities to an
outside caller.

internal/mcp:
  - hand-rolled JSON-RPC 2.0 (the wire format is four fields, and the repo
    vendors its deps, so a library would cost more than it saves);
  - two transports: a stdio subprocess on this box, and streamable HTTP, which
    accepts a plain JSON reply or an SSE frame because servers disagree about
    which they send;
  - Client: initialize handshake, tools/list, tools/call, resources/list,
    resources/read. Text content only — everything downstream is a sentence;
  - Manager: lazy dial, per-server failure that never blocks boot or the other
    servers, backoff reconnect, Status for a web surface, graceful Close;
  - the allowlist encoding: a discovered tool becomes the store row
    "vikunja_list_tasks" with cmd ["mcp","vikunja","list_tasks"], scope
    "mcp:vikunja". No new column, no migration, and ProposeTool, EnableTool,
    the act matcher and the confirm turn all keep working untouched.

Constraints held, in code rather than in prose:
  - OFF unless configured, and a server is dark until "enabled": true.
  - A url server goes through internal/webfetch, so the SSRF guard, the size
    cap, the redirect cap and the per-host rate limit apply. Reaching loopback
    needs allow_private on THAT server, and each server gets its own fetcher so
    one loopback exemption cannot become a hole for a public endpoint.
  - readOnlyHint decides destructive: no hint means "assume it mutates", which
    will route the call through the existing confirm turn. Guessing wrong in
    that direction only costs a question.
  - The catalogue stays small on purpose — allow_tools, and max_tools=12 per
    server. The resident model is a 1.7B with a 4096-token context; a tool name
    it half-remembers is a wrong act.
  - Only the tool name and the router's arguments are sent. There is no API
    here through which a note, a fact or the persona block could travel.

webfetch grows Post (JSON-RPC cannot be a GET) and surfaces response headers
for Mcp-Session-Id. It shares Get's guards exactly: a body buys a caller
nothing, a POST to the LAN is refused for the same reason a GET is.

Verified against the real Vikunja MCP server on homesrv
(http://localhost:9100/mcp): handshake, three discovered tools with update_task
correctly NOT read-only, a live list_projects call, a tool excluded by
allow_tools refused, and the same server refused outright once allow_private
was dropped. Tests cover both transports (the stdio one against a real
subprocess), SSE and JSON framing, session echo, reconnect, and the config
validation.
This commit is contained in:
kami
2026-08-01 04:22:52 +04:00
parent be066a4b04
commit 95ae900a58
13 changed files with 2108 additions and 6 deletions
+72
View File
@@ -3,6 +3,7 @@ package webfetch
import (
"context"
"errors"
"io"
"net"
"net/http"
"net/http/httptest"
@@ -225,3 +226,74 @@ func TestUserAgentIsSent(t *testing.T) {
t.Fatalf("user-agent = %q", ua)
}
}
func TestPostSendsBodyAndHeaders(t *testing.T) {
type seen struct {
method, ctype, accept, ua, custom string
body []byte
}
ch := make(chan seen, 1)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
ch <- seen{r.Method, r.Header.Get("Content-Type"), r.Header.Get("Accept"),
r.Header.Get("User-Agent"), r.Header.Get("X-Thing"), b}
w.Header().Set("Mcp-Session-Id", "sess-9")
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true}`))
}))
defer srv.Close()
f := testFetcher(t, Config{UserAgent: "Maven/test"})
resp, err := f.Post(context.Background(), srv.URL, "application/json",
[]byte(`{"jsonrpc":"2.0"}`), map[string]string{"Accept": "text/event-stream", "X-Thing": "1"})
if err != nil {
t.Fatal(err)
}
if string(resp.Body) != `{"ok":true}` {
t.Fatalf("body = %q", resp.Body)
}
if resp.Header["Mcp-Session-Id"] != "sess-9" {
t.Fatalf("response headers not surfaced: %+v", resp.Header)
}
s := <-ch
if s.method != http.MethodPost {
t.Fatalf("method = %s", s.method)
}
if string(s.body) != `{"jsonrpc":"2.0"}` {
t.Fatalf("request body = %q", s.body)
}
if s.ctype != "application/json" {
t.Fatalf("content-type = %q", s.ctype)
}
if s.accept != "text/event-stream" || s.custom != "1" {
t.Fatalf("caller headers dropped: %+v", s)
}
if s.ua != "Maven/test" {
t.Fatalf("user-agent = %q — a caller must not be able to override it", s.ua)
}
}
// The whole point of routing MCP through webfetch: a POST is guarded exactly
// like a GET. A body does not buy a caller a way onto the LAN.
func TestPostRefusesPrivateAddress(t *testing.T) {
f := New(Config{}) // no AllowPrivate
_, err := f.Post(context.Background(), "http://127.0.0.1:9100/mcp", "application/json", []byte(`{}`), nil)
if !errors.Is(err, ErrPrivate) {
t.Fatalf("error = %v, want ErrPrivate", err)
}
}
func TestPostRefusesNonHTTPScheme(t *testing.T) {
f := New(Config{})
if _, err := f.Post(context.Background(), "file:///etc/passwd", "application/json", nil, nil); !errors.Is(err, ErrScheme) {
t.Fatalf("error = %v, want ErrScheme", err)
}
}
func TestPostObeysDenylist(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
defer srv.Close()
f := testFetcher(t, Config{DenyHosts: []string{"127.0.0.1"}})
if _, err := f.Post(context.Background(), srv.URL, "application/json", []byte(`{}`), nil); !errors.Is(err, ErrBlocked) {
t.Fatalf("error = %v, want ErrBlocked", err)
}
}