kami/Maven
1
0
Fork 0
Code Issues 1 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Talk MCP: a client for external tool servers (#251)

Browse Source 
docs/plans/06-mcp-support.md asks for the host direction — Maven connects OUT
to MCP servers and consumes what they offer. This is the client half: the
protocol, the transports, the connection manager, the config block. Nothing is
wired into a turn yet, and nothing here exposes Maven's own capabilities to an
outside caller.

internal/mcp:
  - hand-rolled JSON-RPC 2.0 (the wire format is four fields, and the repo
    vendors its deps, so a library would cost more than it saves);
  - two transports: a stdio subprocess on this box, and streamable HTTP, which
    accepts a plain JSON reply or an SSE frame because servers disagree about
    which they send;
  - Client: initialize handshake, tools/list, tools/call, resources/list,
    resources/read. Text content only — everything downstream is a sentence;
  - Manager: lazy dial, per-server failure that never blocks boot or the other
    servers, backoff reconnect, Status for a web surface, graceful Close;
  - the allowlist encoding: a discovered tool becomes the store row
    "vikunja_list_tasks" with cmd ["mcp","vikunja","list_tasks"], scope
    "mcp:vikunja". No new column, no migration, and ProposeTool, EnableTool,
    the act matcher and the confirm turn all keep working untouched.

Constraints held, in code rather than in prose:
  - OFF unless configured, and a server is dark until "enabled": true.
  - A url server goes through internal/webfetch, so the SSRF guard, the size
    cap, the redirect cap and the per-host rate limit apply. Reaching loopback
    needs allow_private on THAT server, and each server gets its own fetcher so
    one loopback exemption cannot become a hole for a public endpoint.
  - readOnlyHint decides destructive: no hint means "assume it mutates", which
    will route the call through the existing confirm turn. Guessing wrong in
    that direction only costs a question.
  - The catalogue stays small on purpose — allow_tools, and max_tools=12 per
    server. The resident model is a 1.7B with a 4096-token context; a tool name
    it half-remembers is a wrong act.
  - Only the tool name and the router's arguments are sent. There is no API
    here through which a note, a fact or the persona block could travel.

webfetch grows Post (JSON-RPC cannot be a GET) and surfaces response headers
for Mcp-Session-Id. It shares Get's guards exactly: a body buys a caller
nothing, a POST to the LAN is refused for the same reason a GET is.

Verified against the real Vikunja MCP server on homesrv
(http://localhost:9100/mcp): handshake, three discovered tools with update_task
correctly NOT read-only, a live list_projects call, a tool excluded by
allow_tools refused, and the same server refused outright once allow_private
was dropped. Tests cover both transports (the stdio one against a real
subprocess), SSE and JSON framing, session echo, reconnect, and the config
validation.
This commit is contained in:
kami
2026-08-01 04:22:52 +04:00
parent be066a4b04
commit 95ae900a58
13 changed files with 2108 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
internal/webfetch/webfetch.go
+53 -6
View File
@@ -28,6 +28,7 @@
package webfetch
import (
"bytes"
"context"
"errors"
"fmt"
@@ -92,6 +93,9 @@ type Response struct {
Status int
ContentType string
Body []byte
// Header — the response headers, one value each (the first). Populated for
// every request; MCP needs Mcp-Session-Id, nothing else reads it.
Header map[string]string
}
// Fetcher performs guarded GETs. Safe for concurrent use; the per-host rate
@@ -159,6 +163,37 @@ func New(cfg Config) *Fetcher {
// Get fetches rawURL. The body is capped: a larger response is an error, not a
// truncation, because half an XML document is worse than none.
func (f *Fetcher) Get(ctx context.Context, rawURL string) (*Response, error) {
return f.do(ctx, http.MethodGet, rawURL, nil, nil)
}
// Post sends body to rawURL and returns the reply, under exactly the same
// guards as Get: scheme rule, host lists, the dialer's private-address check on
// every hop, the size cap and the per-host rate limit.
//
// It exists for JSON-RPC over HTTP (internal/mcp), which cannot be expressed as
// a GET. That an outbound request now carries a body does not widen the
// address policy one bit — a POST to the LAN is refused for the same reason a
// GET is, unless AllowPrivate was set for that specific fetcher.
//
// hdr is merged over the defaults; a caller may not override User-Agent or
// Accept-Encoding, because identity encoding and an honest UA are part of the
// contract with whatever is on the other end.
func (f *Fetcher) Post(ctx context.Context, rawURL, contentType string, body []byte, hdr map[string]string) (*Response, error) {
if contentType == "" {
contentType = "application/json"
}
if hdr == nil {
hdr = map[string]string{}
}
merged := make(map[string]string, len(hdr)+1)
for k, v := range hdr {
merged[k] = v
}
merged["Content-Type"] = contentType
return f.do(ctx, http.MethodPost, rawURL, body, merged)
}
func (f *Fetcher) do(ctx context.Context, method, rawURL string, body []byte, hdr map[string]string) (*Response, error) {
u, err := url.Parse(strings.TrimSpace(rawURL))
if err != nil {
return nil, fmt.Errorf("webfetch: bad url %q: %w", rawURL, err)
@@ -170,10 +205,17 @@ func (f *Fetcher) Get(ctx context.Context, rawURL string) (*Response, error) {
return nil, err
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u.String(), nil)
var rdr io.Reader
if body != nil {
rdr = bytes.NewReader(body)
}
req, err := http.NewRequestWithContext(ctx, method, u.String(), rdr)
if err != nil {
return nil, err
}
for k, v := range hdr {
req.Header.Set(k, v)
}
req.Header.Set("User-Agent", f.cfg.UserAgent)
req.Header.Set("Accept-Encoding", "identity")
@@ -190,22 +232,27 @@ func (f *Fetcher) Get(ctx context.Context, rawURL string) (*Response, error) {
}
defer resp.Body.Close()
body, err := io.ReadAll(io.LimitReader(resp.Body, f.cfg.MaxBytes+1))
respBody, err := io.ReadAll(io.LimitReader(resp.Body, f.cfg.MaxBytes+1))
if err != nil {
return nil, err
}
if int64(len(body)) > f.cfg.MaxBytes {
if int64(len(respBody)) > f.cfg.MaxBytes {
return nil, fmt.Errorf("%w (%d bytes)", ErrTooLarge, f.cfg.MaxBytes)
}
if resp.StatusCode < 200 || resp.StatusCode > 299 {
return nil, fmt.Errorf("%w: %d", ErrStatus, resp.StatusCode)
}
return &Response{
out := &Response{
URL: resp.Request.URL.String(),
Status: resp.StatusCode,
ContentType: resp.Header.Get("Content-Type"),
Body: body,
}, nil
Body: respBody,
Header: map[string]string{},
}
for k := range resp.Header {
out.Header[k] = resp.Header.Get(k)
}
return out, nil
}
// checkURL applies the scheme rule and the host lists. The address rule is the
internal/webfetch/webfetch_test.go
+72
View File
@@ -3,6 +3,7 @@ package webfetch
import (
"context"
"errors"
"io"
"net"
"net/http"
"net/http/httptest"
@@ -225,3 +226,74 @@ func TestUserAgentIsSent(t *testing.T) {
t.Fatalf("user-agent = %q", ua)
}
}
func TestPostSendsBodyAndHeaders(t *testing.T) {
type seen struct {
method, ctype, accept, ua, custom string
body []byte
}
ch := make(chan seen, 1)
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
b, _ := io.ReadAll(r.Body)
ch <- seen{r.Method, r.Header.Get("Content-Type"), r.Header.Get("Accept"),
r.Header.Get("User-Agent"), r.Header.Get("X-Thing"), b}
w.Header().Set("Mcp-Session-Id", "sess-9")
w.Header().Set("Content-Type", "application/json")
_, _ = w.Write([]byte(`{"ok":true}`))
}))
defer srv.Close()
f := testFetcher(t, Config{UserAgent: "Maven/test"})
resp, err := f.Post(context.Background(), srv.URL, "application/json",
[]byte(`{"jsonrpc":"2.0"}`), map[string]string{"Accept": "text/event-stream", "X-Thing": "1"})
if err != nil {
t.Fatal(err)
}
if string(resp.Body) != `{"ok":true}` {
t.Fatalf("body = %q", resp.Body)
}
if resp.Header["Mcp-Session-Id"] != "sess-9" {
t.Fatalf("response headers not surfaced: %+v", resp.Header)
}
s := <-ch
if s.method != http.MethodPost {
t.Fatalf("method = %s", s.method)
}
if string(s.body) != `{"jsonrpc":"2.0"}` {
t.Fatalf("request body = %q", s.body)
}
if s.ctype != "application/json" {
t.Fatalf("content-type = %q", s.ctype)
}
if s.accept != "text/event-stream" || s.custom != "1" {
t.Fatalf("caller headers dropped: %+v", s)
}
if s.ua != "Maven/test" {
t.Fatalf("user-agent = %q — a caller must not be able to override it", s.ua)
}
}
// The whole point of routing MCP through webfetch: a POST is guarded exactly
// like a GET. A body does not buy a caller a way onto the LAN.
func TestPostRefusesPrivateAddress(t *testing.T) {
f := New(Config{}) // no AllowPrivate
_, err := f.Post(context.Background(), "http://127.0.0.1:9100/mcp", "application/json", []byte(`{}`), nil)
if !errors.Is(err, ErrPrivate) {
t.Fatalf("error = %v, want ErrPrivate", err)
}
}
func TestPostRefusesNonHTTPScheme(t *testing.T) {
f := New(Config{})
if _, err := f.Post(context.Background(), "file:///etc/passwd", "application/json", nil, nil); !errors.Is(err, ErrScheme) {
t.Fatalf("error = %v, want ErrScheme", err)
}
}
func TestPostObeysDenylist(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {}))
defer srv.Close()
f := testFetcher(t, Config{DenyHosts: []string{"127.0.0.1"}})
if _, err := f.Post(context.Background(), srv.URL, "application/json", []byte(`{}`), nil); !errors.Is(err, ErrBlocked) {
t.Fatalf("error = %v, want ErrBlocked", err)
}
}