From ceb907b2fe53b8222b06fd2460cda1efe7be5127 Mon Sep 17 00:00:00 2001 From: Latte Date: Sat, 4 Jul 2026 18:22:09 +0200 Subject: [PATCH] docs: document the remote-HTTP serve mode, env vars, nginx and Mistral setup - README + getting-started: a "Serve over HTTPS (remote MCP)" section with the `familiar serve` command, the FAMILIAR_HTTP_* env vars, the write-off-by- default note, an nginx location block (proxy_buffering off, long read timeout), the headless-login / copy-the-store options, and the steps to add it as a Mistral custom connector. - architecture: describe the stdio/HTTP transport split, the bearer-auth gate, and the HTTP write default. - CLAUDE.md: add the new commands and env vars to Commands + Architecture. --- CLAUDE.md | 8 +++-- README.md | 56 +++++++++++++++++++++++++++++++++++ doc/architecture.md | 29 +++++++++++++++++- doc/getting-started.md | 67 +++++++++++++++++++++++++++++++++++++++++- 4 files changed, 156 insertions(+), 4 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 05be6da..bdf29b5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -17,9 +17,11 @@ uv run pytest tests/test_market.py::test_get_market_price # single test uv run ruff format . # format uv run ruff check . # lint (run both before committing; CI enforces them) uv run familiar # run the MCP server over stdio (how Claude launches it) +uv run familiar serve # run the streamable-HTTP transport (remote MCP; needs FAMILIAR_HTTP_TOKEN) uv run familiar login # browser EVE SSO login (adds a character; run once per character) +uv run familiar login --no-browser # headless login: paste the callback URL on stdin uv run familiar status # list logged-in characters -uv run familiar writes disable # block write actions (default: enabled) +uv run familiar writes disable # block write actions (default: enabled on stdio, disabled on HTTP) ``` There is no separate build step; `uv build` produces the wheel (CI does this on merge). @@ -28,13 +30,15 @@ There is no separate build step; `uv build` produces the wheel (CI does this on **Server assembly.** [src/familiar/server.py](src/familiar/server.py) creates the single `FastMCP` instance and calls `register(mcp)` on each tool module. Every tool module (market, universe, account, industry, trade, structures, killmails, profile, corp, corp_economy, misc, comms, contacts, fittings, fleet, ui, planets, writes) follows the same shape: **module-level async functions** + a `TOOLS` list + `register(mcp)` that loops `mcp.tool(fn)`. Tools are plain importable functions (so tests call them directly); registration is separate. To add tools: new module → import + register in server.py. +**Transport (stdio default, opt-in HTTP).** `familiar` (no args) runs `mcp.run()` over **stdio** (how Claude launches it) — local, no auth. `familiar serve` runs `server.serve_http()`: the FastMCP **streamable-HTTP** transport bound to loopback (`FAMILIAR_HTTP_HOST`/`FAMILIAR_HTTP_PORT`/`FAMILIAR_HTTP_PATH`, default `127.0.0.1:8080/mcp`) behind a TLS reverse proxy, for remote MCP clients (e.g. a Mistral custom connector). `configure_http()` attaches a FastMCP `StaticTokenVerifier` from `FAMILIAR_HTTP_TOKEN` (bad/missing bearer → 401 + `WWW-Authenticate: Bearer`) and **refuses to start (`SystemExit`) if the token is unset**. This transport-level bearer token is separate from the EVE-SSO/ESI auth in auth.py. In HTTP mode `configure_http` also calls `writes.set_http_context(True)`, flipping the **write default to disabled** unless `FAMILIAR_ENABLE_WRITES` is explicitly truthy (only the default resolution changes; `writes.guard` is untouched). + **ESI access.** [esi.py](src/familiar/esi.py) is a thin async httpx wrapper (`client()` singleton, `get/post/put/delete`, `get_all_pages` for `X-Pages` pagination, Bearer `token=` for authed calls, uniform `EsiError`). Base URL and datasource are env-overridable. [zkill.py](src/familiar/zkill.py) mirrors it for zKillboard. Both expose `set_client()` so tests inject an `httpx.MockTransport`. **Auth is multi-character.** [auth.py](src/familiar/auth.py) does EVE SSO PKCE (no client secret), stores a **dict of characters keyed by id** in one Fernet-encrypted file (`load_store`/`_save_character`/`list_characters`/`remove_character`), and `get_auth(character_id)` returns a valid access token, refreshing via the rotating refresh token when near expiry. The legacy single-character file is migrated on load. Config (client id, callback `localhost:8765`, the 73 read+write scopes split into `READ_SCOPES`/`WRITE_SCOPES`) lives in [config.py](src/familiar/config.py), all env-overridable. **Per-character / per-corp pattern.** [charutil.py](src/familiar/charutil.py) is the backbone of every authenticated tool. `run_per_character(character, fetch)` runs a fetcher for one named character or **all** logged-in ones (when `character` is None), returning a uniform `{"characters": [...]}`. `run_per_corp` additionally resolves each character's `corporation_id` (corp endpoints need the right corp role or ESI 403s). `one_token` picks a single token for thing-scoped lookups (e.g. a structure). `summarize(items, limit)` gives `{count, sample}`. account.py and all read modules build on these. -**Write safety — the choke point.** [writes.py](src/familiar/writes.py) `guard(action, params, confirm, execute)` is the single gate every mutating tool calls. `confirm=False` → dry-run preview (nothing happens); `confirm=True` but writes disabled → `WriteDisabled`; `confirm=True` + enabled → execute + append to an audit log. Writes are **on by default** (opt out via `familiar writes disable`; toggle via `familiar writes enable`/`disable` or `FAMILIAR_ENABLE_WRITES`). Write tools take a *required* `character` and a `confirm` flag; reads take an *optional* `character`. When adding a write tool, route it through `writes.guard` — never call a mutating ESI endpoint directly. +**Write safety — the choke point.** [writes.py](src/familiar/writes.py) `guard(action, params, confirm, execute)` is the single gate every mutating tool calls. `confirm=False` → dry-run preview (nothing happens); `confirm=True` but writes disabled → `WriteDisabled`; `confirm=True` + enabled → execute + append to an audit log. Writes are **on by default over stdio** (opt out via `familiar writes disable`; toggle via `familiar writes enable`/`disable` or `FAMILIAR_ENABLE_WRITES`) but **off by default over HTTP** (see the transport note — `set_http_context` flips the default; `FAMILIAR_ENABLE_WRITES` still overrides). Write tools take a *required* `character` and a `confirm` flag; reads take an *optional* `character`. When adding a write tool, route it through `writes.guard` — never call a mutating ESI endpoint directly. ## Testing conventions diff --git a/README.md b/README.md index e05c00a..a21605e 100644 --- a/README.md +++ b/README.md @@ -77,6 +77,62 @@ To opt out and block all writes globally: uv run familiar writes disable # turn writes off (uv run familiar writes enable to re-allow) ``` +## Serve over HTTPS (remote MCP) + +By default familiar talks over **stdio** — the local transport Claude launches. To reach it from a remote MCP client (for example a [chat.mistral.ai](https://chat.mistral.ai) custom connector), run the opt-in **streamable-HTTP** transport instead: + +```bash +FAMILIAR_HTTP_TOKEN=$(openssl rand -hex 32) uv run familiar serve +``` + +`familiar serve` binds `127.0.0.1:8080/mcp` and expects a TLS-terminating reverse proxy in front of it. It **refuses to start without `FAMILIAR_HTTP_TOKEN`** — an internet-reachable account window must never run unauthenticated. Requests without a valid `Authorization: Bearer ` get `401` with a `WWW-Authenticate: Bearer` header; a valid token lets the MCP handshake through. stdio (`familiar` with no args) is unchanged and never token-gated. + +Environment variables: + +| Variable | Default | Meaning | +| --- | --- | --- | +| `FAMILIAR_HTTP_TOKEN` | *(required)* | Bearer token the client must send. No default — serve refuses to start without it. | +| `FAMILIAR_HTTP_HOST` | `127.0.0.1` | Bind address. Leave loopback and let the proxy face the internet. | +| `FAMILIAR_HTTP_PORT` | `8080` | Bind port. | +| `FAMILIAR_HTTP_PATH` | `/mcp` | MCP endpoint path. | +| `FAMILIAR_ENABLE_WRITES` | *(unset)* | In HTTP mode **writes default off**; set to `1`/`true` to opt in. | + +**Writes are disabled by default over HTTP** — a network-reachable, LLM-driven endpoint should not send mail or set waypoints unless you explicitly opt in with `FAMILIAR_ENABLE_WRITES=1`. (Over stdio they stay on by default, as before.) + +### nginx + +Streamable HTTP keeps a long-lived response open, so disable proxy buffering and give it a generous read timeout: + +```nginx +location /mcp { + proxy_pass http://127.0.0.1:8080/mcp; + proxy_http_version 1.1; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_buffering off; # stream events straight through + proxy_read_timeout 3600s; # don't cut long-lived MCP sessions +} +``` + +### Add characters on a headless server + +`familiar login` opens a browser and captures the callback on localhost, which needs a desktop. On a headless box you have two options: + +1. **Headless login** — print the SSO URL, authorize in any browser, paste the redirected URL back: + + ```bash + uv run familiar login --no-browser + ``` + + The localhost page the browser lands on will fail to load — that's expected; copy its address bar (it contains the `code`) and paste it at the prompt. + +2. **Copy the store** — run `familiar login` on your desktop, then copy the encrypted store (`token.enc` + `key` from your `FAMILIAR_DATA_DIR`) to the server's data dir. + +### Connect it in Mistral + +In [chat.mistral.ai](https://chat.mistral.ai): **Intelligence → Connectors → Add custom connector**, set the URL to `https://your-host/mcp`, choose **Bearer Token** auth, and paste the `FAMILIAR_HTTP_TOKEN` value. + ## Roadmap Built in phases. Each phase is independently useful, so the project stays shippable. diff --git a/doc/architecture.md b/doc/architecture.md index 6187750..2cd69ab 100644 --- a/doc/architecture.md +++ b/doc/architecture.md @@ -40,6 +40,31 @@ Adding tools is mechanical: **write a new module, import it in `server.py`, and > text predates the write actions and is historical framing; the real behaviour is read **and** > guarded write (see [Write Safety](write-safety.md)). +## Transport — stdio (default) and HTTP (opt-in) + +The same `mcp` instance is served over one of two transports; the tools and every layer below are +identical either way. + +- **stdio (default).** `familiar` with no arguments calls `mcp.run()` — the local transport Claude + Desktop / Claude Code / Cowork launch over the process's stdin/stdout. No network, no auth layer: + the transport is local to the machine and trusted. +- **streamable-HTTP (opt-in).** `familiar serve` calls `server.serve_http()`, which runs the FastMCP + streamable-HTTP transport bound to loopback (`FAMILIAR_HTTP_HOST`/`PORT`/`PATH`, default + `127.0.0.1:8080/mcp`) behind a TLS-terminating reverse proxy. `configure_http()` attaches a FastMCP + `StaticTokenVerifier` built from `FAMILIAR_HTTP_TOKEN`, so a request must present a matching + `Authorization: Bearer ` — anything else is a `401` with a `WWW-Authenticate: Bearer` header. + Serving **refuses to start without a token** (a `SystemExit` from `configure_http`), so the endpoint + is never open to the internet unauthenticated. + +The EVE-SSO/ESI auth in `auth.py` is unrelated to this transport-level bearer token: the bearer token +gates the *MCP client* (e.g. Mistral) reaching familiar, while `auth.py` gates familiar reaching ESI. + +**Write default depends on the transport.** `configure_http()` calls `writes.set_http_context(True)`, +which flips the *default* write policy: over HTTP writes are **disabled** unless `FAMILIAR_ENABLE_WRITES` +is explicitly truthy, because a network-reachable, LLM-driven endpoint should not send mail or set +waypoints by accident. Over stdio the default is unchanged (writes on). This only changes the default +resolution — the `writes.guard` choke point and the per-call `confirm=true` requirement are untouched. + ## The tool-module pattern Every tool module — market, universe, account, industry, trade, structures, killmails, profile, corp, @@ -91,7 +116,9 @@ when to call it, so docstrings are written for that audience. **multi-character** encrypted store. Full details in [Authentication](authentication.md). The short version: -- `interactive_login()` runs the browser flow once per character and persists a bundle. +- `interactive_login()` runs the browser flow once per character and persists a bundle; + `manual_login()` is the headless variant (`familiar login --no-browser`) that prints the authorize + URL and reads the pasted callback URL from stdin — no local browser or callback listener needed. - The store is a dict of characters keyed by id, in one Fernet-encrypted file; a legacy single-character file is migrated on load. - `get_auth(character_id)` returns a valid access token, transparently refreshing via the rotating diff --git a/doc/getting-started.md b/doc/getting-started.md index 1214b83..1ad5655 100644 --- a/doc/getting-started.md +++ b/doc/getting-started.md @@ -99,7 +99,72 @@ uv run familiar writes # show current state Full details in [Write Safety](write-safety.md). -## 7. Verify everything works +## 7. Serve over HTTPS (remote MCP) + +Everything above uses **stdio** — the local transport an MCP client launches on your machine. +To reach familiar from a **remote** MCP client (for example a [chat.mistral.ai](https://chat.mistral.ai) +custom connector), run the opt-in **streamable-HTTP** transport instead. stdio stays the default; +nothing here changes how `uv run familiar` behaves with no arguments. + +```bash +FAMILIAR_HTTP_TOKEN=$(openssl rand -hex 32) uv run familiar serve +``` + +`familiar serve` binds `127.0.0.1:8080/mcp` behind a TLS-terminating reverse proxy. It **refuses to +start without `FAMILIAR_HTTP_TOKEN`** — an internet-reachable account window must never run +unauthenticated. A request without a valid `Authorization: Bearer ` gets `401` with a +`WWW-Authenticate: Bearer` header; a valid token lets the MCP handshake through. + +| Variable | Default | Meaning | +| --- | --- | --- | +| `FAMILIAR_HTTP_TOKEN` | *(required)* | Bearer token the client must send; serve refuses to start without it. | +| `FAMILIAR_HTTP_HOST` | `127.0.0.1` | Bind address. Leave loopback; let the proxy face the internet. | +| `FAMILIAR_HTTP_PORT` | `8080` | Bind port. | +| `FAMILIAR_HTTP_PATH` | `/mcp` | MCP endpoint path. | +| `FAMILIAR_ENABLE_WRITES` | *(unset)* | In HTTP mode writes default **off**; set to `1`/`true` to opt in. | + +> **Writes default off over HTTP.** A network-reachable, LLM-driven endpoint should not send mail or +> set waypoints unless you opt in explicitly with `FAMILIAR_ENABLE_WRITES=1`. (Over stdio they stay on +> by default, as in section 6.) + +### nginx location block + +Streamable HTTP holds a long-lived response open, so turn off buffering and raise the read timeout: + +```nginx +location /mcp { + proxy_pass http://127.0.0.1:8080/mcp; + proxy_http_version 1.1; + proxy_set_header Host $host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_buffering off; # stream events straight through + proxy_read_timeout 3600s; # don't cut long-lived MCP sessions +} +``` + +### Logging in on a headless server + +`familiar login` needs a local browser and the `localhost:8765` callback. On a headless box: + +- **Headless login** — print the SSO URL, authorize in any browser, paste the redirected URL back: + + ```bash + uv run familiar login --no-browser + ``` + + The `localhost` page the browser lands on fails to load — that's expected; copy its address bar + (it holds the `code`) and paste it at the prompt. +- **Copy the store** — run `familiar login` on your desktop, then copy the encrypted store + (`token.enc` + `key` from your `FAMILIAR_DATA_DIR`) to the server's data dir. + +### Add it in Mistral + +In [chat.mistral.ai](https://chat.mistral.ai): **Intelligence → Connectors → Add custom connector**, +set the URL to `https://your-host/mcp`, choose **Bearer Token** auth, and paste your +`FAMILIAR_HTTP_TOKEN`. + +## 8. Verify everything works ```bash uv run pytest # full test suite (fails under 80% coverage)