docs: rewrite README and docs for the full-access design, generate tool inventory
This commit is contained in:
1 parent
68bd8700dc
commit
7c7cfc05ff
13 files changed
+1693
-1026
No files matched your search
+121
-180
@@ -1,206 +1,147 @@
|
||||
# Architecture
|
||||
|
||||
AegisGitea MCP turns the Gitea REST API into MCP tools. The design decisions are recorded in
|
||||
[docs/adr](adr/0001-full-access-by-default.md); this page describes how the pieces fit.
|
||||
|
||||
## Overview
|
||||
|
||||
AegisGitea MCP is a Python 3.10+ application split into a **transport-agnostic core** and **two thin transport adapters** that consume it. It bridges an AI client (Claude, Claude Code, Cowork) and a self-hosted Gitea instance, implementing the [Model Context Protocol (MCP)](https://modelcontextprotocol.io).
|
||||
|
||||
```
|
||||
┌──────────────────────── shared core ────────────────────────┐
|
||||
│ registry.py (name -> handler; single source of truth) │
|
||||
│ tools/* (async handlers: gitea, arguments, raw) │
|
||||
│ policy.py (allow/deny, WRITE_MODE gate) │
|
||||
│ authz.py (resource-type-aware authorization) │
|
||||
│ gitea_client · audit · security · response_limits · config │
|
||||
│ errors.ToolError (transport-agnostic error type) │
|
||||
│ NO fastapi / uvicorn imports (locked by a boundary test) │
|
||||
└───────▲───────────────────────────────────────▲─────────────┘
|
||||
│ │
|
||||
┌────────────────┴───────────┐ ┌──────────────┴──────────────┐
|
||||
│ HTTP / OAuth adapter │ │ Local stdio adapter │
|
||||
│ server.py (FastAPI) │ │ stdio_app.py (mcp SDK) │
|
||||
│ per-user OAuth2/OIDC, DCR, │ │ single PAT owner, no OAuth, │
|
||||
│ rate limit, per-user repo │ │ policy + WRITE_MODE + audit │
|
||||
│ authz + resource-type gate │ │ over stdio │
|
||||
│ [server] extra │ │ core install │
|
||||
└─────────────────────────────┘ └──────────────────────────────┘
|
||||
MCP client
|
||||
| stdio, or streamable HTTP (/mcp)
|
||||
v
|
||||
SDK transport + auth verifier (transports.py, auth.py)
|
||||
| access token -> Identity
|
||||
v
|
||||
Server (low-level mcp Server) (server.py)
|
||||
| list_tools / call_tool, filters, argument validation
|
||||
v
|
||||
GiteaExecutor (gitea/executor.py)
|
||||
| path templating, HTTP call with the identity's token, optional Sudo header
|
||||
v
|
||||
Gitea REST API (/api/v1)
|
||||
```
|
||||
|
||||
Where the security layers sit on a dispatched call: **scope check → policy
|
||||
(`policy.py`) → resource-type authorization (`authz.py`) → handler → response
|
||||
limits + secret sanitization → audit**. For `gitea_request`, the handler adds a
|
||||
deterministic write classifier, a known-path gate, and the admin/credential
|
||||
denylist. The HTTP adapter runs the per-user repository-permission probe and the
|
||||
resource-type gate; the stdio adapter trusts the PAT owner and skips the
|
||||
per-user probe while keeping policy, `WRITE_MODE`, and audit.
|
||||
Every tool call ends in one audit event, and every result is wrapped in the same envelope.
|
||||
|
||||
The legacy single-process view below still describes the HTTP adapter:
|
||||
## Catalog
|
||||
|
||||
```
|
||||
AI Client (Claude / Claude Code / Cowork)
|
||||
│
|
||||
│ HTTP (Authorization: Bearer <key>)
|
||||
▼
|
||||
┌────────────────────────────────────────────┐
|
||||
│ FastAPI Server │
|
||||
│ server.py │
|
||||
│ - Route: GET/POST /mcp │
|
||||
│ - Route: POST /mcp/tool/call │
|
||||
│ - Route: GET /mcp/tools │
|
||||
│ - Route: GET /health │
|
||||
│ - Streamable HTTP transport │
|
||||
│ - Legacy SSE alias (GET/POST /mcp/sse) │
|
||||
└───────┬───────────────────┬────────────────┘
|
||||
│ │
|
||||
┌────▼────┐ ┌────▼──────────────┐
|
||||
│ auth │ │ Tool dispatcher │
|
||||
│ auth.py│ │ (server.py) │
|
||||
└────┬────┘ └────────┬──────────┘
|
||||
│ │
|
||||
│ ┌────────▼──────────┐
|
||||
│ │ Tool handlers │
|
||||
│ │ tools/repo.py │
|
||||
│ └────────┬──────────┘
|
||||
│ │
|
||||
│ ┌────────▼──────────┐
|
||||
│ │ GiteaClient │
|
||||
│ │ gitea_client.py │
|
||||
│ └────────┬──────────┘
|
||||
│ │ HTTPS
|
||||
│ ▼
|
||||
│ Gitea instance
|
||||
│
|
||||
┌────▼────────────────────┐
|
||||
│ AuditLogger │
|
||||
│ audit.py │
|
||||
│ /var/log/aegis-mcp/ │
|
||||
│ audit.log │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
`spec/gitea-swagger.v1.json` is the Gitea 28.1.0 swagger, pinned (ADR 0003).
|
||||
`scripts/generate_catalog.py` turns it into `src/aegis_gitea_mcp/catalog/tools.json`, which is
|
||||
committed. At runtime only the JSON is read, through `load_catalog()`.
|
||||
|
||||
---
|
||||
Each `Operation` carries the tool name (`gitea_<domain>_<action>`), domain, HTTP method, path
|
||||
template, parameters (path, query, body, form data, file), the complete JSON input schema,
|
||||
content types, and the flags that become MCP annotations: read-only, destructive, idempotent,
|
||||
paginated. Input schemas set `additionalProperties: false`, give owner, repository and username
|
||||
parameters name patterns, take multipart files as base64 strings, and add an optional `sudo`.
|
||||
|
||||
## Source Modules
|
||||
The catalog has 537 operations in 14 domains. [tools.md](tools.md) is generated from it, and
|
||||
CI fails when `tools.json` or `tools.md` drift from the spec or from each other (see
|
||||
[deployment.md](deployment.md) for upgrading).
|
||||
|
||||
### `server.py`
|
||||
## Server
|
||||
|
||||
The entry point and FastAPI application. Responsibilities:
|
||||
`server.py` builds a low-level `mcp` SDK `Server` (ADR 0002) because the high-level server cannot
|
||||
register tools from raw JSON schemas.
|
||||
|
||||
- Defines all HTTP routes
|
||||
- Reads configuration on startup and initialises `GiteaClient`
|
||||
- Applies authentication middleware to protected routes
|
||||
- Dispatches tool calls to the appropriate handler function
|
||||
- Handles CORS
|
||||
- `list_tools` returns one tool per catalog operation, plus the hand-written
|
||||
`gitea_api_request` ([raw-api.md](raw-api.md)). The filters `TOOLSETS`, `READ_ONLY` and
|
||||
`DISABLED_TOOLS` apply here and again when a tool is called, so a filtered tool is unknown.
|
||||
Unknown names in `DISABLED_TOOLS` are a startup error.
|
||||
- `call_tool` validates the arguments with `jsonschema` (the SDK does not), resolves the
|
||||
identity, runs the executor, writes the audit event and returns the result. Failures of the
|
||||
tool are `is_error` results the model can read; only protocol problems (an unknown tool) are
|
||||
protocol errors. Validation messages never contain the offending value, so a secret typed into
|
||||
the wrong field is not echoed.
|
||||
- Unexpected exceptions are caught, audited as internal errors and returned as a generic
|
||||
message. A stack trace never leaves the process.
|
||||
|
||||
### `auth.py`
|
||||
## Executor
|
||||
|
||||
API key validation. Responsibilities:
|
||||
`GiteaExecutor.execute(operation, arguments, identity)` builds one HTTP request:
|
||||
|
||||
- `APIKeyValidator` class: holds the set of valid keys, tracks failed attempts per IP
|
||||
- Constant-time comparison to prevent timing side-channels
|
||||
- Rate limiting: blocks IPs that exceed `MAX_AUTH_FAILURES` within `AUTH_FAILURE_WINDOW`
|
||||
- Helper functions for key generation and hashing
|
||||
- Singleton pattern (`get_validator()`) with test-friendly reset (`reset_validator()`)
|
||||
- path parameters are validated and percent-encoded; query and JSON or multipart bodies are
|
||||
built from the schema; the `sudo` value becomes the `Sudo` header;
|
||||
- the request carries the identity's Gitea token. Redirects are never followed and cookies are
|
||||
never kept, so one identity cannot leak into another call;
|
||||
- responses: JSON and text are decoded, binary bodies become base64 up to `MAX_BINARY_BYTES`,
|
||||
`Link` and `X-Total-Count` become pagination metadata;
|
||||
- errors: Gitea's message is extracted, sanitized and truncated, then mapped to a stable `kind`;
|
||||
- output: items and characters are bounded, the identity's tokens, the configured secrets and
|
||||
secret values submitted in the request are redacted by exact match, and
|
||||
`SECRET_DETECTION_MODE` masks or blocks secret-like strings;
|
||||
- the envelope (ADR 0005) carries the identity, the request, the status, pagination and the
|
||||
`untrusted_content` notice.
|
||||
|
||||
### `config.py`
|
||||
The executor never raises for upstream or argument problems; each is a result with `is_error`.
|
||||
|
||||
Pydantic `BaseSettings` model. Responsibilities:
|
||||
## Transports
|
||||
|
||||
- Loads all configuration from environment variables or `.env`
|
||||
- Validates values (log level enum, token format, key minimum length)
|
||||
- Parses comma-separated `MCP_API_KEYS` into a list
|
||||
- Exposes computed properties (e.g. base URL for Gitea API)
|
||||
stdio (`MCP_TRANSPORT=stdio`, the default) serves one client over stdin and stdout. stdout is
|
||||
reserved for JSON-RPC, all logging goes to stderr. It always runs as admin-token mode with
|
||||
`GITEA_TOKEN`.
|
||||
|
||||
### `gitea_client.py`
|
||||
streamable HTTP (`MCP_TRANSPORT=http`) is the SDK's Starlette app served by uvicorn at
|
||||
`MCP_PATH`, stateless, wrapped with:
|
||||
|
||||
Async HTTP client for the Gitea API. Responsibilities:
|
||||
- bearer authentication and, with OAuth on, the OAuth endpoints of the SDK;
|
||||
- DNS-rebinding protection (allowed `Host` values: loopback with the port, the host of
|
||||
`PUBLIC_BASE_URL`, `ALLOWED_HOSTS`);
|
||||
- rate limiting per client on the MCP path;
|
||||
- `GET /health` and `GET /` (name, version, MCP path).
|
||||
|
||||
- Wraps `httpx.AsyncClient` with bearer token authentication
|
||||
- Maps HTTP status codes to typed exceptions (`GiteaAuthenticationError`, `GiteaNotFoundError`, etc.)
|
||||
- Enforces file size limit before returning file contents
|
||||
- Logs all API calls to the audit logger
|
||||
Startup is fail-closed: a non-loopback bind without authentication, or OAuth enabled with
|
||||
missing settings, exits with code 2 before listening.
|
||||
|
||||
Key methods:
|
||||
## Authentication and identity
|
||||
|
||||
| Method | Gitea endpoint |
|
||||
ADR 0004 has the full decision. Two modes can run together on HTTP.
|
||||
|
||||
Admin-token mode: the static `MCP_BEARER_TOKEN` is compared in constant time (digests are
|
||||
compared, so length does not leak). A match runs the call as `GITEA_TOKEN`, with the admin login
|
||||
resolved once from `GET /user`. stdio is always this mode.
|
||||
|
||||
Per-user OAuth mode: the server is an OAuth 2.1 authorization server for MCP clients (dynamic
|
||||
client registration, PKCE) and a confidential OAuth client of Gitea. `/oauth/callback` receives
|
||||
Gitea's code, exchanges it, and issues the client opaque access and refresh tokens that map to
|
||||
the user's Gitea tokens. A call with such a token runs with the user's own Gitea access token,
|
||||
so Gitea enforces that user's permissions. The mapping lives in `DATA_DIR/oauth-tokens.json`
|
||||
(mode 0600). Gitea tokens in it are encrypted with Fernet (key derived from
|
||||
`TOKEN_ENCRYPTION_KEY`); our own tokens are stored as SHA-256 hashes. Upstream refresh is
|
||||
automatic.
|
||||
|
||||
One token verifier accepts either kind. The dispatcher turns the verified token into an
|
||||
`Identity` (login, mode, token). The token is kept out of `repr` and out of every result and log.
|
||||
|
||||
## Audit
|
||||
|
||||
Every tool call, including failed and invalid ones, produces exactly one event in the
|
||||
append-only, hash-chained audit log. Only argument names are recorded, never values. Details and
|
||||
integrity validation are in [audit-logging.md](audit-logging.md).
|
||||
|
||||
## Request flow
|
||||
|
||||
1. The client sends `tools/call` (stdio, or HTTP with a bearer token).
|
||||
2. HTTP only: host check, rate limit, token verification (static bearer or OAuth).
|
||||
3. The dispatcher looks the tool up (filters applied), validates the arguments against the
|
||||
schema, and resolves the identity.
|
||||
4. The executor calls Gitea with the identity's token.
|
||||
5. The response is bounded, redacted and masked, and wrapped in the envelope.
|
||||
6. One audit event is written and the result is returned.
|
||||
|
||||
## Module map
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `get_current_user()` | `GET /api/v1/user` |
|
||||
| `list_repositories()` | `GET /api/v1/user/repos` |
|
||||
| `get_repository()` | `GET /api/v1/repos/{owner}/{repo}` |
|
||||
| `get_file_contents()` | `GET /api/v1/repos/{owner}/{repo}/contents/{path}` |
|
||||
| `get_tree()` | `GET /api/v1/repos/{owner}/{repo}/git/trees/{ref}` |
|
||||
|
||||
### `audit.py`
|
||||
|
||||
Structured audit logging using `structlog`. Responsibilities:
|
||||
|
||||
- Initialises a `structlog` logger writing JSON to the configured log file
|
||||
- `log_tool_invocation()`: records tool calls with result and correlation ID
|
||||
- `log_access_denied()`: records failed authentication
|
||||
- `log_security_event()`: records rate limit triggers and other security events
|
||||
- Auto-generates UUID correlation IDs when none is provided
|
||||
|
||||
### `mcp_protocol.py`
|
||||
|
||||
MCP data models and tool registry. Responsibilities:
|
||||
|
||||
- Pydantic models: `MCPTool`, `MCPToolCallRequest`, `MCPToolCallResponse`, `MCPListToolsResponse`
|
||||
- `AVAILABLE_TOOLS` list: the canonical list of tools exposed to clients
|
||||
- `get_tool_by_name()`: lookup helper used by the dispatcher
|
||||
|
||||
### `tools/repository.py`
|
||||
|
||||
Concrete tool handler functions. Responsibilities:
|
||||
|
||||
- `list_repositories_tool()`: calls `GiteaClient.list_repositories()`, formats the result
|
||||
- `get_repository_info_tool()`: calls `GiteaClient.get_repository()`, formats metadata
|
||||
- `get_file_tree_tool()`: calls `GiteaClient.get_tree()`, flattens to a list of paths
|
||||
- `get_file_contents_tool()`: calls `GiteaClient.get_file_contents()`, decodes base64
|
||||
|
||||
All handlers return a plain string. `server.py` wraps this in an `MCPToolCallResponse`.
|
||||
|
||||
---
|
||||
|
||||
## Request Lifecycle
|
||||
|
||||
```
|
||||
1. Client sends POST /mcp/tool/call
|
||||
│
|
||||
2. FastAPI routes the request to the tool-call handler in server.py
|
||||
│
|
||||
3. OAuth middleware validates the Bearer token via Gitea OIDC/JWKS or userinfo
|
||||
├── Fail → AuditLogger.log_access_denied() → HTTP 401 / 429
|
||||
└── Pass → continue
|
||||
│
|
||||
4. AuditLogger.log_tool_invocation(status="pending")
|
||||
│
|
||||
5. Tool dispatcher looks up the tool by name (mcp_protocol.get_tool_by_name)
|
||||
│
|
||||
6. Policy engine checks read/write mode and repository/path policy
|
||||
│
|
||||
7. If GITEA_TOKEN is configured, service-PAT authz checks
|
||||
GET /repos/{owner}/{repo}/collaborators/{user}/permission
|
||||
│
|
||||
8. Tool handler function (tools/repository.py) is called
|
||||
│
|
||||
9. GiteaClient makes an async HTTP call to the Gitea API
|
||||
│
|
||||
10. Result (or error) is returned to server.py
|
||||
│
|
||||
11. AuditLogger.log_tool_invocation(status="success" | "error")
|
||||
│
|
||||
12. MCPToolCallResponse is returned to the client
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Key Design Decisions
|
||||
|
||||
**Read by default, writes opt-in.** Read tools are available by default. Write-capable tools require `WRITE_MODE=true`, repository write policy/whitelist approval, and `write:repository` authorization.
|
||||
|
||||
**Gitea controls repository access.** Without `GITEA_TOKEN`, Gitea enforces repository permissions on API calls made with the user's token. With `GITEA_TOKEN`, the service PAT can only execute after the server verifies the requesting user's actual repository permission through Gitea and writes an audit denial if the check fails.
|
||||
|
||||
**Public tool discovery.** `GET /mcp/tools` requires no authentication so that MCP clients can discover the available tools without credentials. All other endpoints require authentication.
|
||||
|
||||
**Minimal persisted state.** The audit log is persisted for tamper-evident review. Dynamic OAuth client registrations are persisted when DCR is enabled. Rate limit counters and short-lived authz caches are in-memory and reset on restart.
|
||||
|
||||
**Async throughout.** FastAPI + `httpx.AsyncClient` means all Gitea API calls are non-blocking, allowing the server to handle concurrent requests efficiently.
|
||||
| `catalog/` | model, naming, schema building, generator (`build.py`), `tools.json`, loader |
|
||||
| `gitea/` | executor, path and query encoding, pagination, redaction |
|
||||
| `server.py` | SDK server factory, tool list, call dispatch, filters |
|
||||
| `transports.py` | stdio and HTTP runners, health route, startup checks |
|
||||
| `auth.py` | static bearer verifier, identity resolution |
|
||||
| `oauth/` | per-user OAuth provider, token store |
|
||||
| `setup_cmd.py`, `setup_snippets.py`, `setup_template.py` | the `setup` command |
|
||||
| `cli.py` | `serve`, `setup`, `version` |
|
||||
| `config.py` | settings and validation |
|
||||
| `identity.py`, `errors.py` | identity and error types |
|
||||
| `audit.py`, `security.py`, `response_limits.py`, `rate_limit.py`, `request_context.py`, `logging_utils.py` | cross-cutting safety |
|
||||
| `scripts/` | `generate_catalog.py`, `validate_audit_log.py`, `check_docs.py` |
|
||||
Reference in new issue
Block a user