test: integration coverage report, suite docs and harness isolation fix

This commit is contained in:
Latte committed 2026-10-14 17:57:54 +02:00
1 parent ae6d83c3bd
commit 446ec11954
6 files changed
+342 -5

No files matched your search

+39 -1
View File
@@ -21,7 +21,8 @@ message instead of skipping.
- Gitea is configured through `GITEA__section__KEY` env vars (sqlite3,
`INSTALL_LOCK`, random `SECRET_KEY`/`INTERNAL_TOKEN`/OAuth2 JWT secret,
Actions, packages and OAuth2 enabled, mailer off, webhook allowed host list
`*`, `MAX_RESPONSE_ITEMS=50`).
`*`, local network migrations allowed so the instance can mirror itself, repository
hook scripts editable, `MAX_RESPONSE_ITEMS=50`).
- Readiness: polls `GET /api/v1/version` for up to 90 s. On failure the last 100
log lines of the container are written to stderr.
- Seeding: admin `aegis-admin` (random password) via `gitea admin user create`,
@@ -50,3 +51,40 @@ message instead of skipping.
`create_org`, `create_repo`, `unique`.
- `regular_user` (session): non-admin user with its own full-scope token.
- `unique_name` (function): a unique name derived from the test name.
## The suite
Tests call tools through the MCP server (an in-memory `Client` on `build_server`, run as
the admin of the throwaway instance), so schema validation, the result envelope, the audit
log and the executor are all exercised. State lives in `tmp_path`.
- `test_scenario_*.py`: happy paths with real state changes (org and team, repository
files, branches and tags, issues, pull requests and reviews, releases and attachments,
secrets and variables, webhooks, admin and sudo, notifications, current-user features,
generic packages, and a workflow run on an `act_runner` container).
- `test_sweep.py`: every one of the 538 tools is called once against a seeded instance
(`sweep_seed.py` creates the data, `sweep_plan.py` builds arguments and expectations).
Order: read-only first, then creates, updates and deletes (leaf resources first). A call
passes with a 2xx, a status the swagger documents for that operation, or an entry of
`ALLOWANCES` (each with a reason). All failing tools are listed at once.
`test_sweep_isolated.py` covers the tools the sweep must not call.
- `test_per_user_oauth.py`: the HTTP app with per-user OAuth, driven in process with a
seeded OAuth store; calls run as the signed-in user, admin tools and private repositories
of others are refused by Gitea, the static bearer still runs as admin.
- `test_setup_live.py`: `aegis-gitea-mcp setup` against the instance (OAuth app, env file
mode, idempotence, non-admin token).
- `tests/test_sweep_plan.py` (no Docker) keeps the sweep plan consistent with the catalog.
`AEGIS_IT_WRITE_REPORT=1` makes the session write `docs/integration-coverage.md` (per
domain: tools with a 2xx, documented non-2xx statuses, allowances).
### Actions runner
`start_runner` starts `gitea/act_runner` (digest-pinned in `ACT_RUNNER_IMAGE`) on the run's
network, registered with a token from `gitea_actions_repo_create_runner_registration_token`,
label `aegis-host:host`. Jobs run inside the runner container itself: no Docker socket, no
bind mounts. The runner reaches Gitea by container name, so it works in both network modes.
It is removed with the rest of the run. If it cannot start or come online, the test is
marked xfail with the reason; a failure after the run completed is a real failure.
Runtime: about two minutes locally (Gitea start 15 s, sweep 40 s, runner 10 s).