test: integration coverage report, suite docs and harness isolation fix
This commit is contained in:
1 parent
ae6d83c3bd
commit
446ec11954
6 files changed
+342
-5
No files matched your search
@@ -21,7 +21,8 @@ message instead of skipping.
|
||||
- Gitea is configured through `GITEA__section__KEY` env vars (sqlite3,
|
||||
`INSTALL_LOCK`, random `SECRET_KEY`/`INTERNAL_TOKEN`/OAuth2 JWT secret,
|
||||
Actions, packages and OAuth2 enabled, mailer off, webhook allowed host list
|
||||
`*`, `MAX_RESPONSE_ITEMS=50`).
|
||||
`*`, local network migrations allowed so the instance can mirror itself, repository
|
||||
hook scripts editable, `MAX_RESPONSE_ITEMS=50`).
|
||||
- Readiness: polls `GET /api/v1/version` for up to 90 s. On failure the last 100
|
||||
log lines of the container are written to stderr.
|
||||
- Seeding: admin `aegis-admin` (random password) via `gitea admin user create`,
|
||||
@@ -50,3 +51,40 @@ message instead of skipping.
|
||||
`create_org`, `create_repo`, `unique`.
|
||||
- `regular_user` (session): non-admin user with its own full-scope token.
|
||||
- `unique_name` (function): a unique name derived from the test name.
|
||||
|
||||
## The suite
|
||||
|
||||
Tests call tools through the MCP server (an in-memory `Client` on `build_server`, run as
|
||||
the admin of the throwaway instance), so schema validation, the result envelope, the audit
|
||||
log and the executor are all exercised. State lives in `tmp_path`.
|
||||
|
||||
- `test_scenario_*.py`: happy paths with real state changes (org and team, repository
|
||||
files, branches and tags, issues, pull requests and reviews, releases and attachments,
|
||||
secrets and variables, webhooks, admin and sudo, notifications, current-user features,
|
||||
generic packages, and a workflow run on an `act_runner` container).
|
||||
- `test_sweep.py`: every one of the 538 tools is called once against a seeded instance
|
||||
(`sweep_seed.py` creates the data, `sweep_plan.py` builds arguments and expectations).
|
||||
Order: read-only first, then creates, updates and deletes (leaf resources first). A call
|
||||
passes with a 2xx, a status the swagger documents for that operation, or an entry of
|
||||
`ALLOWANCES` (each with a reason). All failing tools are listed at once.
|
||||
`test_sweep_isolated.py` covers the tools the sweep must not call.
|
||||
- `test_per_user_oauth.py`: the HTTP app with per-user OAuth, driven in process with a
|
||||
seeded OAuth store; calls run as the signed-in user, admin tools and private repositories
|
||||
of others are refused by Gitea, the static bearer still runs as admin.
|
||||
- `test_setup_live.py`: `aegis-gitea-mcp setup` against the instance (OAuth app, env file
|
||||
mode, idempotence, non-admin token).
|
||||
- `tests/test_sweep_plan.py` (no Docker) keeps the sweep plan consistent with the catalog.
|
||||
|
||||
`AEGIS_IT_WRITE_REPORT=1` makes the session write `docs/integration-coverage.md` (per
|
||||
domain: tools with a 2xx, documented non-2xx statuses, allowances).
|
||||
|
||||
### Actions runner
|
||||
|
||||
`start_runner` starts `gitea/act_runner` (digest-pinned in `ACT_RUNNER_IMAGE`) on the run's
|
||||
network, registered with a token from `gitea_actions_repo_create_runner_registration_token`,
|
||||
label `aegis-host:host`. Jobs run inside the runner container itself: no Docker socket, no
|
||||
bind mounts. The runner reaches Gitea by container name, so it works in both network modes.
|
||||
It is removed with the rest of the run. If it cannot start or come online, the test is
|
||||
marked xfail with the reason; a failure after the run completed is a real failure.
|
||||
|
||||
Runtime: about two minutes locally (Gitea start 15 s, sweep 40 s, runner 10 s).
|
||||
Reference in new issue
Block a user