106 lines
6.1 KiB
Markdown
106 lines
6.1 KiB
Markdown
# Live-Gitea integration tests
|
|
|
|
These tests run against a throwaway Gitea 28.1.0 started in Docker. They are
|
|
opt-in: without `AEGIS_INTEGRATION=1` every test here is skipped and Docker is
|
|
never touched, so the default `uv run pytest` stays fast.
|
|
|
|
```bash
|
|
AEGIS_INTEGRATION=1 uv run pytest tests/integration -m integration -v
|
|
```
|
|
|
|
An integration-only run covers little of `src/`, so the unit-test coverage gate
|
|
is lowered to 0 for runs that collect nothing but `tests/integration`. If Docker
|
|
is unavailable while `AEGIS_INTEGRATION=1` is set, the session fails with a clear
|
|
message instead of skipping.
|
|
|
|
## What the harness does
|
|
|
|
- Image: `GITEA_IMAGE` in `harness.py` (digest-pinned, single constant).
|
|
- One container and one network per run, both named `aegis-it-<8 hex>`.
|
|
No host bind mounts, no image builds, no `docker prune`.
|
|
- Gitea is configured through `GITEA__section__KEY` env vars (sqlite3,
|
|
`INSTALL_LOCK`, random `SECRET_KEY`/`INTERNAL_TOKEN`/OAuth2 JWT secret,
|
|
Actions, packages and OAuth2 enabled, mailer off, webhook allowed host list
|
|
`*`, local network migrations allowed so the instance can mirror itself, repository
|
|
hook scripts editable, `MAX_RESPONSE_ITEMS=50`).
|
|
- Readiness: polls `GET /api/v1/version` for up to 90 s. On failure the last 100
|
|
log lines of the container are written to stderr.
|
|
- Seeding: admin `aegis-admin` (random password) via `gitea admin user create`,
|
|
plus a full-scope token. All data is fake and random per run. Passwords and
|
|
tokens are kept out of `repr` and error messages.
|
|
- Cleanup: `docker rm -f -v <container>` and `docker network rm <net>` from the
|
|
session fixture finalizer, an `atexit` hook, and SIGTERM/SIGHUP handlers.
|
|
|
|
## Reachability modes
|
|
|
|
1. Local (default): a free loopback port is chosen, published as
|
|
`127.0.0.1:<port>:3000`, and `ROOT_URL` is set to match. Base URL
|
|
`http://127.0.0.1:<port>`.
|
|
2. CI (`AEGIS_IT_NETWORK_MODE=container`): for test runs inside a job container
|
|
that shares the host Docker socket. No port is published. Gitea joins the
|
|
per-run network, the job container (`$HOSTNAME`) is connected to the same
|
|
network with `docker network connect`, and tests reach Gitea at
|
|
`http://aegis-it-<id>:3000`. On cleanup the job container is disconnected
|
|
again before the network is removed.
|
|
|
|
## Fixtures and helpers
|
|
|
|
- `gitea` (session): a `GiteaInstance` with `base_url`, `admin_login`,
|
|
`admin_token`, `api(method, path, token=None, **kw)` (defaults to the admin
|
|
token, `anonymous=True` sends no credentials), `create_user`, `mint_token`,
|
|
`create_org`, `create_repo`, `unique`.
|
|
- `regular_user` (session): non-admin user with its own full-scope token.
|
|
- `unique_name` (function): a unique name derived from the test name.
|
|
|
|
## The suite
|
|
|
|
Tests call tools through the MCP server (an in-memory `Client` on `build_server`, run as
|
|
the admin of the throwaway instance), so schema validation, the result envelope, the audit
|
|
log and the executor are all exercised. State lives in `tmp_path`.
|
|
|
|
- `test_scenario_*.py`: happy paths with real state changes (org and team, repository
|
|
files, branches and tags, issues, pull requests and reviews, releases and attachments,
|
|
secrets and variables, webhooks, admin and sudo, notifications, current-user features,
|
|
generic packages, and a workflow run on an `act_runner` container).
|
|
- `test_sweep.py`: every one of the 538 tools is called once against a seeded instance
|
|
(`sweep_seed.py` creates the data, `sweep_plan.py` builds arguments and expectations).
|
|
Order: read-only first, then creates, updates and deletes (leaf resources first). A call
|
|
passes with a 2xx, a status the swagger documents for that operation, or an entry of
|
|
`ALLOWANCES` (each with a reason). All failing tools are listed at once.
|
|
`sweep_baseline.json` pins the outcome class of every tool (`2xx`, or the exact status of
|
|
a documented non-2xx or allowance). A downgrade (2xx to anything else, or a different
|
|
status) fails the run and all of them are reported together, so an argument or path
|
|
regression cannot hide behind a documented 404. An upgrade only prints a note to refresh
|
|
the file. Regenerate it with `AEGIS_IT_WRITE_BASELINE=1` (only for an intended change,
|
|
and review the diff). `test_sweep_isolated.py` covers the tools the sweep must not call.
|
|
- `test_failure_paths.py`: write tools Gitea refuses (missing repository, invalid color,
|
|
duplicate repository, stale file sha) keep their status and error kind and are audited.
|
|
- `test_per_user_oauth.py`: the HTTP app with per-user OAuth, driven in process with a
|
|
seeded OAuth store; calls run as the signed-in user, admin tools and private repositories
|
|
of others are refused by Gitea, the static bearer still runs as admin.
|
|
- `test_setup_live.py`: `aegis-gitea-mcp setup` against the instance (OAuth app, env file
|
|
mode, idempotence, non-admin token).
|
|
- `tests/test_sweep_plan.py` (no Docker) keeps the sweep plan consistent with the catalog.
|
|
|
|
`AEGIS_IT_WRITE_REPORT=1` makes the session write `docs/integration-coverage.md` (per
|
|
domain: tools with a 2xx, documented non-2xx statuses, allowances).
|
|
|
|
### Actions runner
|
|
|
|
`start_runner` starts `gitea/act_runner` (digest-pinned in `ACT_RUNNER_IMAGE`) on the run's
|
|
network, registered with a token from `gitea_actions_repo_create_runner_registration_token`,
|
|
label `aegis-host:host`. Jobs run inside the runner container itself: no Docker socket, no
|
|
bind mounts. The runner reaches Gitea by container name, so it works in both network modes.
|
|
It is removed with the rest of the run. If it cannot start or come online, the test is
|
|
marked xfail with the reason, unless `CI` or `AEGIS_IT_REQUIRE_RUNNER=1` is set (the CI job
|
|
sets it), in which case it fails: a pipeline must not silently skip the Actions scenario. A
|
|
failure after the run completed is always a real failure.
|
|
|
|
With the runner the scenario also exercises the completed run: the runner get, update and
|
|
delete tools at repository scope, the logs archive, rerun of the run and of one job, and
|
|
deleting the run. `test_runner_management_at_every_scope` registers one more runner for the
|
|
organization, the user and the instance (each serving its own label) and manages it through
|
|
the matching tools.
|
|
|
|
Runtime: about two minutes locally (Gitea start 15 s, sweep 40 s, runner 10 s).
|