Testing

Suites

Suite Path Runs

Unit

development/unit_tests/

In CI on every build. Mirrors the task/, components/ and interface/ trees. Mockito for mocking.

Integration

development/integration_tests/

Manually. Hits real vendor sandboxes — currently the Thomson Reuters CLEAR client.

End-to-end

development/e2e_tests/

Manually. Drives a deployed environment through its HTTP API and asserts on the resulting DynamoDB/S3 state.

Never run the integration or e2e suites in CI or unattended: they call live vendor sandboxes and deployed environments, consume vendor quota, and write real rows.

Running the unit tests

make test            # uv run pytest
python -m pytest     # identical; this is what CodeBuild runs

pytest.ini supplies the arguments:

[pytest]
addopts = development/unit_tests --cov=. --junitxml=report.xml --cov-report term --cov-report xml --cov-report html

So a bare pytest always targets development/unit_tests and produces report.xml (JUnit), coverage.xml (Cobertura) and htmlcov/ (browse htmlcov/index.html, or make coverage).

Coverage gate

[tool.coverage.report] fail_under = 80 — the build fails below 80%. Branch coverage is on.

Deliberately omitted from measurement ([tool.coverage.run] omit): development/, interface/, config_setup.py, settings_setup.py, swf_app.py, web_api_app.py, components/swf_client.py, the four DynamoDB table clients, components/auth/ and utils/.

interface/** being omitted means endpoint-handler tests improve confidence but not the coverage number — and that the gate is effectively about task/ and components/. Write the handler tests anyway: development/unit_tests/interface_/ exists precisely for the Entrust webhook, session and reprocess paths.

Layout and naming

Unit tests mirror the source tree with a trailing underscore on the top-level package to avoid import clashes:

development/unit_tests/
├── components_/
│   ├── entrust/test_client.py
│   ├── test_hubspot_client.py
│   ├── test_payability_internal_api.py
│   └── test_swf_client.py
├── interface_/web_api/endpoint_handling/
│   ├── test_create_entrust_session_hubspot.py
│   ├── test_execute_task_entrust_reprocess.py
│   └── test_handle_entrust_webhook.py
├── task_/
│   ├── au10tix/<task_name>/test_implementation.py
│   ├── entrust/<task_name>/test_implementation.py
│   ├── thomson_reuters/<task_name>/test_implementation.py
│   ├── test_process_factory.py
│   └── test_task_factory.py
└── utils_/
    ├── test_dict_utils.py
    └── test_string_utils.py

Every directory needs an init.py — the suite relies on package imports.

What to test

  • A new task — one test per branch of execute(): happy path, each vendor failure, and the final-status/tool/event writes. Assert on the injected mocks, not on AWS.

  • Every new typed model — edge parsing: a bad payload must produce a clear error.

  • Idempotency, explicitly — deliver the same webhook or activity twice and assert a single side effect. The Entrust task’s guards exist because of this rule.

  • Input validators — one case per rejected field, asserting the incorrect_fields message.

  • Endpoint handlers — auth/source/permission branches and the status codes they produce.

Fixtures and sample data

File Use

examples/thomson_reuters_personal_background_check_response.json

A recorded CLEAR personal report — also loaded by the handle task’s test=true mode.

examples/entrust_successfull_workflow_run.json

A recorded Onfido workflow run.

task/au10tix/handle_identity_document_validation_response/result.mock.json

A mocked Au10tix result.

development/e2e_tests/sample_identity_document.jpg, sample_fake_identity_document.jpg

Images for the e2e identity-document runs.

Mocking style

The suite uses mockito (when, verify, mock), matching the constructor-injection design: build the task with mocked clients, run execute(), verify the calls. Because TaskFactory injects everything, no monkeypatching of module globals is needed — if a test needs that, the production code probably has a hidden dependency.

CI reporting

CodeBuild publishes report.xml as a JUnit report and coverage.xml as a Cobertura report, so failures and coverage trends are visible in the CodeBuild console. See Deployment.