Local development

Prerequisites

  • Python 3.9 (.python-version pins 3.9; requires-python = ">=3.9,<3.10").

  • uv for dependency management.

  • Git access to the private pyb-sdk and EventLoggerPySDK repositories — a credential helper or GIT_TOKEN.

  • AWS credentials for the target environment (needed by config_setup.py to read Secrets Manager, and at runtime for DynamoDB/S3/SWF/EventBridge).

Quick start

make setup ENV_NAME=mydev     # install deps + generate settings + fetch secrets
make run-api                  # start the Web API
make run-worker               # start the SWF worker (separate terminal)
make test

ENV_NAME defaults to dev, and the Makefile exports ENV_FOR_DYNACONF to match it — the two must agree or Dynaconf reads the wrong environment block.

Makefile targets

Target Does

help

List the targets (default goal).

install

uv sync — create .venv and install runtime + dev dependencies.

lock

uv lock — regenerate uv.lock from pyproject.toml.

settings

python settings_setup.py → config/settings.json.

secrets

python config_setup.py → config/.secrets.json + Thomson Reuters certificate material (needs AWS credentials).

setup

install + settings + secrets.

run-api

uvicorn web_api_app:app --reload --host $(API_HOST) --port $(API_PORT) (defaults 0.0.0.0:8000).

run-worker

python -u swf_app.py.

test

uv run pytest — the same invocation CI runs.

coverage

test, then point at htmlcov/index.html.

docker-build

Build both images via deployment/docker-compose.yml.

clean

Remove .venv, caches and test/coverage artifacts (keeps generated config).

clean-config

Remove config/settings.json, config/.secrets.json and the TR PEM/PFX material.

Manual bootstrap (without make)

export GIT_TOKEN=<github-pat-with-repo-read>
export ENV_NAME=mydev            # "release" == production
export ENV_FOR_DYNACONF=$ENV_NAME

uv sync                          # or: pip install -r requirements.txt
python settings_setup.py         # → config/settings.json
python config_setup.py           # → config/.secrets.json (+ TR certs); needs AWS creds

uv run uvicorn web_api_app:app --reload --host 0.0.0.0 --port 8000
uv run python -u swf_app.py

pre_requirements.txt is the minimal dependency subset for the two bootstrap scripts when a full install is not wanted.

Dependency management notes

pyproject.toml carries several deliberate constraints — do not "clean them up" without reading the comments there:

  • package = false — this repository is a deployable app, not an installable library.

  • build-constraint-dependencies = ["setuptools<81"] — pyb-sdk is a PyScaffold package whose setup.py imports pkg_resources, removed in setuptools 81.

  • setuptools<81 as a runtime dependency — dynaconf 3.1.8 imports pkg_resources at runtime and uv venvs do not ship setuptools by default.

  • override-dependencies = ["requests==2.31.0"] — pyb-sdk hard-pins requests==2.28.1, which this project overrides graph-wide.

  • [tool.uv.sources] — pybsdk from the develop-py39 branch, eventlogger from GitHub.

requirements.txt is the pip-based equivalent used by the Docker images and CodeBuild; keep it in step with pyproject.toml when you change a runtime dependency.

Running against a dev environment

A dev ENV_NAME gives you: Thomson Reuters and Au10tix sandboxes, dev-payability-datalake, environment-suffixed DynamoDB tables, the environment name as the SWF domain and EventBridge bus, /docs and /redoc enabled, and the /automatic-uw-tool-web-api root path. See Configuration & secrets.

Because the SWF domain is per-environment, a locally running worker polls only its own environment’s task list — two developers on different ENV_NAME values do not steal each other’s activities. Two workers on the same ENV_NAME, however, will.

Local Entrust webhooks

POST /entrust/webhook needs to be reachable by Onfido and signed with ENTRUST_WEBHOOK_TOKEN. For local work either

  • point a tunnel at your port and register it as an Onfido webhook using the same shared secret, or

  • replay a captured delivery yourself, computing the signature over the exact bytes:

BODY='{"payload":{"resource_type":"workflow_run","action":"workflow_run.completed","object":{"id":"<workflow_run_id>"}}}'
SIG=$(printf '%s' "$BODY" | openssl dgst -sha256 -hmac "$ENTRUST_WEBHOOK_TOKEN" | awk '{print $2}')
curl -X POST http://localhost:8000/entrust/webhook \
  -H "Content-Type: application/json" -H "X-SHA2-Signature: $SIG" -d "$BODY"

The workflow_run_id must already exist as a TaskExecutionTable row (create one through POST /entrust/sessions), otherwise the webhook acknowledges and ignores it.

Docker

export ENV_NAME=<env> AWS_ACCOUNT=<acct> GIT_TOKEN=<pat> \
       ROOT_PATH=/automatic-uw-tool-web-api NAME_TAG=automatic-uw-tool
export ENV_FOR_DYNACONF=$ENV_NAME
docker-compose -f deployment/docker-compose.yml build

See Deployment for what the images contain.

Do not commit

config/settings.json, config/.secrets.json, and any .pem / .pfx / .base64 / pfx_password.txt material under config/thomson_reuters/. They are generated and/or secret. make clean-config removes them.