SWF orchestration

The handle step of every process runs as an AWS Simple Workflow activity. The Web API never calls SWF directly — it goes through the shared B2B Task Manager service.

Start side: Web API → B2B Task Manager → SWF

An InitialTask (or the Entrust webhook handler) calls B2BTaskManager.start_swf(…​). The workflow input embeds the handle task name plus the correlation id:

{
  "platform": "automatic-uw",
  "task_name": "entrust/handle_identity_document_validation_response",
  "task_correlation_id": "b1f7d0c2-3f4e-4a1b-9c8d-2e5f6a7b8c9d",
  "task_input": {
    "supplier_key": "9b1c4d3e-6f2a-4b8c-9d0e-1f2a3b4c5d6e",
    "person_id": "44f2a1b0-7c3d-4e5f-8a9b-0c1d2e3f4a5b",
    "applicant_id": "c3d4e5f6-a7b8-49c0-8d1e-2f3a4b5c6d7e",
    "workflow_run_id": "b1f7d0c2-3f4e-4a1b-9c8d-2e5f6a7b8c9d"
  },
  "gci": "1"
}

start_swf injects gci from settings and posts to POST {B2B_TASK_MANAGER_URL}/executeWorkflow:

def start_swf(self, domain, workflow_type, workflow_type_version, workflow_identifier, input_body):
    input_body.update({'gci': settings.B2B_TASK_MANAGER_GCI})

    response = self.__session.post(
        url='{}/executeWorkflow'.format(settings.B2B_TASK_MANAGER_URL),
        headers={},
        params={
            'domain': domain,
            'workflowTypeName': workflow_type,
            'workflowTypeVersion': workflow_type_version,
            'taskList': 'default',
            'workflowId': workflow_identifier,
            'input': json.dumps(input_body)
        }
    )

A non-2xx raises HTTPError. A 2xx that carries errorMessages or exceptionMessages raises CannotRunWorkflow, whose str() collapses the SWF WorkflowExecutionAlreadyStartedFault into "Task already running". Because CannotRunWorkflow is a TaskExecutionError, task_failure_handling re-raises it untouched and the API answers 400 (or 409 for the Entrust reprocess path) rather than marking the task failed.

Workflow identifiers

The workflowId is the SWF dedup key. The choices in this repository are deliberate:

Trigger workflowId

Au10tix identity document

Automatic UW Tool - {supplier_key} - Validate identity document.

Personal information (KYC)

Automatic UW Tool - {supplier_key} - Validate personal informations.

Personal background report

Automatic UW Tool - {supplier_key} - Download personal background report.

Business background report

Automatic UW Tool - {supplier_key} - Download business background report.

Entrust webhook

Automatic UW Tool - Entrust - {workflow_run_id} — deterministic, so concurrent redeliveries dedup

Entrust manual reprocess

Automatic UW Tool - Entrust - {workflow_run_id} - reprocess - {uuid4} — unique, so the rerun is not deduped against the original

SWF coordinates

Setting Value

SWF_WORKFLOW_TYPE

Update Amazon SC Earnings

SWF_WORKFLOW_TYPE_VERSION

1.1

SWF_DOMAIN

B2B in release, otherwise the environment name

SWF_AUTOMATIC_UW_TOOL_TASK_LIST

automatic-uw-tool-task-list

SWF_APP_THREADS

5

B2B_TASK_MANAGER_GCI

1 in release; a hash of the env name in dev

The workflow type is shared platform-wide (hence the unrelated-sounding name) and its definition lives outside this repository. It schedules an activity onto automatic-uw-tool-task-list, which this repository’s worker polls.

Worker side (swf_app.py)

At startup the worker builds one TaskFactory with every client injected, one SWFClient, and SWF_APP_THREADS threads (named with coolname slugs) all running the same loop:

while True:
    activity_task = swf_client.poll_activity_task(
        domain=settings.SWF_DOMAIN,
        identity=settings.SWF_WORKFLOW_TYPE,
        task_list=settings.SWF_AUTOMATIC_UW_TOOL_TASK_LIST
    )

    task = None
    task_token = activity_task['taskToken']
    process_start_time = time.time()
    try:
        task_definition = json.loads(activity_task['input'])
        task = task_factory.build(task_definition)
        task_input = task.build_task_input(task_definition.get('task_input', {}))
        execution_output = task.execute(task_input)
    except Exception as exception:
        sentry_sdk.capture_exception()
        swf_client.notify_about_task_failure(task_token=task_token, reason=exception)
        if task:
            send_metric(..., name=f'{task.get_process_name()}_failure', value=1.00, unit=CloudWatchMetricUnit.NONE)
    else:
        swf_client.notify_about_task_success(task_token=task_token, result=execution_output.to_dict())
        send_metric(..., name=f'{task.get_process_name()}_success_time_taken', value=round_time_diff(...))

SWFClient.poll_activity_task loops until a task token appears, swallowing ReadTimeoutError and ConnectionClosedError (long-poll timeouts are normal).

Reporting failure safely

RespondActivityTaskFailed caps reason at 256 characters and details at 32768. An over-long reason makes the API call itself raise, which would leave the activity un-failed until its start-to-close timeout. SWFClient therefore derives a short reason (error → error_message → error_type → the raw payload) and puts the full JSON payload in details, truncating both.

An exception raised before task_factory.build succeeds (bad JSON, unknown task_name) means task is None, so the failure metric is skipped — the failure is still reported to SWF and Sentry.

Metrics emitted here

  • {process_name}_failure — count, on any exception.

  • {process_name}_success_time_taken — seconds, on success.

process_name comes from task.get_process_name(), so all Entrust and Au10tix activity is reported under validate_identity_document. See Observability.

Idempotency expectations

SWF may deliver an activity more than once, and Onfido may redeliver a webhook. Handle tasks must therefore tolerate replay:

  • Entrust: an already-processed row short-circuits; documents_uploaded / documents_approved guard the document mutations; FinalStatusTable writes are upserts.

  • Thomson Reuters and Au10tix tasks reload state by correlation_id and overwrite their own row, so a replay re-does vendor work but converges on the same result — with the caveat that it also re-emits ProcessFinishedEvent and appends a new FinalStatusTable version.

New async work must follow the idempotency rule.