Call analysis

Every connected call is graded by an LLM against the caller agent’s own script. The output is a disposition verdict, a 1–10 agent score, a written summary and a list of issues.

When analysis runs

Trigger Path

A campaign call ends

call-target/call.ended → callEndedAnalysis step run-ai-analysis

A Nooks call is logged

nooks/call.logged → nooksCallAnalysis

An operator re-runs one call

POST /api/calls/{id}/analyze

An operator re-runs many calls

POST /api/calls/batch-analyze

When it is skipped

Analysis is skipped when the technical disconnection reason is in BLACKLISTED_AI_ANALYSIS_REASONS — the call never reached a human, so there is nothing to grade:

voicemail_reached, dial_busy, dial_no_answer, dial_failed,
telephony_provider_unavailable, user_declined, invalid_destination,
telephony_provider_permission_denied

For Nooks calls the equivalent set is NOOKS_ANALYSIS_SKIP_DISPOSITIONS: "No Answer", "Voicemail", "Busy".

Because voicemail_reached, machine_detected, dial_no_answer and user_declined also appear in DOUBLE_TAP_REASONS, an eligible double tap returns before reaching the analysis step at all.

Transient infrastructure failures — messages containing EBUSY, ENOTFOUND, Cannot connect to API or Insufficient funds — are swallowed and returned as { saved: false, error }. Anything else re-throws, so a genuine bug surfaces in Sentry rather than silently dropping analyses.

Agent selection

CallAnalysisService.runAndSave(callId, campaignId):

  1. Load the call with its campaign. When campaignId is provided it must match the call’s own campaign, otherwise the run is rejected.

  2. Choose the analysis agent:

    1. the campaign’s preferredAiAgentId, if set (getPreferredAgentIdForCampaign);

    2. otherwise the workspace’s default analysis agent (getDefaultAnalysisAgentWithVersion);

    3. no agent → { saved: false, error: "No analysis AI agent configured" }.

  3. Reject the run if the resolved agent’s type is not analysis.

  4. Resolve the caller agent’s prompt for context: prefer Call.agentVersionId (the exact version live during the call), else the latest version of Call.agentId.

Using the exact caller version matters: an analysis run months later must grade the call against the script that was actually spoken, not the current one.

For human (Nooks) calls there is no caller agent, so DEFAULT_HUMAN_CALL_ANALYSIS_SYSTEM_PROMPT from lib/constants.ts is used instead. It reframes the task as sales-coaching: verify the logged disposition, assess objection handling and rapport, and flag compliance or tone issues.

Prompt composition

Part Source

System prompt

The caller agent’s mainPrompt (the script and persona) plus the DEFAULT_POST_CALL_ANALYSIS schema from lib/retellai/post-call-analysis.ts, so the model knows the field vocabulary

Criteria

The analysis agent’s mainPrompt

Supplementary config

The analysis agent version’s configJson

Model

AiAgent.modelId (a Vercel AI Gateway slug), falling back to the llmModelType default (gpt → openai/gpt-4o, claude → anthropic/claude-sonnet-4)

Call facts supplied to the model

transcript, summary, disposition, retell_disposition, duration_seconds, duration_ms, user_sentiment, in_voicemail, call_screening_detected, ai_suspicion_detected, detected_language, disconnection_reason, target_phone_number, campaign_name.

Output

interface CallAnalysisOutput {
  correct_disposition: boolean
  suggested_disposition?: string        // present when correct_disposition is false
  agent_score: number                   // 1–10, how well the agent handled the conversation
  summary: string
  issues: Array<{
    name: string
    priority: "critical" | "high" | "low"
    description: string
  }>
}

Persisted to CallAiAgentAnalysisResult, unique on (callId, aiAgentId, aiAgentVersionId). The same call can therefore hold one result per agent version — re-running with a new prompt version adds a row, while re-running the same version overwrites it.

Batch re-analysis

Used after changing a prompt, adding an analysis agent, or backfilling a date range.

Filter Query / body key Applied to

Campaign

campaign_id

Call.campaignId

Status

status

Call.status

Date range

date_from, date_to

Call.createdAt (inclusive; date_to is end of day)

HubSpot sync

hubspot_sync_status

Call.hubspotSyncStatus (synced, failed, skipped); omit or all for no filter

All filters are optional; empty or "all" values are not applied.

Flow:

  1. GET /api/calls/batch-analyze returns a preview count, so the operator sees the blast radius before committing.

  2. POST /api/calls/batch-analyze starts batchCallAnalysisWorkflow(filters) fire-and-forget and returns immediately.

  3. Step getCallsToAnalyzeStep calls CallRepository.findCallsForBatchAnalysis(filters), which returns { id, campaignId }[] ordered by createdAt descending. The workflow never touches Prisma directly.

  4. Step runCallAnalysisAndSaveStep runs CallAnalysisService.runAndSave(callId, campaignId) per call — the same code path as per-call analysis, so there is no second implementation to drift.

findCallsForBatchAnalysis and countCallsForBatchAnalysis share the private buildBatchAnalysisWhere(filters) helper, so the preview count and the executed set can never disagree.

The dialog (components/campaign/calls/batch-re-analyze-calls-dialog.tsx) appears on the Calls page and on the campaign Calls tab; on the campaign tab the campaign filter is fixed and hidden. Hooks are useBatchAnalyzePreview(filters, enabled) and useStartBatchAnalyze() in lib/hooks/use-batch-call-analyze.ts.

Reviewing results

/agents/analysis lists every CallAiAgentAnalysisResult for the workspace, backed by GET /api/ai-agents/analysis.

  • Columns — call phone, campaign, AI agent, analysis created, call created, score, disposition, issue count, summary, and a link to the call.

  • Filters — dateFrom / dateTo (on the analysis createdAt, not the call’s), campaignId, aiAgentId, hasIssues.

  • Export — CSV of the current page.

Two distinct timestamps are surfaced because they answer different questions: "when did this conversation happen" versus "when did we last grade it".

/agents/issues (GET /api/ai-agents/issues) flattens the issues arrays into a searchable list with name, priority, agentName, agentVersion and date filters — the entry point for finding systemic prompt problems. POST /api/ai-agents/issues/suggest-prompt asks an LLM (lib/ai/services/prompt-suggestion-service.ts) for a prompt revision that would address a selected issue.