Overview

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usefini.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Create Test Suite regression sets, attach criteria, start runs, and read run results through Fini's public API.

Test sets are the API form of the dashboard [Test Suite](/en/testing/test-suite). A test set groups existing conversations for one agent, attaches one or more grading criteria, and queues asynchronous runs that replay those conversations before judging them.

Use these routes when you want to seed regression checks from conversations already in Fini, manage their criteria from your backend, or trigger a run after changing prompts, knowledge, rules, or actions.

<Info>
  Test set runs evaluate existing conversation IDs. Create or import the conversations first, then pass their IDs in `conversationIds` when you create or update a test set.
</Info>

Test set runs support the same replay modes as [Replays](/en/api-reference/replays): `reuse`, `simulate`, and `execute`. `simulate` is the default because it tests the latest rules while reusing matching saved Action results instead of performing live external calls.

## Endpoints

<CardGroup cols={2}>
  <Card title="List test sets" icon="list" href="/en/api-reference/list-test-sets">
    Return every test set in the workspace, newest first.
  </Card>

  <Card title="Create a test set" icon="plus" href="/en/api-reference/create-test-set">
    Create a set from one to 200 existing conversation IDs.
  </Card>

  <Card title="Get fields context" icon="table-list" href="/en/api-reference/get-test-set-fields-context">
    Read default criteria and deterministic-condition fields.
  </Card>

  <Card title="Get a test set" icon="eye" href="/en/api-reference/get-test-set">
    Fetch a test set with its resolved criteria.
  </Card>

  <Card title="Update a test set" icon="pen" href="/en/api-reference/update-test-set">
    Update the set name, description, or conversation list.
  </Card>

  <Card title="Delete a test set" icon="trash" href="/en/api-reference/delete-test-set">
    Delete a set after active runs finish.
  </Card>

  <Card title="Add criteria" icon="circle-plus" href="/en/api-reference/add-test-set-criteria">
    Attach default or custom criteria to a set.
  </Card>

  <Card title="Update a criterion" icon="pen-to-square" href="/en/api-reference/update-test-set-criterion">
    Change one criterion on a set.
  </Card>

  <Card title="Delete a criterion" icon="circle-minus" href="/en/api-reference/delete-test-set-criterion">
    Remove one criterion from a set.
  </Card>

  <Card title="List runs" icon="bars-progress" href="/en/api-reference/list-test-set-runs">
    Page through run summaries for one set.
  </Card>

  <Card title="Start a run" icon="play" href="/en/api-reference/start-test-set-run">
    Queue an asynchronous evaluation run.
  </Card>

  <Card title="Get a run" icon="chart-simple" href="/en/api-reference/get-test-set-run">
    Fetch one run and its detailed result.
  </Card>
</CardGroup>

## Authentication

<ParamField header="Authorization" type="string" required>
  Bearer token containing your Fini workspace API key. Format: `Bearer fini_...`
</ParamField>

Write routes also require `Content-Type: application/json` when they send a body.

## TestSet object

<ResponseField name="id" type="string">
  Test set ID.
</ResponseField>

<ResponseField name="companyId" type="string">
  Workspace ID that owns the test set. This field is omitted from list summaries.
</ResponseField>

<ResponseField name="name" type="string">
  Test set name.
</ResponseField>

<ResponseField name="description" type="string | null">
  Optional description.
</ResponseField>

<ResponseField name="conversationIds" type="string[]">
  Existing Fini conversation IDs included in the set. The API accepts 1 to 200 UUIDs.
</ResponseField>

<ResponseField name="conversations" type="array">
  Conversation summaries returned by detail responses. Each item includes `id` and `subject`, where `subject` can be `null` when no subject preview is available.
</ResponseField>

<ResponseField name="botId" type="string | null">
  Agent linked to the test set detail. New and updated test sets require all conversations to belong to the same agent; older sets can return `null`.
</ResponseField>

<ResponseField name="createdBy" type="string | null">
  User ID that created the set. API-created records can be `null`.
</ResponseField>

<ResponseField name="createdAt" type="string">
  ISO 8601 creation timestamp.
</ResponseField>

<ResponseField name="updatedAt" type="string">
  ISO 8601 last-update timestamp.
</ResponseField>

List responses return `conversationCount`, `criteriaCount`, `blockingCriteriaCount`, `latestRun`, and `recentRuns` instead of the full `conversationIds` and `conversations` arrays.

## Run snapshot object

Run snapshots appear on list summaries so you can render pass-rate trends without fetching every run detail.

<ResponseField name="runId" type="string">
  Test run ID.
</ResponseField>

<ResponseField name="status" type="string">
  Current run status, such as `running`, `completed`, or `failed`.
</ResponseField>

<ResponseField name="createdAt" type="string">
  ISO 8601 timestamp for when the run started.
</ResponseField>

<ResponseField name="passedConversations" type="number | null">
  Number of conversations that passed. `null` while the run has no completed summary.
</ResponseField>

<ResponseField name="totalConversations" type="number | null">
  Number of conversations evaluated. `null` while the run has no completed summary.
</ResponseField>

<ResponseField name="passRate" type="number | null">
  Rounded pass percentage for the run. `null` when `totalConversations` is missing or zero.
</ResponseField>

## Criterion object

<ResponseField name="id" type="string">
  Criterion ID.
</ResponseField>

<ResponseField name="testSetId" type="string | null">
  Test set that owns the criterion. Default criteria return `null`.
</ResponseField>

<ResponseField name="companyId" type="string | null">
  Workspace that owns the criterion. Fini-provided defaults can return `null`.
</ResponseField>

<ResponseField name="defaultCriterionId" type="string | null">
  Default criterion copied into this test set, if this criterion was created from a default.
</ResponseField>

<ResponseField name="name" type="string">
  Criterion label.
</ResponseField>

<ResponseField name="type" type="string">
  One of `deterministic`, `basic_judge`, or `complex_judge`.
</ResponseField>

<ResponseField name="judgePrompt" type="string | null">
  Judge prompt for LLM-graded criteria.
</ResponseField>

<ResponseField name="passPrompt" type="string | null">
  What should count as a pass for LLM-graded criteria.
</ResponseField>

<ResponseField name="failPrompt" type="string | null">
  What should count as a fail for LLM-graded criteria.
</ResponseField>

<ResponseField name="condition" type="object | null">
  Deterministic condition. Allowed root paths are `knowledgeSearchUsed`, `usedArticles`, `usedKnowledgeFolders`, `intentRules`, `tagGroups`, and `replyTypes`.
</ResponseField>

<ResponseField name="blocking" type="boolean">
  Whether failing this criterion should make the conversation fail overall.
</ResponseField>

<ResponseField name="isActive" type="boolean">
  Whether the criterion is active.
</ResponseField>

## Run object

<ResponseField name="id" type="string">
  Test run ID.
</ResponseField>

<ResponseField name="testSetId" type="string">
  Test set evaluated by the run.
</ResponseField>

<ResponseField name="status" type="string">
  `running`, `completed`, or `failed`.
</ResponseField>

<ResponseField name="result" type="object | null">
  Full result for [Get a run](/en/api-reference/get-test-set-run). While the run is queued or processing, this is `null`.
</ResponseField>

<ResponseField name="createdBy" type="string | null">
  User ID that started the run. API-started runs can be `null`.
</ResponseField>

### Result shape

When a run completes, `result.summary` contains aggregate counts, `result.conversations` contains per-conversation results, and `result.environment` records the replay mode and Behavior version used.

<ResponseField name="result.summary.testSetResult" type="string">
  Overall verdict, `pass` or `fail`.
</ResponseField>

<ResponseField name="result.summary.totalConversations" type="number">
  Number of conversations evaluated.
</ResponseField>

<ResponseField name="result.summary.passedConversations" type="number">
  Conversations with a passing result.
</ResponseField>

<ResponseField name="result.summary.failedConversations" type="number">
  Conversations with a failing result.
</ResponseField>

<ResponseField name="result.summary.errorConversations" type="number">
  Conversations that errored during evaluation.
</ResponseField>

<ResponseField name="result.conversations[].criteriaResults[]" type="array">
  Criterion-level verdicts, reasoning, and evidence for each evaluated conversation.
</ResponseField>

<ResponseField name="result.conversations[].replayInteractionId" type="string | null">
  Replay conversation created for this test scenario. Open this conversation to inspect the exact replay that was judged.
</ResponseField>

<ResponseField name="result.conversations[].botId" type="string | null">
  Agent that handled the evaluated conversation.
</ResponseField>

<ResponseField name="result.conversations[].promptVersionId" type="string | null">
  Behavior version used for this conversation, when the run used a versioned replay environment.
</ResponseField>

<ResponseField name="result.conversations[].promptVersionNumber" type="number | null">
  Human-readable Behavior version number used for this conversation, when available.
</ResponseField>

<ResponseField name="result.conversations[].interactionSubject" type="string | null">
  Subject preview for the evaluated conversation. This can be `null` when the conversation has no subject.
</ResponseField>

<ResponseField name="result.environment.ruleExecutionMode" type="enum">
  Replay mode for the run: `reuse`, `simulate`, or `execute`.
</ResponseField>

<ResponseField name="result.environment.bot" type="object | null">
  Snapshot of the agent environment used for simulated or live-action runs. `reuse` runs return `null` because they reuse historical rule choices and saved results.
</ResponseField>

## Errors

<AccordionGroup>
  <Accordion title="400 Bad Request" icon="circle-exclamation">
    The request body or query parameters failed validation. Common cases: empty names, invalid UUIDs, more than 200 conversation IDs, no criteria before starting a run, or invalid criterion definitions.
  </Accordion>

  <Accordion title="401 Unauthorized" icon="lock">
    The API key is missing, malformed, revoked, or invalid.
  </Accordion>

  <Accordion title="403 Forbidden" icon="shield-halved">
    The API key does not include the required `read` or `write` scope.
  </Accordion>

  <Accordion title="404 Not Found" icon="circle-question">
    The test set, criterion, run, default criterion, or conversation does not exist in your workspace.
  </Accordion>

  <Accordion title="409 Conflict" icon="triangle-exclamation">
    The test set has an active run and cannot be deleted until the run finishes.
  </Accordion>
</AccordionGroup>


## Related topics

- [Overview](/en/api-reference/tags.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.