Overview

> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usefini.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Overview

> Add raw content to Fini, ingest it into source records, and move it into the knowledge workflow.

Sources are the raw inputs that feed Fini's knowledge system. Use these routes to bring content into Fini from the web or connected providers, monitor ingestion, and refresh or remove source records later.

<Info>
  The wire format still uses `document`, `documentId`, and `/v2/documents/...` because that is the current API contract. In this reference, we call them **sources** because they are inputs to knowledge, not the knowledge your agents retrieve from directly.
</Info>

A source becomes useful in Fini when you ingest it, optionally set up your [Knowledge](/en/api-reference/knowledge) structure for first-time imports, generate knowledge from the ingested content, review the result, and then organize and assign that knowledge to the right agent.

## Reference pages

<CardGroup cols={3}>
  <Card title="List sources" icon="list" href="/en/api-reference/list-sources">
    `GET /v2/documents/public` — paginate through every source record in the workspace.
  </Card>

  <Card title="Get source" icon="file" href="/en/api-reference/get-source">
    `GET /v2/documents/public/:id` — fetch one source record by ID.
  </Card>

  <Card title="List provider resources" icon="folder-magnifying-glass" href="/en/api-reference/list-provider-resources">
    `GET /v2/documents/public/resources/:provider` — discover what is importable from a connected provider.
  </Card>

  <Card title="Register provider resources" icon="folder-plus" href="/en/api-reference/register-provider-resources">
    `POST /v2/documents/public/resources/:provider` — create source records for the resources you picked.
  </Card>

  <Card title="Ingest sources" icon="play" href="/en/api-reference/ingest-sources">
    `POST /v2/documents/public` — queue ingestion or refresh jobs.
  </Card>

  <Card title="Crawl links" icon="magnifying-glass" href="/en/api-reference/deep-crawl-links">
    `POST /v2/documents/public/deep-crawl/links` — discover more web URLs from one or more seed links.
  </Card>

  <Card title="Refresh sources" icon="arrows-rotate" href="/en/api-reference/refresh-sources">
    `POST /v2/documents/public/refresh` — refresh existing source IDs.
  </Card>

  <Card title="Delete sources" icon="trash" href="/en/api-reference/delete-sources">
    `DELETE /v2/documents/public` — delete source records, optionally deleting linked Articles too.
  </Card>

  <Card title="Knowledge" icon="wand-magic-sparkles" href="/en/api-reference/knowledge">
    Generate knowledge from sources, scaffold the first tree, and move from drafts into live knowledge.
  </Card>

  <Card title="Manage knowledge" icon="book-open" href="/en/api-reference/manage-knowledge">
    Read, create, update, draft, publish, and delete live articles directly.
  </Card>

  <Card title="Organize knowledge" icon="folder" href="/en/api-reference/organize-knowledge">
    Manage folders, move articles between folders, and assign them to agents.
  </Card>
</CardGroup>

The three `GET` routes require `read`. Resource registration, ingestion, crawl, refresh, and delete routes require `write`.

## Endpoint map

| Method | Path | Scope | Purpose |
| - | - | - | - |
| `GET` | `/v2/documents/public` | `read` | List source records in the workspace. |
| `GET` | `/v2/documents/public/:id` | `read` | Fetch one source record by ID. |
| `GET` | `/v2/documents/public/resources/:provider` | `read` | Discover importable content from a connected provider. |
| `POST` | `/v2/documents/public/resources/:provider` | `write` | Register provider resources as source records. |
| `POST` | `/v2/documents/public` | `write` | Ingest new sources or refresh existing ones through the add-sources route. |
| `POST` | `/v2/documents/public/deep-crawl/links` | `write` | Discover more web URLs from one or more seed links before ingesting them. |
| `POST` | `/v2/documents/public/refresh` | `write` | Refresh existing source IDs. |
| `DELETE` | `/v2/documents/public` | `write` | Delete source records, optionally deleting linked Articles too. |

## Add sources

There are two ways to add sources into Fini through the public API:

* Web links: pass URLs directly to [Ingest sources](/en/api-reference/ingest-sources).
* Connected providers: discover provider resources, register them, then ingest the returned source IDs.

Currently supported connected providers in the public API are `zendesk`, `confluence`, and `notion`.

| Source type | How to add it | What the discovery response looks like |
| - | - | - |
| `web` | Call [Ingest sources](/en/api-reference/ingest-sources) directly with URLs in `documentIdsToAdd` | No discovery call. You can optionally crawl first. |
| `zendesk` | Call [List provider resources](/en/api-reference/list-provider-resources), then [Register provider resources](/en/api-reference/register-provider-resources), then [Ingest sources](/en/api-reference/ingest-sources) | Category tree with nested sections and articles |
| `confluence` | Call [List provider resources](/en/api-reference/list-provider-resources), then [Register provider resources](/en/api-reference/register-provider-resources), then [Ingest sources](/en/api-reference/ingest-sources) | Array of spaces with nested pages |
| `notion` | Call [List provider resources](/en/api-reference/list-provider-resources), then [Register provider resources](/en/api-reference/register-provider-resources), then [Ingest sources](/en/api-reference/ingest-sources) | Flat array of importable resources |

Provider-specific response objects and full examples live on [List provider resources](/en/api-reference/list-provider-resources). That page owns the actual provider response shapes: `NotionResource[]`, `ZendeskCategory[]`, and `ConfluenceSpace[]`.

### Optional: crawl web links before ingesting

Crawling is useful when you have one or a few seed URLs and want Fini to discover more web pages before you ingest them. It is a separate API call, not behavior inside `POST /v2/documents/public`.

<Steps>
  <Step title="Crawl the seed URLs">
    Call [Crawl links](/en/api-reference/deep-crawl-links). The response returns discovered URLs in `data`.
  </Step>

  <Step title="Pick the URLs you want to ingest">
    Use the returned URLs as the input set for your web-source ingestion flow.
  </Step>

  <Step title="Ingest them with the add-sources endpoint">
    Call [Ingest sources](/en/api-reference/ingest-sources) with `source: "web"` and place those URLs into `documentIdsToAdd`.
  </Step>
</Steps>

<Note>
  The endpoint map above is the full public route surface for Sources. Use the child pages in this section for the detailed request and response references for each route family.
</Note>

## Refresh knowledge from changed sources

If you already have knowledge in Fini for a set of sources and the upstream content changes, the refresh path is: refresh the sources, filter the ones that actually changed, then bulk generate and save knowledge for just those source IDs.

<Steps>
  <Step title="Refresh the existing source IDs">
    Call [Refresh sources](/en/api-reference/refresh-sources) with the source IDs you already have. For web content, those are the source records created the first time you ingested the URLs.
  </Step>

  <Step title="Poll until the refresh completes">
    Use [List sources](/en/api-reference/list-sources) or [Get source](/en/api-reference/get-source) and wait for `linkedJobStatus` to move to `COMPLETED`.
  </Step>

  <Step title="Filter to only the sources whose content changed">
    Call [List sources](/en/api-reference/list-sources) with `changed=true`. If you want only web content, also pass `source=web`. Keep the source records that already have a `linkedKnowledgeId` because those are the ones that already back knowledge in Fini.
  </Step>

  <Step title="Bulk generate and save knowledge for those changed source IDs">
    Call [Bulk generate knowledge](/en/api-reference/bulk-generate-knowledge) with the changed `documentIds`. Set `isDraft: true` if you want review before the update goes live. Set `isDraft: false` only when you want the update saved live immediately.
  </Step>

  <Step title="Poll the knowledge jobs, then review if needed">
    Use [Check knowledge jobs](/en/api-reference/check-knowledge-jobs) until the queued jobs finish. If you kept the updates as drafts, review and publish them before expecting agent answers to change.
  </Step>
</Steps>

<Info>
  When the generate-and-save job succeeds for a source-backed update, Fini syncs the linked article and clears that source's `changed` flag.
</Info>

## Why the Sources API isn't working

<AccordionGroup>
  <Accordion title="The source record exists but nothing shows up in agent answers" icon="circle-question">
    This is usually expected. Source registration and ingestion only get the raw content into Fini. You still need to create knowledge from that source, and the recommended path is to review the result before it goes live.
  </Accordion>

  <Accordion title="The ingestion job stays PENDING" icon="clock">
    Ingestion is asynchronous. Poll [List sources](/en/api-reference/list-sources) or [Get source](/en/api-reference/get-source) and watch `linkedJobStatus`, `success`, and `error`. If a source stays `PENDING` for longer than expected, inspect `error` on the source record.
  </Accordion>

  <Accordion title="A web source import fails immediately" icon="link-slash">
    When `source` is `web`, the values in `documentIdsToAdd` and `documentIdsToRefresh` must be URLs, not source IDs. The wire format reuses the same field names across sources, which is easy to miss.
  </Accordion>

  <Accordion title="A provider-resources call returns 400" icon="triangle-exclamation">
    The provider isn't valid for the public route, or it's not connected in the workspace yet. The public provider routes accept `notion`, `zendesk`, and `confluence`. Connect the provider in the dashboard first.
  </Accordion>

  <Accordion title="Provider content I expected is missing from discovery" icon="eye-slash">
    Resources that have already been imported successfully from that provider are excluded from the response. Discovery is for finding new content, not auditing what is already in Fini. Use [List sources](/en/api-reference/list-sources) with the provider as a `source` filter to see what is already imported.
  </Accordion>

  <Accordion title="A crawl or refresh request returns 400" icon="circle-exclamation">
    Common causes are a malformed body, an empty required array, or trying to refresh a raw-file source. Web crawl expects seed URLs in `links`, and refresh expects existing source IDs in `documentIds`.
  </Accordion>

  <Accordion title="Delete or refresh fails with a server error" icon="triangle-exclamation">
    Retry once, then inspect the affected source IDs in [List sources](/en/api-reference/list-sources). If the problem persists, the workspace may contain one or more invalid or inaccessible records that need support help.
  </Accordion>
</AccordionGroup>


## Related topics

- [Overview](/en/api-reference/tags.md)


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.