> ## Documentation Index
> Fetch the complete documentation index at: https://novita.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Tavily Crawl

Tavily is a web data retrieval API designed for developers and AI applications. This document describes the Tavily passthrough endpoints exposed by the platform gateway. It is based on Tavily's Crawl API with platform route prefixes applied.

Base URL example: `https://api.novita.ai`

Crawl a site starting from a root URL, following links to retrieve content from multiple pages.

## Request Headers

All endpoints require platform API authentication.

<ParamField header="Content-Type" type="string" required>
  Use `application/json`.
</ParamField>

<ParamField header="Authorization" type="string" required>
  Platform API key, formatted as `Bearer <api_key>`.
</ParamField>

## Request Body

<ParamField body="url" type="string" required>
  Crawl root URL.
</ParamField>

<ParamField body="instructions" type="string" required={false}>
  Crawler instructions in natural language.
</ParamField>

<ParamField body="chunks_per_source" type="integer" required={false}>
  Max number of content snippets per source to return. Default is `3`; range is `1` to `5`. Available only when `instructions` are set.
</ParamField>

<ParamField body="max_depth" type="integer" required={false}>
  Crawler depth limit. Default is `1`; range is `1` to `5`.
</ParamField>

<ParamField body="max_breadth" type="integer" required={false}>
  Max links per page. Default is `20`; range is `1` to `500`.
</ParamField>

<ParamField body="limit" type="integer" required={false}>
  Link processing limit for crawler. Default is `50`; minimum is `1`.
</ParamField>

<ParamField body="select_paths" type="string[]" required={false}>
  Restrict URL paths matching these regex patterns.
</ParamField>

<ParamField body="select_domains" type="string[]" required={false}>
  Restrict domains matching these regex patterns.
</ParamField>

<ParamField body="exclude_paths" type="string[]" required={false}>
  Exclude URL paths matching these regex patterns.
</ParamField>

<ParamField body="exclude_domains" type="string[]" required={false}>
  Exclude domains matching these regex patterns.
</ParamField>

<ParamField body="allow_external" type="boolean" required={false}>
  Include external domain links.
</ParamField>

<ParamField body="include_images" type="boolean" required={false}>
  Returns a list of images along with the response.
</ParamField>

<ParamField body="extract_depth" type="string" required={false}>
  Extraction mode. Known values include `basic` and `advanced`. `basic` is set as the default value.
</ParamField>

<ParamField body="format" type="string" required={false}>
  Output format of extracted content. Common values include `markdown` and `text`. `markdown` is set as the default value.
</ParamField>

<ParamField body="include_favicon" type="boolean" required={false}>
  Include favicon URL from result.
</ParamField>

<ParamField body="timeout" type="float" required={false}>
  Crawl operation time limit in seconds. Default is `150`; range is `10` to `150`.
</ParamField>

## Request Example

```bash theme={"system"}
curl -s -X POST 'https://api.novita.ai/v3/tavily/crawl' \
  -H 'Authorization: Bearer sk-123' \
  -H 'Content-Type: application/json' \
  -d '{
    "url": "https://docs.tavily.com",
    "chunks_per_source": 3,
    "extract_depth": "basic",
    "format": "markdown",
    "max_depth": 1,
    "max_breadth": 5,
    "limit": 5,
    "allow_external": false,
    "include_images": false,
    "timeout": 120
  }'
```

## Response

<ResponseField name="base_url" type="string" required={false}>
  Crawl base URL.
</ResponseField>

<ResponseField name="results" type="object[]" required={false}>
  Extracted content from crawled URLs. See [Result Object](#result-object).
</ResponseField>

<ResponseField name="response_time" type="float" required={false}>
  Request duration in seconds.
</ResponseField>

<ResponseField name="request_id" type="string" required={false}>
  Unique request identifier.
</ResponseField>

### Result Object

<ResponseField name="url" type="string" required={false}>
  Crawled URL.
</ResponseField>

<ResponseField name="raw_content" type="string" required={false}>
  Full page content.
</ResponseField>

<ResponseField name="favicon" type="string" required={false}>
  Site favicon URL. Returned if `include_favicon` is `true`.
</ResponseField>

## Errors

The platform may return standard HTTP errors before forwarding the request, and Tavily may return upstream errors after forwarding.

<ResponseField name="400" type="status" required={false}>
  Invalid request body or unsupported parameter value.
</ResponseField>

<ResponseField name="401" type="status" required={false}>
  Missing or invalid API key.
</ResponseField>

<ResponseField name="403" type="status" required={false}>
  Access denied by platform or upstream provider, or the URL is not supported.
</ResponseField>

<ResponseField name="404" type="status" required={false}>
  Route or requested resource not found.
</ResponseField>

<ResponseField name="429" type="status" required={false}>
  Rate limit exceeded.
</ResponseField>

<ResponseField name="500" type="status" required={false}>
  Internal server error.
</ResponseField>

<ResponseField name="502" type="status" required={false}>
  Upstream provider error.
</ResponseField>

<ResponseField name="503" type="status" required={false}>
  Service unavailable.
</ResponseField>

## Notes

* All request bodies are JSON.
* Extra Tavily parameters not listed here may be passed through.
* Response shapes can vary depending on request options.
* This document intentionally omits billing-related fields.

## References

For more details, see the [Tavily Crawl API reference](https://docs.tavily.com/documentation/api-reference/endpoint/crawl).