# Tavily Crawl - Documentation

> For the complete documentation index, see [llms.txt](/llms.txt). Markdown is available with `Accept: text/markdown` and `.md` URL variants.

Source: /docs/api-reference/model-apis-tavily-crawl

# Tavily Crawl

POST

/

v3

/

tavily

/

crawl

Try it

Tavily Crawl

cURL

```
curl --request POST \
--url https://api.novita.ai/v3/tavily/crawl \
--header 'Authorization: <authorization>' \
--header 'Content-Type: <content-type>' \
--data '
{
"url": "<string>",
"instructions": "<string>",
"chunks_per_source": 123,
"max_depth": 123,
"max_breadth": 123,
"limit": 123,
"select_paths": [
"<string>"
],
"select_domains": [
"<string>"
],
"exclude_paths": [
"<string>"
],
"exclude_domains": [
"<string>"
],
"allow_external": true,
"include_images": true,
"extract_depth": "<string>",
"format": "<string>",
"include_favicon": true,
"timeout": 123
}
'
```

```
import requestsurl = "https://api.novita.ai/v3/tavily/crawl"payload = { "url": "<string>", "instructions": "<string>", "chunks_per_source": 123, "max_depth": 123, "max_breadth": 123, "limit": 123, "select_paths": ["<string>"], "select_domains": ["<string>"], "exclude_paths": ["<string>"], "exclude_domains": ["<string>"], "allow_external": True, "include_images": True, "extract_depth": "<string>", "format": "<string>", "include_favicon": True, "timeout": 123}headers = { "Content-Type": "<content-type>", "Authorization": "<authorization>"}response = requests.post(url, json=payload, headers=headers)print(response.text)
```

```
const options = { method: 'POST', headers: {'Content-Type': '<content-type>', Authorization: '<authorization>'}, body: JSON.stringify({ url: '<string>', instructions: '<string>', chunks_per_source: 123, max_depth: 123, max_breadth: 123, limit: 123, select_paths: ['<string>'], select_domains: ['<string>'], exclude_paths: ['<string>'], exclude_domains: ['<string>'], allow_external: true, include_images: true, extract_depth: '<string>', format: '<string>', include_favicon: true, timeout: 123 })};fetch('https://api.novita.ai/v3/tavily/crawl', options) .then(res => res.json()) .then(res => console.log(res)) .catch(err => console.error(err));
```

```
<?php$curl = curl_init();curl_setopt_array($curl, [ CURLOPT_URL => "https://api.novita.ai/v3/tavily/crawl", CURLOPT_RETURNTRANSFER => true, CURLOPT_ENCODING => "", CURLOPT_MAXREDIRS => 10, CURLOPT_TIMEOUT => 30, CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1, CURLOPT_CUSTOMREQUEST => "POST", CURLOPT_POSTFIELDS => json_encode([ 'url' => '<string>', 'instructions' => '<string>', 'chunks_per_source' => 123, 'max_depth' => 123, 'max_breadth' => 123, 'limit' => 123, 'select_paths' => [ '<string>' ], 'select_domains' => [ '<string>' ], 'exclude_paths' => [ '<string>' ], 'exclude_domains' => [ '<string>' ], 'allow_external' => true, 'include_images' => true, 'extract_depth' => '<string>', 'format' => '<string>', 'include_favicon' => true, 'timeout' => 123 ]), CURLOPT_HTTPHEADER => [ "Authorization: <authorization>", "Content-Type: <content-type>" ],]);$response = curl_exec($curl);$err = curl_error($curl);curl_close($curl);if ($err) { echo "cURL Error #:" . $err;} else { echo $response;}
```

```
package mainimport (	"fmt"	"strings"	"net/http"	"io")func main() {	url := "https://api.novita.ai/v3/tavily/crawl"	payload := strings.NewReader("{\n \"url\": \"<string>\",\n \"instructions\": \"<string>\",\n \"chunks_per_source\": 123,\n \"max_depth\": 123,\n \"max_breadth\": 123,\n \"limit\": 123,\n \"select_paths\": [\n \"<string>\"\n ],\n \"select_domains\": [\n \"<string>\"\n ],\n \"exclude_paths\": [\n \"<string>\"\n ],\n \"exclude_domains\": [\n \"<string>\"\n ],\n \"allow_external\": true,\n \"include_images\": true,\n \"extract_depth\": \"<string>\",\n \"format\": \"<string>\",\n \"include_favicon\": true,\n \"timeout\": 123\n}")	req, _ := http.NewRequest("POST", url, payload)	req.Header.Add("Content-Type", "<content-type>")	req.Header.Add("Authorization", "<authorization>")	res, _ := http.DefaultClient.Do(req)	defer res.Body.Close()	body, _ := io.ReadAll(res.Body)	fmt.Println(string(body))}
```

```
HttpResponse<String> response = Unirest.post("https://api.novita.ai/v3/tavily/crawl") .header("Content-Type", "<content-type>") .header("Authorization", "<authorization>") .body("{\n \"url\": \"<string>\",\n \"instructions\": \"<string>\",\n \"chunks_per_source\": 123,\n \"max_depth\": 123,\n \"max_breadth\": 123,\n \"limit\": 123,\n \"select_paths\": [\n \"<string>\"\n ],\n \"select_domains\": [\n \"<string>\"\n ],\n \"exclude_paths\": [\n \"<string>\"\n ],\n \"exclude_domains\": [\n \"<string>\"\n ],\n \"allow_external\": true,\n \"include_images\": true,\n \"extract_depth\": \"<string>\",\n \"format\": \"<string>\",\n \"include_favicon\": true,\n \"timeout\": 123\n}") .asString();
```

```
require 'uri'require 'net/http'url = URI("https://api.novita.ai/v3/tavily/crawl")http = Net::HTTP.new(url.host, url.port)http.use_ssl = truerequest = Net::HTTP::Post.new(url)request["Content-Type"] = '<content-type>'request["Authorization"] = '<authorization>'request.body = "{\n \"url\": \"<string>\",\n \"instructions\": \"<string>\",\n \"chunks_per_source\": 123,\n \"max_depth\": 123,\n \"max_breadth\": 123,\n \"limit\": 123,\n \"select_paths\": [\n \"<string>\"\n ],\n \"select_domains\": [\n \"<string>\"\n ],\n \"exclude_paths\": [\n \"<string>\"\n ],\n \"exclude_domains\": [\n \"<string>\"\n ],\n \"allow_external\": true,\n \"include_images\": true,\n \"extract_depth\": \"<string>\",\n \"format\": \"<string>\",\n \"include_favicon\": true,\n \"timeout\": 123\n}"response = http.request(request)puts response.read_body
```

200

```
{
"400": {},
"401": {},
"403": {},
"404": {},
"429": {},
"500": {},
"502": {},
"503": {},
"base_url": "<string>",
"results": [
{}
],
"response_time": 123,
"request_id": "<string>",
"url": "<string>",
"raw_content": "<string>",
"favicon": "<string>"
}
```

Tavily is a web data retrieval API designed for developers and AI applications. This document describes the Tavily passthrough endpoints exposed by the platform gateway. It is based on Tavily’s Crawl API with platform route prefixes applied.
Base URL example: `https://api.novita.ai`
Crawl a site starting from a root URL, following links to retrieve content from multiple pages.

##

[​](#request-headers)

Request Headers

All endpoints require platform API authentication.

[​](#param-content-type)

Content-Type

string

required

Use `application/json`.

[​](#param-authorization)

Authorization

string

required

Platform API key, formatted as `Bearer <api_key>`.

##

[​](#request-body)

Request Body

[​](#param-url)

url

string

required

Crawl root URL.

[​](#param-instructions)

instructions

string

Crawler instructions in natural language.

[​](#param-chunks-per-source)

chunks_per_source

integer

Max number of content snippets per source to return. Default is `3`; range is `1` to `5`. Available only when `instructions` are set.

[​](#param-max-depth)

max_depth

integer

Crawler depth limit. Default is `1`; range is `1` to `5`.

[​](#param-max-breadth)

max_breadth

integer

Max links per page. Default is `20`; range is `1` to `500`.

[​](#param-limit)

limit

integer

Link processing limit for crawler. Default is `50`; minimum is `1`.

[​](#param-select-paths)

select_paths

string[]

Restrict URL paths matching these regex patterns.

[​](#param-select-domains)

select_domains

string[]

Restrict domains matching these regex patterns.

[​](#param-exclude-paths)

exclude_paths

string[]

Exclude URL paths matching these regex patterns.

[​](#param-exclude-domains)

exclude_domains

string[]

Exclude domains matching these regex patterns.

[​](#param-allow-external)

allow_external

boolean

Include external domain links.

[​](#param-include-images)

include_images

boolean

Returns a list of images along with the response.

[​](#param-extract-depth)

extract_depth

string

Extraction mode. Known values include `basic` and `advanced`. `basic` is set as the default value.

[​](#param-format)

format

string

Output format of extracted content. Common values include `markdown` and `text`. `markdown` is set as the default value.

[​](#param-include-favicon)

include_favicon

boolean

Include favicon URL from result.

[​](#param-timeout)

timeout

float

Crawl operation time limit in seconds. Default is `150`; range is `10` to `150`.

##

[​](#request-example)

Request Example

```
curl -s -X POST 'https://api.novita.ai/v3/tavily/crawl' \
-H 'Authorization: Bearer sk-123' \
-H 'Content-Type: application/json' \
-d '{
"url": "https://docs.tavily.com",
"chunks_per_source": 3,
"extract_depth": "basic",
"format": "markdown",
"max_depth": 1,
"max_breadth": 5,
"limit": 5,
"allow_external": false,
"include_images": false,
"timeout": 120
}'
```

##

[​](#response)

Response

[​](#param-base-url)

base_url

string

Crawl base URL.

[​](#param-results)

results

object[]

Extracted content from crawled URLs. See [Result Object](#result-object).

[​](#param-response-time)

response_time

float

Request duration in seconds.

[​](#param-request-id)

request_id

string

Unique request identifier.

###

[​](#result-object)

Result Object

[​](#param-url-1)

url

string

Crawled URL.

[​](#param-raw-content)

raw_content

string

Full page content.

[​](#param-favicon)

favicon

string

Site favicon URL. Returned if `include_favicon` is `true`.

##

[​](#errors)

Errors

The platform may return standard HTTP errors before forwarding the request, and Tavily may return upstream errors after forwarding.

[​](#param-400)

400

status

Invalid request body or unsupported parameter value.

[​](#param-401)

401

status

Missing or invalid API key.

[​](#param-403)

403

status

Access denied by platform or upstream provider, or the URL is not supported.

[​](#param-404)

404

status

Route or requested resource not found.

[​](#param-429)

429

status

Rate limit exceeded.

[​](#param-500)

500

status

Internal server error.

[​](#param-502)

502

status

Upstream provider error.

[​](#param-503)

503

status

Service unavailable.

##

[​](#notes)

Notes

- All request bodies are JSON.

- Extra Tavily parameters not listed here may be passed through.

- Response shapes can vary depending on request options.

- This document intentionally omits billing-related fields.

##

[​](#references)

References

For more details, see the [Tavily Crawl API reference](https://docs.tavily.com/documentation/api-reference/endpoint/crawl).

Last modified on June 23, 2026