- Conducting evaluations and data analysis.
- Classifying extensive datasets.
- Generating document summaries in an offline mode.
Quick Start
1. Preparing Batch Files
The Batch API uses .jsonl format files as input, with each line representing the details of an API inference request. Available endpoints include/v1/chat/completions
and /v1/completions
.
Set the
endpoint
parameter to /v1/chat/completions
or /v1/completions
for OpenAI API compatibility.custom_id
to locate inference results in the output file after batch completion. Parameters in the body
field of each line are sent as actual inference request parameters to the endpoint.
Below is an example input file containing 2 requests:
2. Upload Batch Input File
Upload the batch input file to ensure it can be accurately referenced when creating a batch. Use the Files API to upload your .jsonl file and set the purpose tobatch
. Note that the file will be retained for 15 days.
For how to get API key, refer to the API Key Management.
3. Creating a Batch
Once the input file is successfully uploaded, you can initiate a batch using the ID of the uploaded File object. The completion window is fixed at48h
and is currently non-adjustable.
Code Example
Python
4. Check the Status of a Batch
You can check a batchโs status at any time to receive the latest batch information. The status enumeration values of the Batch object are as follows:Status | Description |
---|---|
VALIDATING | The input file is being validated before the batch can begin |
PROGRESS | Batch is in progress |
COMPLETED | Batch processing completed successfully |
FAILED | Batch processing failed |
EXPIRED | Batch exceeded deadline |
CANCELLING | Batch is being cancelled |
CANCELLED | Batch was cancelled |
5. Retrieve the Results
Once the batch inference is complete, you can download the result output file using theoutput_file_id
field from the Batch object.
The result output file will be deleted 30 days after the batch inference concludes, so please retrieve it promptly via the interface.
Code Example
Python
Instructions
Supported Models
deepseek/deepseek-r1-0528
Limitations
- Each batch can contain up to 50,000 requests.
- The maximum input file size per batch is 100MB.
Error Handling
Errors encountered during batch processing are recorded in a separate error file, accessible via the error_file_id field. Common error codes include:Error Code | Description | Solution |
---|---|---|
400 | Invalid request format | Check JSONL syntax and required fields |
401 | Authentication failed | Verify API key |
404 | Batch not found | Check batch ID |
429 | Rate limit exceeded | Reduce request frequency |
500 | Server error | Contact us |