We provide compatibility with the OpenAI API standard, allowing for easier integration into existing applications.
Base URL
https://api.novita.ai/v3/openai
The APIs we support are:
The Models we support are:
You can find all the models we support here: https://novita.ai/llm-api or request the List Models API to get all available models.
Example with Python Client
pip install 'openai>=1.0.0'
from openai import OpenAI
client = OpenAI(
base_url="https://api.novita.ai/v3/openai",
# Get the Novita AI API Key from: https://novita.ai/settings/key-management.
api_key="<YOUR Novita AI API Key>",
)
model = "meta-llama/llama-3.1-8b-instruct"
stream = True # or False
max_tokens = 512
chat_completion_res = client.chat.completions.create(
model=model,
messages=[
{
"role": "system",
"content": "Act like you are a helpful assistant.",
},
{
"role": "user",
"content": "Hi there!",
}
],
stream=stream,
max_tokens=max_tokens,
)
if stream:
for chunk in chat_completion_res:
print(chunk.choices[0].delta.content or "", end="")
else:
print(chat_completion_res.choices[0].message.content)
from openai import OpenAI
client = OpenAI(
base_url="https://api.novita.ai/v3/openai",
# Get the Novita AI API Key from: https://novita.ai/settings/key-management.
api_key="<YOUR Novita AI API Key>",
)
model = "meta-llama/llama-3.1-8b-instruct"
stream = True # or False
max_tokens = 512
completion_res = client.completions.create(
model=model,
prompt="A chat between a curious user and an artificial intelligence assistant.\nYou are a cooking assistant.\nBe edgy in your cooking ideas.\nUSER: How do I make pasta?\nASSISTANT: First, boil water. Then, add pasta to the boiling water. Cook for 8-10 minutes or until al dente. Drain and serve!\nUSER: How do I make it better?\nASSISTANT:",
stream=stream,
max_tokens=max_tokens,
)
if stream:
for chunk in completion_res:
print(chunk.choices[0].text or "", end="")
else:
print(completion_res.choices[0].text)
Example with Curl Client
# Get the Novita AI API Key from: https://novita.ai/settings/key-management.
export API_KEY="{YOUR Novita AI API Key}"
curl "https://api.novita.ai/v3/openai/chat/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${API_KEY}" \
-d '{
"model": "meta-llama/llama-3.1-8b-instruct",
"messages": [
{
"role": "system",
"content": "Act like you are a helpful assistant."
},
{
"role": "user",
"content": "Hi there!"
}
],
"max_tokens": 512
}'
# Get the Novita AI API Key from: https://novita.ai/settings/key-management.
export API_KEY="{YOUR Novita AI API Key}"
curl "https://api.novita.ai/v3/openai/completions" \
-H "Content-Type: application/json" \
-H "Authorization: Bearer ${API_KEY}" \
-d '{
"model": "meta-llama/llama-3.1-8b-instruct",
"prompt": "A chat between a curious user and an artificial intelligence assistant.\nYou are a cooking assistant.\nBe edgy in your cooking ideas.\nUSER: How do I make pasta?\nASSISTANT: First, boil water. Then, add pasta to the boiling water. Cook for 8-10 minutes or until al dente. Drain and serve!\nUSER: How do I make it better?\nASSISTANT:",
"max_tokens": 512
}'
If you’re already using OpenAI’s chat completion endpoint, you can simply set the base URL to https://api.novita.ai/v3/openai
, obtain and set your API Key (detailed instructions are available at https://novita.ai/guides/quickstart#_2-manage-api-key), and update the model name according to your needs. With these steps, you’re good to go.
Error codes
If the response status code is not 200, we will return the error code and message in JSON format in the response body. The format is as follows:
{
"code": integer,
"reason": "string",
"message": "string"
}
Code | Reason | Description |
---|
401 | INVALID_API_KEY | The API key is invalid. You can check your API key here: Manage API Key |
403 | NOT_ENOUGH_BALANCE | Your credit is not enough. You can top up more credit here: Top Up Credit |
404 | MODEL_NOT_FOUND | The requested model is not found. You can find all the models we support here: https://novita.ai/llm-api or request the List models API to get all available models. |
429 | RATE_LIMIT_EXCEEDED | You have exceeded the rate limit. Please refer to Rate limits for more information. |
500 | MODEL_NOT_AVAILABLE | The requested model is not available now. This is usually due to the model being under maintenance. You can contact us on Discord for more information. |