We provide compatibility with the OpenAI API standard, allowing for easier integration into existing applications.

Base URL

https://api.novita.ai/v3/openai

The APIs we support are:

The Models we support are:

You can find all the models we support here: https://novita.ai/llm-api or request the List Models API to get all available models.

Example with Python Client

pip install 'openai>=1.0.0'
  • Chat Completions API
from openai import OpenAI

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    # Get the Novita AI API Key from: https://novita.ai/settings/key-management.
    api_key="<YOUR Novita AI API Key>",
)

model = "meta-llama/llama-3.1-8b-instruct"
stream = True  # or False
max_tokens = 512

chat_completion_res = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "system",
            "content": "Act like you are a helpful assistant.",
        },
        {
            "role": "user",
            "content": "Hi there!",
        }
    ],
    stream=stream,
    max_tokens=max_tokens,
)

if stream:
    for chunk in chat_completion_res:
        print(chunk.choices[0].delta.content or "", end="")
else:
    print(chat_completion_res.choices[0].message.content)
  • Completions API
from openai import OpenAI

client = OpenAI(
    base_url="https://api.novita.ai/v3/openai",
    # Get the Novita AI API Key from: https://novita.ai/settings/key-management.
    api_key="<YOUR Novita AI API Key>",
)

model = "meta-llama/llama-3.1-8b-instruct"
stream = True  # or False
max_tokens = 512

completion_res = client.completions.create(
    model=model,
    prompt="A chat between a curious user and an artificial intelligence assistant.\nYou are a cooking assistant.\nBe edgy in your cooking ideas.\nUSER: How do I make pasta?\nASSISTANT: First, boil water. Then, add pasta to the boiling water. Cook for 8-10 minutes or until al dente. Drain and serve!\nUSER: How do I make it better?\nASSISTANT:",
    stream=stream,
    max_tokens=max_tokens,
)

if stream:
    for chunk in completion_res:
        print(chunk.choices[0].text or "", end="")
else:
    print(completion_res.choices[0].text)

Example with Curl Client

  • Chat Completions API
# Get the Novita AI API Key from: https://novita.ai/settings/key-management.
export API_KEY="{YOUR Novita AI API Key}"

curl "https://api.novita.ai/v3/openai/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${API_KEY}" \
  -d '{
    "model": "meta-llama/llama-3.1-8b-instruct",
    "messages": [
        {
            "role": "system",
            "content": "Act like you are a helpful assistant."
        },
        {
            "role": "user",
            "content": "Hi there!"
        }
    ],
    "max_tokens": 512
}'
  • Completions API
# Get the Novita AI API Key from: https://novita.ai/settings/key-management.
export API_KEY="{YOUR Novita AI API Key}"

curl "https://api.novita.ai/v3/openai/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${API_KEY}" \
  -d '{
    "model": "meta-llama/llama-3.1-8b-instruct",
    "prompt": "A chat between a curious user and an artificial intelligence assistant.\nYou are a cooking assistant.\nBe edgy in your cooking ideas.\nUSER: How do I make pasta?\nASSISTANT: First, boil water. Then, add pasta to the boiling water. Cook for 8-10 minutes or until al dente. Drain and serve!\nUSER: How do I make it better?\nASSISTANT:",
    "max_tokens": 512
}'

If you’re already using OpenAI’s chat completion endpoint, you can simply set the base URL to https://api.novita.ai/v3/openai, obtain and set your API Key (detailed instructions are available at https://novita.ai/guides/quickstart#_2-manage-api-key), and update the model name according to your needs. With these steps, you’re good to go.

Error codes

If the response status code is not 200, we will return the error code and message in JSON format in the response body. The format is as follows:

{
    "code": integer,
    "reason": "string",
    "message": "string"
}
CodeReasonDescription
401INVALID_API_KEYThe API key is invalid. You can check your API key here: Manage API Key
403NOT_ENOUGH_BALANCEYour credit is not enough. You can top up more credit here: Top Up Credit
404MODEL_NOT_FOUNDThe requested model is not found. You can find all the models we support here: https://novita.ai/llm-api or request the List models API to get all available models.
429RATE_LIMIT_EXCEEDEDYou have exceeded the rate limit. Please refer to Rate limits for more information.
500MODEL_NOT_AVAILABLEThe requested model is not available now. This is usually due to the model being under maintenance. You can contact us on Discord for more information.