Model Library/Llama 3.1 8B Instruct
Llama 3.1 8B Instruct

Llama 3.1 8B Instruct

meta-llama/llama-3.1-8b-instruct
by Meta
Input price
$0.02/ M tokens
Output price
$0.05/ M tokens
Context window
16Ktokens
Max output
16Ktokens

Meta's latest class of models, Llama 3.1, launched with a variety of sizes and configurations. The 8B instruct-tuned version is particularly fast and efficient. It has demonstrated strong performance in human evaluations, outperforming several leading closed-source models.

Features

Serverless API

Docs

meta-llama/llama-3.1-8b-instruct is available via Novita's serverless API, where you pay per token. There are several ways to call the API, including OpenAI-compatible endpoints with exceptional reasoning performance.

On-demand Deployments

Docs

On-demand deployments allow you to use meta-llama/llama-3.1-8b-instruct on dedicated GPUs with high-performance serving stack with high reliability and no rate limits.

Available Serverless

Run queries immediately, pay only for usage

Input$0.02 / M Tokens
Output$0.05 / M Tokens

  1. Get your API key

    Sign up and create a key to run the code below. The API is OpenAI-compatible.

    Log inGet your API key

  2. Make your first request

    The API is OpenAI-compatible: point any OpenAI SDK at the base URL and pass this model ID.

    Base URL
    https://api.novita.ai/openai
    Model ID
    meta-llama/llama-3.1-8b-instruct
    API Endpoint
    chat/completionscompletions
    1from openai import OpenAI
2
3client = OpenAI(
4    api_key="<Your API Key>",
5    base_url="https://api.novita.ai/openai"
6)
7
8response = client.chat.completions.create(
9    model="meta-llama/llama-3.1-8b-instruct",
10    messages=[
11        {"role": "system", "content": "You are a helpful assistant."},
12        {"role": "user", "content": "Hello, how are you?"}
13    ],
14    max_tokens=16384,
15    temperature=0.7
16)
17
18print(response.choices[0].message.content)