# MLflow - Documentation

> Trace Novita AI calls in MLflow with OpenAI-compatible SDK instrumentation for Python and JavaScript.

> For the complete documentation index, see [llms.txt](/llms.txt). Markdown is available with `Accept: text/markdown` and `.md` URL variants.

Source: /docs/guides/mlflow

# MLflow

Trace Novita AI calls in MLflow with OpenAI-compatible SDK instrumentation for Python and JavaScript.

This guide shows how to integrate Novita AI with MLflow Tracing. By using Novita AI’s OpenAI-compatible endpoint (`https://api.novita.ai/openai`), you can capture prompts, responses, latency, token usage, and model metadata in MLflow.

![MLflow trace details and timeline](https://mintcdn.com/novitaai/TDkIugxCqQKXM7rz/images/third-party/mlflow-trace-details-timeline.png?fit=max&auto=format&n=TDkIugxCqQKXM7rz&q=85&s=e91bc43cf88dc473905c4001015b8cfa)

#

[​](#prerequisites)

Prerequisites

Before you start, make sure you have:

- Novita AI API key: create one in [Key Management](https://novita.ai/settings/key-management).

- A running MLflow tracking server. You can use local default `http://localhost:5000`.

- Python or JavaScript runtime.

#

[​](#integration-steps)

Integration Steps

##

[​](#step-1-install-dependencies)

Step 1: Install Dependencies

Python

JavaScript / TypeScript

```
pip install 'mlflow[genai]' openai
```

##

[​](#step-2-start-mlflow-server)

Step 2: Start MLflow Server

If you have a local Python environment >= 3.10, you can start MLflow with:

```
mlflow server
```

MLflow also provides a Docker Compose setup:

```
git clone --depth 1 --filter=blob:none --sparse https://github.com/mlflow/mlflow.git
cd mlflow
git sparse-checkout set docker-compose
cd docker-compose
cp .env.dev.example .env
docker compose up -d
```

Then open `http://localhost:5000` to confirm the MLflow UI is accessible.

##

[​](#step-3-enable-tracing-and-call-novita-ai)

Step 3: Enable Tracing and Call Novita AI

Python

JavaScript / TypeScript

```
import openai
import mlflow

# Enable auto-tracing for OpenAI-compatible calls
mlflow.openai.autolog()

# Optional: set tracking target and experiment
mlflow.set_tracking_uri("http://localhost:5000")
mlflow.set_experiment("Novita AI")

client = openai.OpenAI(
base_url="https://api.novita.ai/openai",
api_key="",
)

response = client.chat.completions.create(
model="deepseek/deepseek-r1",
messages=[
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "What is the capital of France?"},
],
)

print(response.choices[0].message.content)
```

##

[​](#step-4-view-traces-in-mlflow-ui)

Step 4: View Traces in MLflow UI

Open your MLflow UI (for example `http://localhost:5000`) and go to your configured experiment to inspect traces.
You should see:

- Prompt and completion content

- Latency and token usage

- Model and request metadata

- Errors/exceptions (if any)

##

[​](#step-5-advanced-tracing-references)

Step 5: Advanced Tracing References

###

[​](#streaming-and-async)

Streaming and Async

MLflow supports tracing for streaming and async Novita AI APIs. See:

- [OpenAI Tracing](https://mlflow.org/docs/latest/genai/tracing/integrations/listing/openai/)

###

[​](#combine-with-frameworks-or-manual-tracing)

Combine with frameworks or manual tracing

Python

JavaScript / TypeScript

```
import json
from openai import OpenAI
import mlflow
from mlflow.entities import SpanType

# Initialize the OpenAI client with Novita AI API endpoint
client = OpenAI(
base_url="https://api.novita.ai/openai",
api_key="",
)

# Create a parent span for the Novita AI call
@mlflow.trace(span_type=SpanType.CHAIN)
def answer_question(question: str):
messages = [{"role": "user", "content": question}]
response = client.chat.completions.create(
model="deepseek/deepseek-r1",
messages=messages,
)

# Attach session/user metadata to the trace
mlflow.update_current_trace(
metadata={
"mlflow.trace.session": "session-12345",
"mlflow.trace.user": "user-a",
}
)
return response.choices[0].message.content

answer = answer_question("What is the capital of France?")
```

For full upstream reference:

- MLflow Novita AI integration page: [Tracing Novita AI](https://mlflow.org/docs/latest/genai/tracing/integrations/listing/novitaai/)

- MLflow OpenAI tracing docs: [OpenAI Tracing](https://mlflow.org/docs/latest/genai/tracing/integrations/listing/openai/)

For Novita model details and endpoint usage, see:

- Novita LLM API guide: [LLM API](/docs/guides/llm-api)

Last modified on March 2, 2026