Harbor Integration

Harbor is an open-source framework for evaluating and optimizing agents and language models in sandboxed environments. It can run benchmark tasks against different agents and execution backends, collect trajectories, verify task results, and summarize benchmark scores. Novita Sandbox is integrated into Harbor as a cloud environment provider. After installing Harbor with the Novita extra, users can run Harbor tasks on Novita-hosted sandboxes by selecting --env novita. The Harbor CLI still runs on your local machine or server, while the task environment, agent process, command execution, and verification run inside the Novita Sandbox. Install Harbor with the novita extra so the Novita Sandbox SDK and related dependencies are available.

CLI

uv tool install 'harbor[novita]'

# Or, if you use pip:
pip install 'harbor[novita]'

Harbor requires Python 3.12 or later. The [novita] extra installs the optional dependencies needed by Harbor’s Novita environment provider; it does not change the Harbor command syntax. Set NOVITA_API_KEY so Harbor can create Novita templates and sandboxes. If you also want the agent to call Novita’s OpenAI-compatible LLM API, pass the same key through the OpenAI-compatible environment variables.

CLI

export NOVITA_API_KEY=<YOUR_NOVITA_API_KEY>
export OPENAI_API_KEY="$NOVITA_API_KEY"
export OPENAI_BASE_URL="https://api.novita.ai/openai"
export OPENAI_API_BASE="https://api.novita.ai/openai"

OPENAI_BASE_URL is used by OpenAI-compatible clients. OPENAI_API_BASE is included because some agent runtimes and LiteLLM-based tools read that variable name instead. The following command runs Harbor’s registry dataset hello-world@1.0 on Novita Sandbox using mini-swe-agent and the Novita model moonshotai/kimi-k3. This example does not require cloning the Harbor source repository.

CLI

harbor run \
  --dataset hello-world@1.0 \
  --n-tasks 1 \
  --agent mini-swe-agent \
  --model openai/moonshotai/kimi-k3 \
  --n-concurrent 1 \
  --env novita \
  --ae OPENAI_API_KEY="$OPENAI_API_KEY" \
  --ae OPENAI_BASE_URL="$OPENAI_BASE_URL" \
  --ae OPENAI_API_BASE="$OPENAI_API_BASE" \
  --agent-timeout-multiplier 5

You can also use Harbor as a Python package instead of shelling out to the CLI. The key setting is EnvironmentConfig(type=EnvironmentType.NOVITA), which is the code equivalent of --env novita. Harbor still manages the dataset, agent execution, verifier, job output, and Novita Sandbox lifecycle.

CLI

pip install "harbor[novita]"

The following Python script is equivalent to the CLI smoke test above:

Python

import asyncio
import os

from harbor.job import Job
from harbor.models.environment_type import EnvironmentType
from harbor.models.job.config import DatasetConfig, JobConfig
from harbor.models.trial.config import AgentConfig, EnvironmentConfig


async def main():
    # Harbor uses NOVITA_API_KEY on the host process to create Novita sandboxes.
    if "NOVITA_API_KEY" not in os.environ:
        raise RuntimeError("NOVITA_API_KEY is required")

    # These environment variables are passed to the agent for model calls.
    os.environ.setdefault("OPENAI_API_KEY", os.environ["NOVITA_API_KEY"])
    os.environ.setdefault("OPENAI_BASE_URL", "https://api.novita.ai/openai")
    os.environ.setdefault("OPENAI_API_BASE", "https://api.novita.ai/openai")

    config = JobConfig(
        datasets=[
            DatasetConfig(
                name="hello-world",
                version="1.0",
                n_tasks=1,
            )
        ],
        agents=[
            AgentConfig(
                name="mini-swe-agent",
                model_name="openai/moonshotai/kimi-k3",
                env={
                    "OPENAI_API_KEY": "${OPENAI_API_KEY}",
                    "OPENAI_BASE_URL": "${OPENAI_BASE_URL}",
                    "OPENAI_API_BASE": "${OPENAI_API_BASE}",
                },
            )
        ],
        environment=EnvironmentConfig(
            type=EnvironmentType.NOVITA,
        ),
        n_concurrent_trials=1,
        agent_timeout_multiplier=5,
    )

    job = await Job.create(config)
    result = await job.run()
    print(result.model_dump_json(indent=2))


if __name__ == "__main__":
    asyncio.run(main())

For a local task path, replace datasets=[...] with tasks=[TaskConfig(path=Path("/path/to/task"))] and import TaskConfig from harbor.models.trial.config. Keep the same Novita environment config and agent model settings. Do not put the Novita Sandbox key only inside AgentConfig.env. NOVITA_API_KEY must be available to the host Python process so Harbor can create the sandbox. The OpenAI-compatible model settings belong in AgentConfig.env, because those values are forwarded to the agent running inside the sandbox. The smoke test above uses Harbor’s registry dataset, so it does not need local Harbor source code. If you use --path, Harbor treats the value as a local file-system path and the directory must exist on the machine running the Harbor CLI.

CLI

harbor run \
  --path /absolute/path/to/your/harbor-task \
  --agent mini-swe-agent \
  --model openai/moonshotai/kimi-k3 \
  --n-concurrent 1 \
  --env novita \
  --ae OPENAI_API_KEY="$OPENAI_API_KEY" \
  --ae OPENAI_BASE_URL="$OPENAI_BASE_URL" \
  --ae OPENAI_API_BASE="$OPENAI_API_BASE"

Use --dataset hello-world@1.0 for the simplest installed-package smoke test. Use --path only when you already have a local Harbor task directory. When --env novita is selected, Harbor resolves the execution backend to its Novita environment provider. For a standard single-container task, Harbor builds or reuses a Novita template for the task environment, creates a Novita Sandbox from that template, uploads task files, and runs the agent and verifier commands inside the sandbox. The agent is installed and executed inside the sandbox. On first use, agent setup may install runtime dependencies such as Python packages or Node.js tooling inside the sandbox. Harbor writes agent logs and task results back to the local jobs/ directory after the run.

Setting	Description
`NOVITA_API_KEY`	Required. API key used by Harbor to create Novita templates and sandboxes.
`OPENAI_API_KEY`	Required when the selected agent calls Novita’s OpenAI-compatible LLM API.
`OPENAI_BASE_URL`	OpenAI-compatible base URL. Use `https://api.novita.ai/openai`.
`OPENAI_API_BASE`	Compatibility alias for agent runtimes that read this variable instead of `OPENAI_BASE_URL`.
`--env novita`	Selects Novita Sandbox as the Harbor execution environment.
`--ae KEY=VALUE`	Passes an environment variable to the agent process inside the sandbox.
`--agent-timeout-multiplier`	Optional multiplier for agent execution timeout. Increase it for slow models or slow first-time agent setup.
`--override-cpus`	Optional Harbor runtime override for sandbox CPU allocation.
`--override-memory-mb`	Optional Harbor runtime override for sandbox memory allocation in MB.
`--force-build`	Forces Harbor to rebuild the environment template instead of reusing a cached template.

Use Novita Sandbox when you want Harbor workloads to run in remote, isolated environments instead of relying on local Docker capacity. This is useful for CI, shared development servers, high-concurrency benchmark runs, and experiments that need reproducible sandbox lifecycle management across many trials.

Get started

Model APIs

Novita Sandbox

GPUs

Observability

Resources