# Harbor Integration - Documentation

> For the complete documentation index, see [llms.txt](/llms.txt). Markdown is available with `Accept: text/markdown` and `.md` URL variants.

Source: /docs/guides/sandbox-integrations-harbor

# Harbor Integration

Copy pageCopy page

Copy pageCopy page

[Harbor](https://github.com/harbor-framework/harbor) is an open-source framework for evaluating and optimizing agents and language models in sandboxed environments. It can run benchmark tasks against different agents and execution backends, collect trajectories, verify task results, and summarize benchmark scores.
Novita Sandbox is integrated into Harbor as a cloud environment provider. After installing Harbor with the Novita extra, users can run Harbor tasks on Novita-hosted sandboxes by selecting `--env novita`. The Harbor CLI still runs on your local machine or server, while the task environment, agent process, command execution, and verification run inside the Novita Sandbox.
Install Harbor with the `novita` extra so the Novita Sandbox SDK and related dependencies are available.

CLI

```
uv tool install 'harbor[novita]'

# Or, if you use pip:
pip install 'harbor[novita]'
```

Harbor requires Python 3.12 or later. The `[novita]` extra installs the optional dependencies needed by Harbor’s Novita environment provider; it does not change the Harbor command syntax.
Set `NOVITA_API_KEY` so Harbor can create Novita templates and sandboxes. If you also want the agent to call Novita’s OpenAI-compatible LLM API, pass the same key through the OpenAI-compatible environment variables.

CLI

```
export NOVITA_API_KEY=
export OPENAI_API_KEY="$NOVITA_API_KEY"
export OPENAI_BASE_URL="https://api.novita.ai/openai"
export OPENAI_API_BASE="https://api.novita.ai/openai"
```

`OPENAI_BASE_URL` is used by OpenAI-compatible clients. `OPENAI_API_BASE` is included because some agent runtimes and LiteLLM-based tools read that variable name instead.
The following command runs Harbor’s registry dataset `hello-world@1.0` on Novita Sandbox using `mini-swe-agent` and the Novita model `moonshotai/kimi-k3`. This example does not require cloning the Harbor source repository.

CLI

```
harbor run \
--dataset hello-world@1.0 \
--n-tasks 1 \
--agent mini-swe-agent \
--model openai/moonshotai/kimi-k3 \
--n-concurrent 1 \
--env novita \
--ae OPENAI_API_KEY="$OPENAI_API_KEY" \
--ae OPENAI_BASE_URL="$OPENAI_BASE_URL" \
--ae OPENAI_API_BASE="$OPENAI_API_BASE" \
--agent-timeout-multiplier 5
```

You can also use Harbor as a Python package instead of shelling out to the CLI. The key setting is `EnvironmentConfig(type=EnvironmentType.NOVITA)`, which is the code equivalent of `--env novita`. Harbor still manages the dataset, agent execution, verifier, job output, and Novita Sandbox lifecycle.

CLI

```
pip install "harbor[novita]"
```

The following Python script is equivalent to the CLI smoke test above:

Python

```
import asyncio
import os

from harbor.job import Job
from harbor.models.environment_type import EnvironmentType
from harbor.models.job.config import DatasetConfig, JobConfig
from harbor.models.trial.config import AgentConfig, EnvironmentConfig

async def main():
# Harbor uses NOVITA_API_KEY on the host process to create Novita sandboxes.
if "NOVITA_API_KEY" not in os.environ:
raise RuntimeError("NOVITA_API_KEY is required")

# These environment variables are passed to the agent for model calls.
os.environ.setdefault("OPENAI_API_KEY", os.environ["NOVITA_API_KEY"])
os.environ.setdefault("OPENAI_BASE_URL", "https://api.novita.ai/openai")
os.environ.setdefault("OPENAI_API_BASE", "https://api.novita.ai/openai")

config = JobConfig(
datasets=[
DatasetConfig(
name="hello-world",
version="1.0",
n_tasks=1,
)
],
agents=[
AgentConfig(
name="mini-swe-agent",
model_name="openai/moonshotai/kimi-k3",
env={
"OPENAI_API_KEY": "${OPENAI_API_KEY}",
"OPENAI_BASE_URL": "${OPENAI_BASE_URL}",
"OPENAI_API_BASE": "${OPENAI_API_BASE}",
},
)
],
environment=EnvironmentConfig(
type=EnvironmentType.NOVITA,
),
n_concurrent_trials=1,
agent_timeout_multiplier=5,
)

job = await Job.create(config)
result = await job.run()
print(result.model_dump_json(indent=2))

if __name__ == "__main__":
asyncio.run(main())
```

For a local task path, replace `datasets=[...]` with `tasks=[TaskConfig(path=Path("/path/to/task"))]` and import `TaskConfig` from `harbor.models.trial.config`. Keep the same Novita environment config and agent model settings.
Do not put the Novita Sandbox key only inside `AgentConfig.env`. `NOVITA_API_KEY` must be available to the host Python process so Harbor can create the sandbox. The OpenAI-compatible model settings belong in `AgentConfig.env`, because those values are forwarded to the agent running inside the sandbox.
The smoke test above uses Harbor’s registry dataset, so it does not need local Harbor source code. If you use `--path`, Harbor treats the value as a local file-system path and the directory must exist on the machine running the Harbor CLI.

CLI

```
harbor run \
--path /absolute/path/to/your/harbor-task \
--agent mini-swe-agent \
--model openai/moonshotai/kimi-k3 \
--n-concurrent 1 \
--env novita \
--ae OPENAI_API_KEY="$OPENAI_API_KEY" \
--ae OPENAI_BASE_URL="$OPENAI_BASE_URL" \
--ae OPENAI_API_BASE="$OPENAI_API_BASE"
```

Use `--dataset hello-world@1.0` for the simplest installed-package smoke test. Use `--path` only when you already have a local Harbor task directory.
When `--env novita` is selected, Harbor resolves the execution backend to its Novita environment provider. For a standard single-container task, Harbor builds or reuses a Novita template for the task environment, creates a Novita Sandbox from that template, uploads task files, and runs the agent and verifier commands inside the sandbox.
The agent is installed and executed inside the sandbox. On first use, agent setup may install runtime dependencies such as Python packages or Node.js tooling inside the sandbox. Harbor writes agent logs and task results back to the local `jobs/` directory after the run.

SettingDescription`NOVITA_API_KEY`Required. API key used by Harbor to create Novita templates and sandboxes.`OPENAI_API_KEY`Required when the selected agent calls Novita’s OpenAI-compatible LLM API.`OPENAI_BASE_URL`OpenAI-compatible base URL. Use `https://api.novita.ai/openai`.`OPENAI_API_BASE`Compatibility alias for agent runtimes that read this variable instead of `OPENAI_BASE_URL`.`--env novita`Selects Novita Sandbox as the Harbor execution environment.`--ae KEY=VALUE`Passes an environment variable to the agent process inside the sandbox.`--agent-timeout-multiplier`Optional multiplier for agent execution timeout. Increase it for slow models or slow first-time agent setup.`--override-cpus`Optional Harbor runtime override for sandbox CPU allocation.`--override-memory-mb`Optional Harbor runtime override for sandbox memory allocation in MB.`--force-build`Forces Harbor to rebuild the environment template instead of reusing a cached template.

Use Novita Sandbox when you want Harbor workloads to run in remote, isolated environments instead of relying on local Docker capacity. This is useful for CI, shared development servers, high-concurrency benchmark runs, and experiments that need reproducible sandbox lifecycle management across many trials.

Last modified on August 5, 2026

[Mount S3 Storage to Sandbox](/docs/guides/sandbox-mount-cloudstorage)[List sandboxes](/docs/guides/sandbox-list)