# Run inference with Clarifai models locally using OpenHands

OpenHands is an AI-powered code assistance framework designed to streamline development by harnessing the flexibility of large language models (LLMs).

Let's walk through how to run inference with Clarifai models locally using OpenHands, while leveraging its intelligent assistance to boost your productivity.

## Step 1: Perform Prerequisites

- **Get a PAT (Personal Access Token) key** — To use OpenHands with Clarifai models, you need a [PAT](https://docs.clarifai.com/control/authentication/pat) for authentication. Log in or sign up for a Clarifai account, then go to **Settings** in the collapsible left sidebar and select **Secrets**. From there, you can create a new token or copy an existing one to use with your requests.

- **Get a model** — Clarifai's [Community platform](/content/explore/index.html) offers a wide selection of cutting-edge language models that you can run using OpenHands. Browse the platform to find a model that best fits your use case. For this example, we'll use the [GPT-OSS-120B](/content/openai/chat-completion/models/gpt-oss-120b/index.html) model.

- **Install Docker Desktop** — OpenHands runs inside a Docker container, so you’ll need Docker installed and running on your system. You can download and install Docker Desktop for your operating system from the [official Docker website](https://www.docker.com/products/docker-desktop). Be sure to follow the installation steps specific to your OS (Windows, macOS, or Linux).

- **Get a GitHub token** — If you want to integrate OpenHands with GitHub, you’ll need a personal access token for authentication. You can follow [GitHub’s documentation](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) to generate one.

## Step 2: Run OpenHands Locally

### Option A: CLI Launcher (Recommended)

The simplest way to run OpenHands locally is by using the CLI launcher with the [`uv`](https://docs.astral.sh/uv/) package manager. This approach provides better isolation from your existing project’s virtual environment and is needed for OpenHands’ default MCP servers.

If you don’t already have `uv` installed, follow the official [uv installation guide](https://docs.astral.sh/uv/getting-started/installation/) for the latest instructions for your platform.

Then, use either of the following commands to launch OpenHands.

Launch by starting the GUI server:

```bash
uvx --python 3.12 openhands serve
```

Or, start in CLI mode:

```bash
uvx --python 3.12 openhands
```

**Example Output**

```text
uvx --python 3.12 openhands serve
Installed 164 packages in 450ms
🚀 Launching OpenHands GUI server...

Pulling required Docker images...
Running Docker command: docker pull docker.openhands.dev/openhands/runtime:latest-nikolaik
latest-nikolaik: Pulling from openhands/runtime
Digest: sha256:95fc4d8e753f41d34cd564e78d3d64217c5d22296ea1cb7bf92614c8198de8f5
Status: Image is up to date for docker.openhands.dev/openhands/runtime:latest-nikolaik
docker.openhands.dev/openhands/runtime:latest-nikolaik

✅ Starting OpenHands GUI server...
The server will be available at: http://localhost:3000
Press Ctrl+C to stop the server.

Running Docker command: docker run -it --rm --pull=always -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.openhands.dev/openhands/runtime:latest-nikolaik -e LOG_ALL_EVENTS=true -v /var/run/docker.sock:/var/run/docker.sock -v /Users/macbookpro/.openhands:/.openhands -p 3000:3000 --add-host host.docker.internal:host-gateway --name openhands-app docker.openhands.dev/openhands/openhands:latest
latest: Pulling from openhands/openhands
Digest: sha256:87c868e4f89ee453bf2ac57f89c71af2619c6d85b05748635b9da78254092d13
Status: Image is up to date for docker.openhands.dev/openhands/openhands:latest
Starting OpenHands...
Running OpenHands as root
18:07:28 - openhands:INFO: utils.py:152 - config.toml not found: [Errno 2] No such file or directory: 'config.toml'. Toml values have not been applied.
...
INFO:     Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)
```

### Option B: Docker

You can also run OpenHands using Docker.

Note that OpenHands uses a dedicated Docker image to provide a sandboxed execution environment. Pulling this image ahead of time from the All-Hands AI Docker registry can speed up the initial startup process.

> **Note:** Always check the [official OpenHands documentation](https://github.com/OpenHands/OpenHands?tab=readme-ov-file#option-2-docker) for the latest recommended image tag, as it may change over time.

- CLI

```bash
docker pull docker.openhands.dev/openhands/runtime:0.62-nikolaik
```

Next, start OpenHands using the following comprehensive `docker run` command.

```python
docker run -it --rm --pull=always \
    -e SANDBOX_RUNTIME_CONTAINER_IMAGE=docker.openhands.dev/openhands/runtime:0.62-nikolaik \
    -e LOG_ALL_EVENTS=true \
    -v /var/run/docker.sock:/var/run/docker.sock \
    -v ~/.openhands:/.openhands \
    -p 3000:3000 \
    --add-host host.docker.internal:host-gateway \
    --name openhands-app \
    docker.openhands.dev/openhands/openhands:0.62
```

### Run OpenHands

After launching OpenHands, monitor the terminal for log output. Once the application finishes its startup process, open your preferred web browser and go to:

```text
http://localhost:3000
```

At this point, OpenHands is successfully installed and running on your local machine.

## Step 3: Connect to an LLM

To configure OpenHands to use a Clarifai model as its cognitive engine, click the settings icon in the bottom-left corner of the sidebar.

In the menu that appears, select **LLM** to open the LLM configuration panel.

Here, you can set up the large language model that OpenHands will use for reasoning and task execution.

Fill in the following fields:

- **LLM Provider** — From the list of providers, select `Clarifai`.
- **LLM Model** — Choose the Clarifai model you want OpenHands to use.
- **API Key** — Enter your Clarifai PAT as described earlier.

After filling in the fields, click the **Save Changes** button in the bottom-right corner of the interface.

### Advanced LLM Settings

To access [advanced](https://docs.openhands.dev/openhands/usage/settings/llm-settings#advanced-llm-settings) configuration options, toggle the **Advanced** switch in the LLM configuration panel.

This allows you to specify a custom Clarifai model and adjust additional LLM options. This is helpful when the model you want to use is not available in the basic dropdown selection.

Then, fill in the following fields:

- **Custom Model** — Enter the Clarifai model URL you want to use. To ensure OpenAI compatibility, prefix the model path with `openai/`, followed by the full Clarifai model URL. Here is an example: `openai/https://clarifai.com/openai/chat-completion/models/gpt-oss-120b`.
- **Base URL** — Enter Clarifai's [OpenAI-compatible](https://docs.clarifai.com/compute/inference/#predict-with-openai-compatible-format) API endpoint: `https://api.clarifai.com/v2/ext/openai/v1`.
- **API Key** — Enter your PAT, as explained earlier.

> **Note:** You can leave all other fields at their default values, including the selected agent unless you require custom behavior.

After filling in the fields, click the **Save Changes** button at the bottom-right corner of the interface.

## Step 4: Integrate with GitHub

Integrating OpenHands with GitHub allows you to take advantage of version control, repository access, and collaboration features.

To set this up, open the **Settings** panel and navigate to the **Integrations** tab. Then, enter your GitHub personal access token in the designated field.

> **Optionally:** If you are using a self-hosted or enterprise instance, you may also provide your GitHub host URL.

Once everything is filled in, click **Save Changes** in the bottom-right corner to apply your integration settings.

## Step 5: Start a Conversation

Next, click the plus ( **+**) **Start new conversation** button at the top of the sidebar. From there, connect to a repository by selecting the desired repo and its branch.

Once selected, click the **Launch** button to begin your session.

## Step 6: Start Building

In the main interface, use the input field to prompt the agent and begin working on your task. You can continue interacting with the agent to refine the results, extend the functionality, or troubleshoot issues as they arise.

OpenHands sends each request to the large language model you configured — powered by Clarifai — and the model generates the responses.

Once you're satisfied with the output, you can seamlessly push your changes to GitHub directly from the interface, making it easy to commit, publish, and share your work.

That's it!