## Run and test models locally, while exposing them publicly through Clarifai’s API

Local Runners are a powerful feature that let you securely expose your locally running models or servers via a public URL, allowing you to quickly develop, test, and share any models running on your own hardware.

Instead of running solely in the cloud, you can run your models anywhere that supports Python and has an internet connection — whether it's your workstation or on-premise server.

With Local Runners, you can connect your own models to Clarifai's compute plane. This seamless integration enables you to leverage the Clarifai cloud API, workflows, and other platform capabilities with your custom models.

Your model can securely receive and process requests from anywhere, just as it would in a production cloud deployment.

> **Note:** A runner is the actual running instance of your model

## Prerequisites

### Sign Up or Log In

Log in to your existing Clarifai account or sign up for a new one. Next, retrieve the following credentials required for setup:

- **User ID** – In the collapsible left sidebar, select **Settings** and choose **Account** from the dropdown list. Then, locate your user ID.

- **Personal Access Token (PAT)** – From the same **Settings** option, choose **Secrets** to generate or copy your [PAT](https://docs.clarifai.com/control/authentication/pat). This token is used to authenticate your connection with the Clarifai platform.

### Install Clarifai CLI

Install the latest version of the [Clarifai CLI](https://docs.clarifai.com/sdk/cli) tool. It includes built-in support for Local Runners.

- Bash

```bash
pip install --upgrade clarifai
```

> **Note:** You'll need **[Python 3.11 or 3.12](https://docs.clarifai.com/resources/api-overview/python-sdk#python-requirements)** installed to successfully run the Local Runners.

## Quick Start

Once you've completed the prerequisites above, run the following commands and follow the prompts in your terminal to quickly get started with Local Runners.

### Log in to Clarifai

Connect your environment to the Clarifai platform and [create a context](https://docs.clarifai.com/compute/local-runners/#step-2-create-a-context-optional) profile.

- CLI

```bash
clarifai login
```

### Set up a Model

Scaffold a sample [model](https://docs.clarifai.com/compute/local-runners/#step-1-build-a-model) with the necessary files. You can start from a blank Python template, or pick a built-in toolkit pre-configured for your inference engine:

- CLI

```bash
# Local LLM servers (most common for local runners):
clarifai model init --toolkit ollama --model-name llama3.1
clarifai model init --toolkit lmstudio --model-name <model>

# GPU LLM serving (if running on a local GPU):
clarifai model init --toolkit vllm --model-name Qwen/Qwen3-0.6B
clarifai model init --toolkit sglang --model-name meta-llama/Llama-3-8B
clarifai model init --toolkit huggingface --model-name bert-base-uncased

# Other:
clarifai model init --toolkit mcp my-mcp-server         # MCP tool server (FastMCP)
clarifai model init --toolkit openai my-openai-wrapper  # OpenAI-compatible wrapper

# Blank Python template:
clarifai model init
```

Each toolkit pre-fills `model.py` with a working `ModelClass` implementation, `config.yaml` with reasonable defaults, and `requirements.txt`. After scaffolding, the next step is to serve the model locally.

### Serve Your Model Locally

Start the model using `clarifai model serve`. The CLI will guide you through creating any required platform resources (compute clusters, nodepools, and deployments) — described [below](https://docs.clarifai.com/compute/local-runners/#table).

- CLI

```bash
clarifai model serve
```

> **Note:** The `local-runner` command is still supported as an alias for `serve`.

Once your runner launches successfully, your model will be running and accessible via a public URL. You can then open a new terminal, copy the sample code provided in the output, and [test your model](https://docs.clarifai.com/compute/local-runners/#step-4-test-with-snippet)!

## Use Cases for Local Runners

- **Streamlined model development** — Local Runners make it easy to build and test new models directly within your local environment. You can spin up a runner on your machine, route API calls through our public cloud endpoint, and watch requests hit your model in real time. This allows you to debug, set breakpoints, return results, and validate outputs.

- **Leverage your own compute resources** — If you have powerful hardware, you can take advantage of that local compute without relying on Clarifai's [autoscaling](https://docs.clarifai.com/compute/deployments/clusters-nodepools/#step-3-set-node-autoscaling-range) infrastructure. Your model remains accessible through our API with full authentication, even though it runs locally.

- **Locally connect agents** — Because Local Runners execute on your chosen hardware, they can interact with local file systems, make OS-level calls, or access private data stores. With our [MCP (Model Context Protocol)](https://docs.clarifai.com/compute/agents/mcp) model type, you can give your cloud-hosted agents or any MCP-enabled clients authenticated access to your locally controlled information, regardless of their deployment location.

- **Run models anywhere** — Whether on a [local development machine](/content/blog/how-to-run-ai-models-locally-2025-tools-setup-tips/index.html), an on-premises server, or a private cloud cluster, Local Runners seamlessly connect your models to our platform. This enables you to keep sensitive data and custom-built models securely within your own environment.

## Step 1: Build a Model

Start by building the model you want to run using Local Runners.

You can either create a custom model from scratch or leverage pre-trained models from external sources like Hugging Face.

If you're building your own model, follow our comprehensive [step-by-step](https://docs.clarifai.com/compute/models/model-upload/) guide to get started.

You can also explore our [examples repository](https://github.com/Clarifai/runners-examples) to see models built for compatibility with the Clarifai platform.

You can automatically generate a default model by running the [`clarifai model init`](https://docs.clarifai.com/resources/api-overview/cli#clarifai-model-init) command in the terminal from your current directory. After the model's files are created, you can modify them as needed or go with the default options.

## Step 2: Create a Context (Optional)

Running the local development runner relies on certain variables defined in your current context. The _context_ refers to the active environment settings that determine how your commands interact with the Clarifai platform.

> **Note:** You can create this context using the provided default values when you run `clarifai login` and `local-runner` commands.

Any configurations you create locally — such as the computer cluster and app — will also be created on the Clarifai platform, making them reusable whenever you test your model with the local development runner.

These are the variables required to create a runner:

| Variable | Description |
| --- | --- |
| CLARIFAI_PAT | Personal Access Token (PAT) for authentication |
| CLARIFAI_USER_ID (`user_id`) | User ID of the account owning the model |
| CLARIFAI_APP_ID (`app_id`) | App ID containing the model |
| CLARIFAI_MODEL_ID (`model_id`) | The model ID for the model to be run locally |
| CLARIFAI_COMPUTE_CLUSTER_ID (`compute_cluster_id`) | [Compute cluster](https://docs.clarifai.com/compute/deployments/clusters-nodepools) where the Local Runner will reside. _Note that the `user_id` of the compute cluster must match the `user_id` of the model._ |
| CLARIFAI_NODEPOOL_ID (`nodepool_id`) | Nodepool within the compute cluster |
| CLARIFAI_DEPLOYMENT (`deployment_id`) | [Deployment](https://docs.clarifai.com/compute/deployments/deploy-model) for a model into the cluster and nodepool |
| CLARIFAI_RUNNER_ID (`runner_id`) | Auto-generated unique runner ID, created by the API and stored in the context |

The Clarifai CLI follows a defined order of precedence when launching Local Runners. If a value is set as an environment variable in your terminal, it takes priority and is used first.

If no corresponding environment variable is found, the CLI automatically falls back to the value defined in the active context.

## Step 3: Run Your Model

To run your model with the local development runner, navigate to the directory where your custom model is located.

### Log in to Clarifai

Run the following command to log in to the Clarifai platform and establish a connection.

- CLI

```bash
clarifai login
```

After running the `login` command, you'll be prompted to enter the following details to authenticate your connection:

- **Context name** — You can provide a custom name for your Clarifai configuration context, or simply press Enter to use the default name, "default". This helps you manage different configurations if needed.
- **User ID** — Enter your Clarifai user ID.
- **PAT** — Enter your Clarifai [PAT](/content/compute/local-runners/#get-a-pat-key-and-user-id/index.html). Note that if you press Enter, and you have set the `CLARIFAI_PAT` environment variable, it will use that token automatically.

### Serve Your Model

Start the model using `clarifai model serve` (or the alias `local-runner`):

- CLI

```bash
clarifai model serve
```

Or with a specific model path:

- CLI

```bash
clarifai model serve [OPTIONS] [MODEL_PATH]
```

> **Note:**`MODEL_PATH` is an optional path to the model directory. If omitted, the current directory is used by default.

Options

| Option | Default | Description |
| --- | --- | --- |
| `--mode` | `none` | `none` (current env), `env` (auto-creates a virtualenv), `container` (Docker) |
| `--grpc` | off | Standalone gRPC server — no Clarifai login required |
| `--port` | 8000 | Server port (with `--grpc`) |
| `--concurrency` | 32 | Max concurrent requests (equivalent to `--pool_size`) |
| `--keep-image` | off | Keep Docker image after exit (container mode only) |
| `--health-check-port` | auto | Port for the health check HTTP server |
| `--disable-health-check` | off | Disable the health check server entirely |
| `--auto-find-health-check-port` | off | Automatically find an available port if the default is in use |
| `-v, --verbose` | off | Show detailed logs (useful for debugging toolkit servers like [Ollama](https://docs.clarifai.com/compute/toolkits/ollama/)) |

#### Standalone gRPC Mode

For offline development without a Clarifai account, use `--grpc` to start a standalone gRPC server:

- CLI

```bash
# Start standalone gRPC server (no login required)
clarifai model serve ./my-model --grpc

# Custom port
clarifai model serve ./my-model --grpc --port 9000
```

This is useful for quick local testing and debugging without needing an internet connection or Clarifai credentials.

#### Virtual Environment Mode

Use `--mode env` to automatically create a virtualenv with the model's dependencies:

- CLI

```bash
clarifai model serve ./my-model --mode env
```

This isolates the model's packages from your system Python, preventing dependency conflicts.

### Test Your Runner

When the local development runner starts in your terminal, it automatically generates an example client code snippet based on the [model's signature](https://docs.clarifai.com/compute/inference/#structure-of-prediction-methods).

You can run this snippet in a separate terminal (within the same directory) to receive the model’s response output.

Example Code Snippet

```text
# Clarifai Model Client Script
# Set the environment variables `CLARIFAI_DEPLOYMENT_ID` and `CLARIFAI_PAT` to run this script.
# Example usage:
import os

from clarifai.client import Model
from clarifai.runners.utils import data_types

model = Model(
    "https://clarifai.com/alfrick/local-runner-app/models/local-runner-model",
    deployment_id='local-runner-deployment',  # Only needed for dedicated deployed models
    base_url="https://api.clarifai.com",
)

response = model.predict(
    text1="What is the future of AI?",
)
print(response)
```

The terminal also displays a link to the [AI Playground](https://docs.clarifai.com/getting-started/quickstart-playground), which you can copy and use to interact with the model directly.

When you’re done testing, simply close the terminal running the local development runner to shut it down.

## Additional Examples

- [Simple example for running Hello World model locally with Clarifai’s Local Runners](https://github.com/Clarifai/runners-examples/tree/main/hello-world)
- [YouTube video on how to build your own local AI agent using Clarifai’s Local Runners](https://www.youtube.com/watch?v=JOdtZDmCFfk)