Local Runners | Clarifai Docs

Run and test models locally, while exposing them publicly through Clarifai’s API

Local Runners are a powerful feature that let you securely expose your locally running models or servers via a public URL, allowing you to quickly develop, test, and share any models running on your own hardware.

Instead of running solely in the cloud, you can run your models anywhere that supports Python and has an internet connection — whether it's your workstation or on-premise server.

With Local Runners, you can connect your own models to Clarifai's compute plane. This seamless integration enables you to leverage the Clarifai cloud API, workflows, and other platform capabilities with your custom models.

Your model can securely receive and process requests from anywhere, just as it would in a production cloud deployment.

Note: A runner is the actual running instance of your model

Prerequisites

Sign Up or Log In

Log in to your existing Clarifai account or sign up for a new one. Next, retrieve the following credentials required for setup:

• User ID – In the collapsible left sidebar, select Settings and choose Account from the dropdown list. Then, locate your user ID.

• Personal Access Token (PAT) – From the same Settings option, choose Secrets to generate or copy your PAT. This token is used to authenticate your connection with the Clarifai platform.

Install Clarifai CLI

Install the latest version of the Clarifai CLI tool. It includes built-in support for Local Runners.

• Bash

pip install --upgrade clarifai

Note: You'll need Python 3.11 or 3.12 installed to successfully run the Local Runners.

Quick Start

Once you've completed the prerequisites above, run the following commands and follow the prompts in your terminal to quickly get started with Local Runners.

Log in to Clarifai

Connect your environment to the Clarifai platform and create a context profile.

• CLI

clarifai login

Set up a Model

Scaffold a sample model with the necessary files. You can start from a blank Python template, or pick a built-in toolkit pre-configured for your inference engine:

• CLI

# Local LLM servers (most common for local runners):
clarifai model init --toolkit ollama --model-name llama3.1
clarifai model init --toolkit lmstudio --model-name <model>

# GPU LLM serving (if running on a local GPU):
clarifai model init --toolkit vllm --model-name Qwen/Qwen3-0.6B
clarifai model init --toolkit sglang --model-name meta-llama/Llama-3-8B
clarifai model init --toolkit huggingface --model-name bert-base-uncased

# Other:
clarifai model init --toolkit mcp my-mcp-server         # MCP tool server (FastMCP)
clarifai model init --toolkit openai my-openai-wrapper  # OpenAI-compatible wrapper

# Blank Python template:
clarifai model init

Each toolkit pre-fills model.py with a working ModelClass implementation, config.yaml with reasonable defaults, and requirements.txt. After scaffolding, the next step is to serve the model locally.

Serve Your Model Locally

Start the model using clarifai model serve. The CLI will guide you through creating any required platform resources (compute clusters, nodepools, and deployments) — described below.

• CLI

clarifai model serve

Note: The local-runner command is still supported as an alias for serve.

Once your runner launches successfully, your model will be running and accessible via a public URL. You can then open a new terminal, copy the sample code provided in the output, and test your model!

Use Cases for Local Runners

• Streamlined model development — Local Runners make it easy to build and test new models directly within your local environment. You can spin up a runner on your machine, route API calls through our public cloud endpoint, and watch requests hit your model in real time. This allows you to debug, set breakpoints, return results, and validate outputs.

• Leverage your own compute resources — If you have powerful hardware, you can take advantage of that local compute without relying on Clarifai's autoscaling infrastructure. Your model remains accessible through our API with full authentication, even though it runs locally.

• Locally connect agents — Because Local Runners execute on your chosen hardware, they can interact with local file systems, make OS-level calls, or access private data stores. With our MCP (Model Context Protocol) model type, you can give your cloud-hosted agents or any MCP-enabled clients authenticated access to your locally controlled information, regardless of their deployment location.

• Run models anywhere — Whether on a local development machine, an on-premises server, or a private cloud cluster, Local Runners seamlessly connect your models to our platform. This enables you to keep sensitive data and custom-built models securely within your own environment.

Step 1: Build a Model

Start by building the model you want to run using Local Runners.

You can either create a custom model from scratch or leverage pre-trained models from external sources like Hugging Face.

If you're building your own model, follow our comprehensive step-by-step guide to get started.

You can also explore our examples repository to see models built for compatibility with the Clarifai platform.

You can automatically generate a default model by running the clarifai model init command in the terminal from your current directory. After the model's files are created, you can modify them as needed or go with the default options.

Step 2: Create a Context (Optional)

Running the local development runner relies on certain variables defined in your current context. The context refers to the active environment settings that determine how your commands interact with the Clarifai platform.

Note: You can create this context using the provided default values when you run clarifai login and local-runner commands.

Any configurations you create locally — such as the computer cluster and app — will also be created on the Clarifai platform, making them reusable whenever you test your model with the local development runner.

These are the variables required to create a runner:

The Clarifai CLI follows a defined order of precedence when launching Local Runners. If a value is set as an environment variable in your terminal, it takes priority and is used first.

If no corresponding environment variable is found, the CLI automatically falls back to the value defined in the active context.

Step 3: Run Your Model

To run your model with the local development runner, navigate to the directory where your custom model is located.

Log in to Clarifai

Run the following command to log in to the Clarifai platform and establish a connection.

• CLI

clarifai login

After running the login command, you'll be prompted to enter the following details to authenticate your connection:

• Context name — You can provide a custom name for your Clarifai configuration context, or simply press Enter to use the default name, "default". This helps you manage different configurations if needed.
• User ID — Enter your Clarifai user ID.
• PAT — Enter your Clarifai PAT. Note that if you press Enter, and you have set the CLARIFAI_PAT environment variable, it will use that token automatically.

Serve Your Model

Start the model using clarifai model serve (or the alias local-runner):

• CLI

clarifai model serve

Or with a specific model path:

• CLI

clarifai model serve [OPTIONS] [MODEL_PATH]

Note:MODEL_PATH is an optional path to the model directory. If omitted, the current directory is used by default.

Options

Standalone gRPC Mode

For offline development without a Clarifai account, use --grpc to start a standalone gRPC server:

• CLI

# Start standalone gRPC server (no login required)
clarifai model serve ./my-model --grpc

# Custom port
clarifai model serve ./my-model --grpc --port 9000

This is useful for quick local testing and debugging without needing an internet connection or Clarifai credentials.

Virtual Environment Mode

Use --mode env to automatically create a virtualenv with the model's dependencies:

• CLI

clarifai model serve ./my-model --mode env

This isolates the model's packages from your system Python, preventing dependency conflicts.

Test Your Runner

When the local development runner starts in your terminal, it automatically generates an example client code snippet based on the model's signature.

You can run this snippet in a separate terminal (within the same directory) to receive the model’s response output.

Example Code Snippet

# Clarifai Model Client Script
# Set the environment variables `CLARIFAI_DEPLOYMENT_ID` and `CLARIFAI_PAT` to run this script.
# Example usage:
import os

from clarifai.client import Model
from clarifai.runners.utils import data_types

model = Model(
    "https://clarifai.com/alfrick/local-runner-app/models/local-runner-model",
    deployment_id='local-runner-deployment',  # Only needed for dedicated deployed models
    base_url="https://api.clarifai.com",
)

response = model.predict(
    text1="What is the future of AI?",
)
print(response)

The terminal also displays a link to the AI Playground, which you can copy and use to interact with the model directly.

When you’re done testing, simply close the terminal running the local development runner to shut it down.

Additional Examples

• Simple example for running Hello World model locally with Clarifai’s Local Runners
• YouTube video on how to build your own local AI agent using Clarifai’s Local Runners