OpenAI | Clarifai Docs

Run inferences on Clarifai models using OpenAI

You can run inferences on Clarifai-hosted models using the OpenAI client library by leveraging the Clarifai’s OpenAI-compatible API endpoint.

This allows you to use the same code and tools you would with OpenAI, in either Python or JavaScript, by simply configuring the client to point to Clarifai and providing your PAT (Personal Access Token) key.

Prerequisites

Install OpenAI Package

Install the openai package.

Python
Node.js

 pip install openai

 npm install openai

Get a PAT Key

You need a Personal Access Token (PAT) key to authenticate your connection to the Clarifai platform. You can get one by navigating to Settings in the collapsible left sidebar, selecting Secrets, and creating or copying an existing token from there.

You can then set the PAT as an environment variable using CLARIFAI_PAT:

Unix-Like Systems
Windows

 export CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE

 set CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE

Get a Clarifai Model

Go to the Clarifai Community platform and select the model you want to use for making predictions.

Some Clarifai models that support OpenAI

https://clarifai.com/openai/chat-completion/models/gpt-oss-120b
https://clarifai.com/deepseek-ai/deepseek-chat/models/DeepSeek-R1-0528-Qwen3-8B
https://clarifai.com/meta/Llama-3/models/Llama-3_2-3B-Instruct
https://clarifai.com/anthropic/completion/models/claude-sonnet-4
https://clarifai.com/qwen/qwenLM/models/Qwen3-14B
https://clarifai.com/mistralai/completion/models/Devstral-Small-2505_gguf-4bit
https://clarifai.com/clarifai/main/models/general-image-recognition
https://clarifai.com/xai/chat-completion/models/grok-3
https://clarifai.com/openai/chat-completion/models/gpt-4o
https://clarifai.com/openai/chat-completion/models/gpt-4_1
https://clarifai.com/gcp/generate/models/gemini-2_5-flash
https://clarifai.com/anthropic/completion/models/claude-3_5-haiku
https://clarifai.com/qwen/qwenLM/models/Qwen3-30B-A3B-GGUF
https://clarifai.com/gcp/generate/models/gemini-2_0-flash
https://clarifai.com/gcp/generate/models/gemma-3-12b-it
https://clarifai.com/microsoft/text-generation/models/Phi-4-reasoning-plus
https://clarifai.com/openbmb/miniCPM/models/MiniCPM3-4B
https://clarifai.com/microsoft/text-generation/models/phi-4-mini-instruct
https://clarifai.com/qwen/qwen-VL/models/Qwen2_5-VL-7B-Instruct
https://clarifai.com/microsoft/text-generation/models/phi-4
https://clarifai.com/xai/chat-completion/models/grok-2-vision-1212
https://clarifai.com/xai/image-generation/models/grok-2-image-1212
https://clarifai.com/xai/chat-completion/models/grok-2-1212
https://clarifai.com/qwen/qwenLM/models/QwQ-32B-AWQ
https://clarifai.com/gcp/generate/models/gemini-2_0-flash-lite
https://clarifai.com/anthropic/completion/models/claude-opus-4
https://clarifai.com/openai/chat-completion/models/o4-mini
https://clarifai.com/openai/chat-completion/models/o3
https://clarifai.com/openbmb/miniCPM/models/MiniCPM-o-2_6-language
https://clarifai.com/deepseek-ai/deepseek-chat/models/DeepSeek-R1-Distill-Qwen-7B
https://clarifai.com/qwen/qwenCoder/models/Qwen2_5-Coder-7B-Instruct

Responses API

The OpenAI Responses API endpoint provides a powerful interface for generating model responses, allowing you to leverage OpenAI's most advanced capabilities. It’s highly versatile, supporting both text and image inputs and producing text outputs, as well as advanced features like streaming and tool calling.

Text Inputs

Python

import os
from openai import OpenAI

# Initialize the OpenAI client, pointing to Clarifai's API
client = OpenAI(
    base_url="https://api.clarifai.com/v2/ext/openai/v1",  # Clarifai's OpenAI-compatible API endpoint
    api_key=os.environ["CLARIFAI_PAT"]  # Ensure CLARIFAI_PAT is set as an environment variable
)

# Make a response request to a Clarifai-hosted model
response = client.responses.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    input="Tell me a three sentence bedtime story about a unicorn.",
    max_output_tokens=100,
    temperature=0.7,
)

# Print the model's response
print(response.output_text)

Example Output

Under a silver‑shimmering moon, a gentle unicorn named Lira tiptoed into the sleepy meadow, her horn casting soft, glittering lullabies over the swaying wildflowers. As the night wind whispered sweet dreams, she gathered a handful of moon‑kissed dew and sprinkled it over the slumbering forest creatures, coax

Chat Completions API

The OpenAI Chat Completions API endpoint enables you to generate a model response by providing a list of messages that constitute a conversation.

Supported Message Roles

Clarifai's LLM endpoint supports the following message roles: system, user, assistant, developer, and tool. The developer role is used for developer-level instructions (similar to system), while tool is used for passing back the results of tool calls during agentic interactions.

Text Inputs

Python

import os
from openai import OpenAI

# Make a chat completion request to a Clarifai-hosted model
response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who are you?"}
    ],
    max_completion_tokens=100,
    temperature=0.7,
)

# Print the model's response
print(response.choices[0].message.content)

Multimodal Inputs

The OpenAI Chat Completions API supports multimodal inputs.

Python

import os
import requests
import base64
from pathlib import Path
from openai import OpenAI

# Option 1: Use a local image file
# path = "local/path/of/image.png"
# image_base64 = base64.b64encode(Path(path).read_bytes()).decode()

# Option 2: Download image from URL and convert to base64
def get_image_base64(image_url):
    """Download image and convert it to a base64-encoded string."""
    response = requests.get(image_url)
    return base64.b64encode(response.content).decode("utf-8")

image_url = "https://samples.clarifai.com/cat1.jpeg"
image_base64 = get_image_base64(image_url)

# Initialize OpenAI client using Clarifai's OpenAI-compatible endpoint
client = OpenAI(
    base_url="https://api.clarifai.com/v2/ext/openai/v1",
    api_key=os.environ["CLARIFAI_PAT"],
)

response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-4o",
    messages=[
        {"role": "user", "content": [
            {"type": "text", "text": "Describe the image"},
            {
                "type": "image_url",
                "image_url": {
                    "url": f"data:image/jpeg;base64,{image_base64}"
                }
            }
        ]}
    ],
    temperature=0.7,
    max_tokens=1024
)

# Output the result
print(f"Response: {response.choices[0].message.content}")

Example Output

Response: The image shows a ginger tabby cat lying down on a stone surface. The cat has a striped pattern on its fur and is looking directly at the camera with alert, bright eyes. The background features a textured wall, and the lighting highlights the cat's features, giving a warm and cozy atmosphere.

Streaming

The OpenAI Chat Completions API supports streaming.

Python

import os
from openai import OpenAI

# Make a chat completion request to a Clarifai-hosted model
response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who are you?"}
    ],
    max_completion_tokens=100,
    temperature=0.7,
    stream=True  # Enables streaming the response token by token
)

print("Assistant's Response:")
for chunk in response:
    if chunk.choices and 
       chunk.choices[0].delta and 
       chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end='')
print("\n")

Tool Calling

The OpenAI Chat Completions API supports tool calling (also known as function calling). Here is an example code that sets up a basic tool-calling interaction. It simulates a weather API and shows how the LLM would "call" that tool when asked about the weather.

Python

import os
from openai import OpenAI

# Initialize the OpenAI-compatible client for Clarifai
client = OpenAI(
    base_url="https://api.clarifai.com/v2/ext/openai/v1",
    api_key=os.environ["CLARIFAI_PAT"]  # Ensure CLARIFAI_PAT is set as an environment variable
)

# Define the external tools (functions) that the LLM can call.
# In this example, it's a 'get_weather' function.
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Returns the current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country, e.g., 'Bogotá, Colombia'"
                }
            },
            "required": ["location"],
            "additionalProperties": False
        }
    }
}]

# Create a chat completion request with tool-calling enabled
response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    messages=[
        {"role": "user", "content": "What is the weather like in New York today?"}
    ],
    tools=tools,
    tool_choice='auto'
)

# Print the tool call proposed by the model, if any
tool_calls = response.choices[0].message.tool_calls
print("Tool calls:", tool_calls)

Structured Outputs

The Chat Completions API supports generating structured JSON outputs from any supported OpenAI-compatible large language model. This feature lets you go beyond plain text generation by enforcing that the model’s output follows a strict, predictable data structure. Structured responses make it easier to integrate model outputs into downstream applications with greater safety, consistency, and reliability.

To define, validate, and serialize these structured responses, Pydantic is used — a powerful data modeling library for Python.