# Run inferences on Clarifai models using OpenAI

You can run inferences on Clarifai-hosted models using the OpenAI client library by leveraging the Clarifai’s [OpenAI-compatible API endpoint](https://docs.clarifai.com/compute/inference/#predict-with-openai-compatible-format).

This allows you to use the same code and tools you would with OpenAI, in either Python or JavaScript, by simply configuring the client to point to Clarifai and providing your PAT (Personal Access Token) key.

## Prerequisites
### Install OpenAI Package
Install the `openai` package.
- Python
- Node.js

```bash
 pip install openai
```

```bash
 npm install openai
```

### Get a PAT Key
You need a [Personal Access Token (PAT)](https://docs.clarifai.com/control/authentication/pat) key to authenticate your connection to the Clarifai platform. You can get one by navigating to **Settings** in the collapsible left sidebar, selecting **Secrets**, and creating or copying an existing token from there.

You can then set the PAT as an environment variable using `CLARIFAI_PAT`:
- Unix-Like Systems
- Windows

```bash
 export CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE
```

```bash
 set CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE
```

### Get a Clarifai Model
Go to the Clarifai [Community](/content/explore/index.html) platform and select the model you want to use for making predictions.

Some Clarifai models that support OpenAI

```python
https://clarifai.com/openai/chat-completion/models/gpt-oss-120b
https://clarifai.com/deepseek-ai/deepseek-chat/models/DeepSeek-R1-0528-Qwen3-8B
https://clarifai.com/meta/Llama-3/models/Llama-3_2-3B-Instruct
https://clarifai.com/anthropic/completion/models/claude-sonnet-4
https://clarifai.com/qwen/qwenLM/models/Qwen3-14B
https://clarifai.com/mistralai/completion/models/Devstral-Small-2505_gguf-4bit
https://clarifai.com/clarifai/main/models/general-image-recognition
https://clarifai.com/xai/chat-completion/models/grok-3
https://clarifai.com/openai/chat-completion/models/gpt-4o
https://clarifai.com/openai/chat-completion/models/gpt-4_1
https://clarifai.com/gcp/generate/models/gemini-2_5-flash
https://clarifai.com/anthropic/completion/models/claude-3_5-haiku
https://clarifai.com/qwen/qwenLM/models/Qwen3-30B-A3B-GGUF
https://clarifai.com/gcp/generate/models/gemini-2_0-flash
https://clarifai.com/gcp/generate/models/gemma-3-12b-it
https://clarifai.com/microsoft/text-generation/models/Phi-4-reasoning-plus
https://clarifai.com/openbmb/miniCPM/models/MiniCPM3-4B
https://clarifai.com/microsoft/text-generation/models/phi-4-mini-instruct
https://clarifai.com/qwen/qwen-VL/models/Qwen2_5-VL-7B-Instruct
https://clarifai.com/microsoft/text-generation/models/phi-4
https://clarifai.com/xai/chat-completion/models/grok-2-vision-1212
https://clarifai.com/xai/image-generation/models/grok-2-image-1212
https://clarifai.com/xai/chat-completion/models/grok-2-1212
https://clarifai.com/qwen/qwenLM/models/QwQ-32B-AWQ
https://clarifai.com/gcp/generate/models/gemini-2_0-flash-lite
https://clarifai.com/anthropic/completion/models/claude-opus-4
https://clarifai.com/openai/chat-completion/models/o4-mini
https://clarifai.com/openai/chat-completion/models/o3
https://clarifai.com/openbmb/miniCPM/models/MiniCPM-o-2_6-language
https://clarifai.com/deepseek-ai/deepseek-chat/models/DeepSeek-R1-Distill-Qwen-7B
https://clarifai.com/qwen/qwenCoder/models/Qwen2_5-Coder-7B-Instruct
```

## Responses API
The [OpenAI Responses](https://platform.openai.com/docs/api-reference/responses) API endpoint provides a powerful interface for generating model responses, allowing you to leverage OpenAI's most advanced capabilities.
It’s highly versatile, supporting both text and image inputs and producing text outputs, as well as advanced features like streaming and tool calling.

### Text Inputs
- Python

```python
import os
from openai import OpenAI

# Initialize the OpenAI client, pointing to Clarifai's API
client = OpenAI(
    base_url="https://api.clarifai.com/v2/ext/openai/v1",  # Clarifai's OpenAI-compatible API endpoint
    api_key=os.environ["CLARIFAI_PAT"]  # Ensure CLARIFAI_PAT is set as an environment variable
)

# Make a response request to a Clarifai-hosted model
response = client.responses.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    input="Tell me a three sentence bedtime story about a unicorn.",
    max_output_tokens=100,
    temperature=0.7,
)

# Print the model's response
print(response.output_text)
```

Example Output

```text
Under a silver‑shimmering moon, a gentle unicorn named Lira tiptoed into the sleepy meadow, her horn casting soft, glittering lullabies over the swaying wildflowers. As the night wind whispered sweet dreams, she gathered a handful of moon‑kissed dew and sprinkled it over the slumbering forest creatures, coax
```

## Chat Completions API
The [OpenAI Chat Completions](https://platform.openai.com/docs/api-reference/chat) API endpoint enables you to generate a model response by providing a list of messages that constitute a conversation.

Supported Message Roles

Clarifai's LLM endpoint supports the following message roles: `system`, `user`, `assistant`, `developer`, and `tool`. The `developer` role is used for developer-level instructions (similar to `system`), while `tool` is used for passing back the results of tool calls during agentic interactions.

### Text Inputs
- Python

```python
import os
from openai import OpenAI

# Make a chat completion request to a Clarifai-hosted model
response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who are you?"}
    ],
    max_completion_tokens=100,
    temperature=0.7,
)

# Print the model's response
print(response.choices[0].message.content)
```

## Multimodal Inputs
The OpenAI Chat Completions API supports multimodal inputs.
- Python

```python
import os
import requests
import base64
from pathlib import Path
from openai import OpenAI

# Option 1: Use a local image file
# path = "local/path/of/image.png"
# image_base64 = base64.b64encode(Path(path).read_bytes()).decode()

# Option 2: Download image from URL and convert to base64
def get_image_base64(image_url):
    """Download image and convert it to a base64-encoded string."""
    response = requests.get(image_url)
    return base64.b64encode(response.content).decode("utf-8")

image_url = "https://samples.clarifai.com/cat1.jpeg"
image_base64 = get_image_base64(image_url)

# Initialize OpenAI client using Clarifai's OpenAI-compatible endpoint
client = OpenAI(
    base_url="https://api.clarifai.com/v2/ext/openai/v1",
    api_key=os.environ["CLARIFAI_PAT"],
)

response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-4o",
    messages=[
        {"role": "user", "content": [
            {"type": "text", "text": "Describe the image"},
            {
                "type": "image_url",
                "image_url": {
                    "url": f"data:image/jpeg;base64,{image_base64}"
                }
            }
        ]}
    ],
    temperature=0.7,
    max_tokens=1024
)

# Output the result
print(f"Response: {response.choices[0].message.content}")
```

Example Output

```text
Response: The image shows a ginger tabby cat lying down on a stone surface. The cat has a striped pattern on its fur and is looking directly at the camera with alert, bright eyes. The background features a textured wall, and the lighting highlights the cat's features, giving a warm and cozy atmosphere.
```

### Streaming
The OpenAI Chat Completions API supports streaming.
- Python

```python
import os
from openai import OpenAI

# Make a chat completion request to a Clarifai-hosted model
response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Who are you?"}
    ],
    max_completion_tokens=100,
    temperature=0.7,
    stream=True  # Enables streaming the response token by token
)

print("Assistant's Response:")
for chunk in response:
    if chunk.choices and 
       chunk.choices[0].delta and 
       chunk.choices[0].delta.content is not None:
        print(chunk.choices[0].delta.content, end='')
print("\n")
```

### Tool Calling
The OpenAI Chat Completions API supports tool calling (also known as function calling).
Here is an example code that sets up a basic tool-calling interaction. It simulates a weather API and shows how the LLM would "call" that tool when asked about the weather.
- Python

```python
import os
from openai import OpenAI

# Initialize the OpenAI-compatible client for Clarifai
client = OpenAI(
    base_url="https://api.clarifai.com/v2/ext/openai/v1",
    api_key=os.environ["CLARIFAI_PAT"]  # Ensure CLARIFAI_PAT is set as an environment variable
)

# Define the external tools (functions) that the LLM can call.
# In this example, it's a 'get_weather' function.
tools = [{
    "type": "function",
    "function": {
        "name": "get_weather",
        "description": "Returns the current temperature for a given location.",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and country, e.g., 'Bogotá, Colombia'"
                }
            },
            "required": ["location"],
            "additionalProperties": False
        }
    }
}]

# Create a chat completion request with tool-calling enabled
response = client.chat.completions.create(
    model="https://clarifai.com/openai/chat-completion/models/gpt-oss-120b",
    messages=[
        {"role": "user", "content": "What is the weather like in New York today?"}
    ],
    tools=tools,
    tool_choice='auto'
)

# Print the tool call proposed by the model, if any
tool_calls = response.choices[0].message.tool_calls
print("Tool calls:", tool_calls)
```

## Structured Outputs
The Chat Completions API supports generating structured JSON outputs from any supported OpenAI-compatible large language model. This feature lets you go beyond plain text generation by enforcing that the model’s output follows a strict, predictable data structure. Structured responses make it easier to integrate model outputs into downstream applications with greater safety, consistency, and reliability.

To define, validate, and serialize these structured responses, Pydantic is used — a powerful data modeling library for Python.