Run inferences on Clarifai models using Vercel

Vercel offers a TypeScript toolkit called the AI SDK, which streamlines integration with language models in modern web applications.

The SDK supports the OpenAI Compatible Provider package that enables seamless interaction with any OpenAI-compatible API — including Clarifai’s OpenAI-compatible endpoint.

Prerequisites

Install Package

Vercel offers a Clarifai provider, which is available through the @ai-sdk/openai-compatible module and is compatible with the OpenAI API. You can install it using:

• pnpm
• npm
• yarn

pnpm add @ai-sdk/openai-compatible

npm install @ai-sdk/openai-compatible

yarn add @ai-sdk/openai-compatible

Get a PAT Key

You need a PAT (Personal Access Token) key to authenticate your connection to the Clarifai platform. You can get one by navigating to Settings in the collapsible left sidebar, selecting Secrets, and creating or copying an existing token from there.

You can then set the PAT as an environment variable using CLARIFAI_PAT:

• Unix-Like Systems
• Windows

export CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE

set CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE

Get a Clarifai Model

Go to the Clarifai Community platform and select the model you want to use for making predictions.

Some Clarifai models that support Vercel AI SDK, and their capabilities

Create Provider Instance

To use Clarifai with Vercel for inference, create a custom provider instance using the createOpenAICompatible function from @ai-sdk/openai-compatible.

• TypeScript

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';

const clarifai = createOpenAICompatible({
  name: 'clarifai',
  baseURL: 'https://api.clarifai.com/v2/ext/openai/v1',
  apiKey: process.env.CLARIFAI_PAT,
});

Generating Text

Here's an example of how you can generate text using the Vercel AI SDK with a Clarifai model.

• TypeScript

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { generateText } from 'ai';

const clarifai = createOpenAICompatible({
  name: 'clarifai',
  baseURL: 'https://api.clarifai.com/v2/ext/openai/v1',
  apiKey: process.env.CLARIFAI_PAT,
});

const model = clarifai.chatModel(
  'https://clarifai.com/openai/chat-completion/models/gpt-oss-120b',
);

const { text, usage, finishReason } = await generateText({
  model,
  prompt: 'Give one fun fact about photosynthesis',
});

console.log(text);
console.log('Token usage:', usage);
console.log('Finish reason:', finishReason);

Example Output

**Fun fact:** Some plants can “talk” to each other through the air—when they’re stressed (e.g., attacked by insects), they release volatile organic compounds that neighboring plants detect and use to ramp up their own photosynthetic defenses, like producing extra protective pigments. In a sense, they’re sending a chemical SOS to help each other stay green!

Streaming

You can use the Vercel AI SDK to stream responses with a Clarifai model.

• TypeScript

import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
import { streamText } from 'ai';

const clarifai = createOpenAICompatible({
  name: 'clarifai',
  baseURL: 'https://api.clarifai.com/v2/ext/openai/v1',
  apiKey: process.env.CLARIFAI_PAT,
});

const model = clarifai.chatModel(
  'https://clarifai.com/openai/chat-completion/models/gpt-oss-120b',
);

const result = streamText({
  model,
  prompt: 'Give one fun fact about photosynthesis',
});

for await (const message of result.textStream) {
  console.log(message);
}

Tool Calling

The Vercel AI SDK supports tool calling.

Note: To run this example, install the Zod validation library: npm install zod.

• TypeScript

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText, tool } from "ai";
import { z } from "zod";

const clarifai = createOpenAICompatible({
  baseURL: "https://api.clarifai.com/v2/ext/openai/v1",
  apiKey: process.env.CLARIFAI_PAT,
});

const model = clarifai(
  "https://clarifai.com/openai/chat-completion/models/gpt-oss-120b"
);

const result = await generateText({
  model,
  tools: {
    weather: tool({
      description: "Get the current weather in a specific location",
      parameters: z.object({
        location: z.string().describe("The city and state, e.g. San Francisco, CA"),
      }),
      execute: async ({ location }) => ({
        location,
        temperature: 72,
      }),
    }),
    cityAttractions: tool({
      description: "Get popular tourist attractions for a specific city",
      parameters: z.object({
        city: z.string().describe("The name of the city")
      }),
      execute: async ({ city }) => ({
        city,
        attractions: ["Golden Gate Bridge", "Alcatraz", "Fisherman's Wharf"],
      }),
    }),
  },
  prompt: "What is the weather in San Francisco and what attractions should I visit?",
});

console.log(JSON.stringify(result.toolResults, null, 2));

Build your next AI app, test and tune popular LLMs models, and much more.