Model Inference | Clarifai Docs
Perform predictions with models
Clarifai’s Compute Orchestration capabilities enable you to make prediction calls efficiently, tailored to a variety of use cases. You can use these features to run inferences seamlessly and get results from your model with ease.
Predict With Compute Orchestration
To make a prediction request using our Compute Orchestration capabilities, you'll need to set up a cluster, create a nodepool, and deploy your model to it.
Once deployed, you’ll need to reference the model’s deployment_id — or alternatively, specify both the compute_cluster_id and nodepool_id. These parameters ensure that your prediction request is routed correctly to the intended compute resources.
For example, you can route requests to a GCP cluster by selecting a corresponding deployment ID, use a different deployment ID for an AWS cluster, and yet another for an on-premises deployment.
This gives you full control over performance, costs, and security, allowing you to focus on building cutting-edge AI solutions while we handle the infrastructure complexity.
Learn how Clarifai routes prediction requests for optimal performance here.
Structure of Prediction Methods
Supported Input and Output Data Types
Click here to explore the wide range of input and output data types supported by Clarifai’s model framework. You'll also find client-side examples that show how to work with these rich data formats effectively.
Clarifai models are mostly built using three primary files: model.py, requirements.txt, and config.yaml. As described here, the core prediction logic resides in model.py, which defines how your model processes inputs and generates outputs.
When making predictions on the client side, the structure of the prediction methods directly reflects the method signatures defined in the model.py file. This one-to-one mapping allows you to make custom predictions with flexible naming and argument structures, giving you full control over how you invoke models.
Here are some examples of this method mapping approach:
model.py Model Implementation |
Client-Side Usage Pattern |
|---|---|
@ModelClass.method def predict(...) |
model.predict(...) |
@ModelClass.method def generate(...) |
model.generate(...) |
@ModelClass.method def stream(...) |
model.stream(...) |
This design provides flexibility in how to make model predictions. For example, a method could be defined as @ModelClass.method def analyze_video(...) in model.py, and then you can call it on the client side using model.analyze_video(...).
Here are some key characteristics of this design:
- Method names must match exactly between
model.pyand client usage. - Parameters retain the same names and types as defined in the method.
- Return types follow the structure defined by the model's outputs.
Predict With OpenAI-Compatible Format
Clarifai supports various providers that you can use to interact with different models. We offer an OpenAI-compatible API endpoint, allowing any OpenAI-compatible library or client to seamlessly send requests directly to Clarifai.
Base URL for Clarifai's OpenAI endpoint: https://api.clarifai.com/v2/ext/openai/v1.
This integration capability offers several advantages, including:
- Access to diverse models — Harness Clarifai's rich array of models directly within your OpenAI projects, expanding your AI capabilities.
- Standardized interaction — Interact with Clarifai-hosted models using familiar OpenAI API patterns and interfaces, reducing the learning curve and streamlining development.
- Enhanced flexibility — Leverage the power of Clarifai's platform while maintaining the flexibility of your chosen OpenAI development environment.
Usage-based billing is handled directly through Clarifai — not through OpenAI or any other provider. Also, while most OpenAI parameters are supported, certain advanced features may be unavailable depending on the specific model or endpoint.