Set up capabilities that match your computational needs

A compute cluster serves as the main environment where models are deployed, whether for training or inference. Each cluster can contain multiple nodepools, which are groups of virtual machine instances with similar configurations (such as CPU/GPU type, memory).

After creating a custom cluster, you can configure nodepools within it to optimize resource usage. These nodepools will help tailor the infrastructure to meet the specific hardware, performance, cost, or regulatory compliance of your machine learning needs.

For example, you may create a nodepool for GPU-intensive tasks and another for lighter workloads running on CPUs.

With clusters and nodepools, you can organize and manage ( orchestrate) the compute resources necessary for running your models and workflows.

Connect Your Own Cloud

You can connect your existing AWS, Google Cloud (GCP), or Oracle infrastructure to leverage your current investments. This lets you maintain full control over your data while optimizing for cost and flexibility.

Via the UI

Step 1: Start Creating a Cluster

Log in to the Clarifai platform and select the Compute option in the collapsible left sidebar.

Then, select the Clusters option from the dropdown list. You’ll be redirected to the clusters page, where you can view and manage all compute clusters across your environments.

Click the Create a Cluster button to begin setting up a new cluster along with its associated nodepool.

Step 2: Select an Instance

Select an instance type that aligns with your specific requirements. The table displays essential details for each instance type, which helps you make an informed decision.

Tips

See Supported Cloud Instances to learn more about the instance types we provide.

You can narrow down the displayed options using the filters and tools provided:

• Search bar — Quickly find instance types by name.
• Provider filter — Choose from available cloud providers to match your preferred infrastructure.
• Region filter — Select the geographic location for the instance.
• Instance type filter — Directly filter for a specific instance type.
• Hardware filter — Filter based on the instance's available hardware.
• Price/hour filter — Filter instances by their hourly cost.
• Sorting controls — Sort values in ascending or descending order.

Step 3: Set Node Autoscaling Range

Define the minimum and maximum number of nodes your nodepool can scale to based on workload demand. This ensures your system automatically adjusts its capacity by adding nodes during high traffic and scaling down during low usage.

Step 4: Enable Spot Instances

You can enable this option if you want to rent spare, unused compute capacity at significantly lower prices compared to regular on-demand instances.

Step 5: Provide Cluster and Nodepool Details

Fill out the form to specify the details for your cluster and nodepool.

• Personal Access Token (PAT) — Use a PAT to authenticate your identity when connecting to the cluster.
• Cluster ID — Enter a unique identifier for your cluster.
• Nodepool ID — Enter a unique identifier for your nodepool.

Step 6: Finalize and Create the Cluster

Before you enter the required details for creating a cluster, the Create Cluster button will be disabled. After providing the details, the button will become active.

Click it to launch your cluster and nodepool. You'll then be redirected to your newly created cluster's page, where the details of its associated nodepool will also be displayed.

Via the API

Prerequisites

Installation

To begin, install the latest version of the clarifai Python package. This will also install the Clarifai Command Line Interface.

pip install --upgrade clarifai

Get a PAT

You need a PAT (Personal Access Token) key to authenticate with the Clarifai platform. After generating it, set the token as an environment variable.

• Unix-Like Systems

 export CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE

• Windows

 set CLARIFAI_PAT=YOUR_PERSONAL_ACCESS_TOKEN_HERE

Set up Project Directory

• Create an overarching directory to store your project files.
• Inside this directory, create a Python file for your Compute Orchestration code.
• Create a configs folder to store your YAML configuration files for clusters, nodepools, and deployments.

Here’s the structure of the directory:

project-directory/               # Overarching project folder
│
├── compute_orchestration.py     # Python file for orchestration code
│
└── configs/                     # Folder for configuration files
    ├── compute_cluster_config.yaml
    ├── nodepool_config.yaml
    └── deployment_config.yaml

Example Configurations

Here are example YAML configurations for compute cluster, nodepool, and deployment settings.

compute_cluster_config.yaml

compute_cluster:
  id: "test-compute-cluster"
  description: "My AWS compute cluster"
  cloud_provider:
      id: "aws"
  region: "us-east-1"
  managed_by: "clarifai"
  cluster_type: "dedicated"
  visibility:
    gettable: 10

nodepool_config.yaml

nodepool:
  id: "test-nodepool"
  compute_cluster:
    id: "test-compute-cluster"
  description: "First nodepool in AWS in a proper compute cluster"
  instance_types:
    - id: "g5.2xlarge"
      compute_info:
        cpu_limit: "8"
        cpu_memory: "28Gi"
        accelerator_type:
          - "a10"
        num_accelerators: 1
        accelerator_memory: "40Gi"
  node_capacity_type:
    capacity_types:
      - 1
  min_instances: 0
  max_instances: 1

deployment_config.yaml

deployment:
 id: "test-deployment"
 description: "some random deployment"
 autoscale_config:
   min_replicas: 0
   max_replicas: 5
   traffic_history_seconds: 300
   scale_down_delay_seconds: 300
   scale_to_zero_delay_seconds: 1800
   scale_up_delay_seconds: 300
   disable_packing: false
 worker:
   model:
     id: "Llama-3_2-3B-Instruct"
     model_version:
       id: "fe271b43266a45a5b068766b6437687f"
     user_id: "meta"
     app_id: "Llama-3"
 scheduling_choice: 4
 nodepools:
   - id: "test-nodepool"
     compute_cluster:
         id: "test-compute-cluster"

Create a Cluster

To create a new compute cluster, pass the compute_cluster_id and config_filepath arguments to the create_compute_cluster method of the User class.

Here's an example using Python SDK:

from clarifai.client.user import User

client = User(
    user_id="YOUR_USER_ID_HERE"
)

compute_cluster = client.create_compute_cluster(
    compute_cluster_id="test-compute-cluster",
    config_filepath="./configs/compute_cluster_config.yaml"
)