Create and Manage Your Inputs | Clarifai Docs

Manage your inputs and training data on the Clarifai platform

Clarifai provides an easy-to-use UI that lets you explore your data and maximize the performance of your models. The platform offers several features to help you effectively manage and optimize your machine learning projects, including:

Data exploration: You can upload, visualize, and organize your datasets, including images, videos, and text. The UI provides tools for searching and filtering data to better understand and manage the information.
Annotation: The interface includes robust labeling tools, enabling you to prepare your data for training. This includes adding labels, drawing bounding boxes, and applying other relevant annotations.

Inputs-Manager Screen

If you click the Inputs option in your app's collapsible left sidebar, you'll be redirected to the Inputs-Manager screen, which is the main page showcasing all the inputs in your app.

Note: An input is any data added to the Clarifai platform.

On this screen, you can:

Perform a smart search: Use the search field to rank, sort, and retrieve inputs based on a predicted match to your query caption text. Learn more about it here.
Upload inputs: Add new inputs to your app by clicking the Upload Inputs button in the upper-right section of the page. Learn more about it here.
Filter inputs: Use the Input Filters section to customize how inputs are displayed. By default, inputs are sorted by creation date, with the newest at the top. You can filter inputs by dataset, labels, input type (audio, image, text, or video), or metadata. The number of inputs matching your criteria will be displayed at the top.
Zoom in/out: Adjust the zoom level of the displayed inputs to suit your preference.
Toggle layout: Toggle the layout of the displayed items.

Input Mode

The Input Mode is the default mode of the Inputs-Manager screen. It displays a gallery showcasing all inputs in your app. Image inputs are shown as thumbnails, while text inputs are truncated.

When you hover over an input, icons will appear in the left corner of the highlighted input, allowing you to:

Click the small empty box to select the input.
Click the magnifying glass to perform a smart search by similarity. This lets you rank, sort, and retrieve inputs based on their similarity to a provided query input. Learn more about it here.
Click the info button to see details of the input, such as input ID, creation date, and resolution.
Click the delete button to remove the input from your app.

When you select an input by clicking the small empty box in its upper-left corner, a section will appear at the bottom of the page, allowing you to:

Click the Label as… button to add or remove annotations from the selected input. Learn more about it here.
Click the Edit Metadata button to add or remove metadata information associated with the selected input.
Click the Dataset… button to add or remove the selected input from a dataset.
Click the Delete button to remove the selected input from your app.

Object Mode

The Object Mode is a gallery that shows the objects that have been labeled on your inputs. You can select it by clicking the Object Mode button in the upper-right section of the Inputs-Manager screen.

If you hover over an input and select it by clicking the small empty box in its upper-left corner, several options will appear, allowing you to perform various management tasks.

The magnifying glass icon appearing in the left side of the input allows you to use it to perform Smart Object Search and retrieve annotated objects within other inputs based on their content and similarity.
The delete icon allows you to delete the input from your app.
The Add Concepts button that appears at the bottom of the page allows you to create new object annotations with the selected concepts.
The Remove Annotations button that appears at the bottom of the page allows you to remove the selected annotation from the input.

Additionally, while in Object Mode, you can click the ellipsis next to the Labels section to reveal various options for filtering the display of your inputs. You can filter the display by: Operator — or or and, Labeler, or Status — Success, Awaiting Review, Awaiting Consensus Review, or Pending.

Inputs and Outputs Guide

Example

When choosing one of Clarifai's pre-built models, you might see something like this from some of our models:

Input Type	Output Type
image	regions[...].data.concepts, regions[...].region_info.bounding_box

These inputs and outputs can be clarified with the following table explaining these data types:

Table of Uploadable Data Types

Data Type	Meaning
text	This is freeform plain text which can be uploaded via raw text or specified with a URI.
image	This is an image in an accepted format, which currently includes JPG, PNG, TIFF, BMP, WEBP, CSV, and TSV. It can be uploaded via base64 bytes or specified with a URI.
video	This is video in an accepted format, which currently includes AVI, MP4, WMV, MOV, GIF, and 3GPP. It can be uploaded via base64 bytes or specified with a URI.

All these data formats are read in as raw bytes in base64 format.

Table of Single Data Types Passed Between Models

Data Type	Meaning
embeddings	Vector representations of data passed from model to model. These are not uploadable by users.
clusters	These are IDs that identify clusters. These are primarily used for image search.
concepts	The list of concepts used in a model. For the general model, these would be the top 20 concepts with classified with the highest confidence.

Table of `regions[...]` Data Types

The notation of [...] means that the variable is a list of things, so regions[...] represents a list of regions of data. This could be parts of an image, text, video, or audio:

Data Type	Meaning
regions[...].region_info.point	This is a list of points which specify regions of an image.
regions[...].region_info.bounding_box	This is a list of regions each containing the four corners of a bounding box in a specific region of an image. Each corner coordinate is normalized to [0,1].
regions[...].region_info.mask	The mask is an overlay of the entire image, with the specific concepts pixels set to a certain color.
regions[...].data.text	This is a list of regions and their associated text. This could be OCR data for an image, or subtext within a larger text for NLP.
regions[...].data.embeddings	This is a list of regions and their associated vector representations.
regions[...].data.concepts	This is a list of regions and their associated or high confidence concepts.

Table of `frames[...]` Data Types

The notation of [...] means that the variable is a list of things, so frames[...] represents a list of frames of video or audio, and therefore frames[...].data.regions[...] represents a 2D matrix of the number of frames by the number of regions in each frame:

Data Type	Meaning
frames[...].data.regions[...].region_info.bounding_box	These are the four corners of a bounding box in a specific region of a specific frame of video. Each corner coordinate is normalized to [0,1].
frames[...].data.regions[...].data.concepts	This is the matrix of frames and regions containing the concepts used in a model. For the general model, these would be the top 20 concepts classified with the highest confidence in a specific region of a specific frame of video.
frames[...].data.regions[...].track_id	This is the matrix of frames and regions containing the tracking ids used to track objects across frames of a video.