Automatically generate annotations on Input-Viewer page

AI-assisted labeling is an innovative Clarifai feature that leverages artificial intelligence technology to assist and optimize the process of annotating data.

You can request predictions from any model or workflow available to you on a particular input, and then review, correct, or validate them before converting them into annotations.

AI-Assist provides you with several benefits, including:

• Significantly accelerate the labeling process, reducing the time required to create labeled datasets;
• Automation can reduce the labor costs associated with manual labeling;
• AI models can provide consistent labeling, minimizing human errors;
• It allows for the efficient and scalable handling of large datasets or rapidly changing data.

feature availability

This AI-assist feature is only available to users on our paid plans.

Let’s demonstrate how you can perform AI-assisted labeling using the Annotate mode of the Input-Viewer screen.

1. Activate AI-Assist Settings

On the Input-Viewer, click the Edit button located in the upper-right corner of the page.

You'll be redirected to the AI-assist sidebar that enables you to choose a model or workflow for using in labeling your inputs. Ensure the AI Assist toggle is switched on.

2. Choose a Model or Workflow

Use the Select Model or Workflow search box to choose a model or workflow to generate predictions and assist with annotations. You can choose your own customized model or workflow, or look for a public one from the Community platform.

To select a public model or workflow from the Community, click the Explore Community Models / Workflows button. In the pop-up window, use the search bar to find the desired model or workflow.

Classifications Labeling

First, let’s choose the Community’s general-image-recognition model, which is a visual classification model that identifies a variety of concepts in images.

Objects Labeling

Similarly, let’s select another input on the Input-Viewer screen. And on the AI Assist sidebar, let’s choose the Community’s General-Detection workflow, which identifies a variety of common objects in images.

3. Generate Annotations

After choosing a model or workflow, it could take a few moments to automatically generate the annotations. The generated labels are sorted in descending order based on their concept probability values.

Classifications Labeling

The Classifications pane lists the concepts generated by the classification model, alongside their probability values.

Objects Labeling

The Objects pane displays the bounding boxes identified by the detection workflow, alongside their probability values.

4. Review and Accept Predictions

Finally, you can review the model or workflow prediction suggestions and accept them as needed.

The AI assist probability threshold section in the right sidebar allows you to filter predictions by probability values. Use the slider control to display only the predictions that fall within your selected probability range.

To accept all AI-assisted suggestions for annotating your input(s) with the selected threshold, simply click the Accept all AI assist predictions button.

The Search concepts field allows you to find specific concepts and display their annotations on the page.

Classifications Labeling

The Classifications pane allows you to review and accept classification predictions.

• Concepts count display — The pane displays the number of concepts used to annotate an image.
• Accept or remove annotations — To manually accept a prediction, simply click on the checkmark next to it.
• View individual annotators — If you hover over the person icon on an annotation field, you can view the annotator(s) who added that annotation.
• Update annotations — To update an already accepted annotation, hover over it and click the pencil icon that appears.
• Delete annotation — If you hover over an already accepted annotation field, a delete icon will appear that you can use to remove the annotation.
• Hotkey assignment — Each annotation is assigned a hotkey number.

Objects Labeling

The Objects pane allows you to review and accept objects predictions. It displays categories of concepts used for annotations, along with individual annotation instances.

Reviewing and accepting object predictions is largely similar to classification labeling, with a few minor differences.