Manage annotations

Overview

Annotating a document means tagging it with one or more categories included in the taxonomy.

During experiments, annotations in training library documents allow the model to "learn" how to predict similar categories. In test library documents annotations allow to compute metrics to determine model quality.

Annotations should be done for all the expected output categories of the project taxonomy before runing experiments, based on the principle: "no annotations, no categorizations".

You manage annotations for a given library in the Documents tab of the project dashboard.

The Main topics strip, at the top of the document in the detail view, can be a hint for proper annotations.

Warning

Annotation features are disabled for documents in a language other than the project language.

Annotate a document

Annotating a document means to manually select a category for a text. The selected category is visually represented by an indigo chip . It is an operation initially performed in the training library, but it is also important for the test library.

After an annotation or a change, an icon suggests a page refresh.

Note

A colored dot beside the category in the Taxonomy panel in the list view represents the population count.

= Less than 6 documents annotated for the category.
= Between 6 and 20 documents annotated for the category.
No dots = more than 20 annotations

Annotate a document in the list view

Select or hover over the document preview.
Select Add category or Add category if the preview document is selected.
Enter the category name, then press Enter.

Info

A suggestion is displayed if the first two chars you are entering are included in a taxonomy category and an alert if they are not included.

Tip

Use the Taxonomy panel on the right side to check if a category is in the taxonomy. You can also search for it using the search bar placed above the taxonomy itself.

Or:

Select the documents you want to annotate, multiple selections with Ctrl and Shift are allowed.
Drag the documents in the desired category displayed in the Taxonomy panel on the right side or select Annotate documents in the toolbar and enter the category in Annotate documents.

Tip

If you think that the displayed text is not enough to decide the category of the annotation, select show more or switch to the detail view.

Annotate in immersive view or in Pdf document view

Select Add category
Enter the category name, then press Enter.

Or:

Drag a category from the Taxonomy panel into the text.

Annotate a document in the detail view

Select a document from the left panel list or consider the opened one.
Select Add category .
Enter the category name, then press Enter.

Or:

Drag a category from the Taxonomy panel into the text.

After manual annotations, the Ann annotation counter displays the annotation count.

Annotate suggestions coming from the Building magic Taxonomy procedure

If you create the taxonomy with the Building magic Taxonomy procedure, the documents have annotations suggestions displayed with a dashed outline.

To find suggestions for a category, enter $SUG and the category name in the search bar .
To hide the suggestions, in the list view, select Hide suggestions .
To show suggestions, in the list view, select Show suggestions .
To annotate the document categories, the procedure is the same as already described, but the chip visualization is different .

Annotate resulting categories

Annotating categories in a document means confirming or rejecting categories resulting from an experiment, or even change the already annotated ones. This is an operation that is performed generally in the test library and after running an experiment.

Suggested resulting categories are represented by a blue chip:

In detail view, the category counter Cat shows the number of resulting categories. The resulting categories visualization depends on the used engine type in the experiment.

With AutoML Categorization:

With Explainable Categorization or CPK:

In this case, in addition to the category, the frequency and the hit number that triggered the category are displayed.

Select the chip to check the score and display the annotation bar.

Select the hit number to highlight the hits that triggered the category in the text.

Hits are differently highlighted based on their quality. Hits referring to true positive categories are highlighted in green and the text color is green. Hits referring to false positive categories are highlighted in orange and the text color is orange. Below the underlining, hover over CAT to expand its tooltip.

Note

CAT refers to Categories, so the hits that made the category trigger.

To annotate a category, in list and detail view:

Select the category chip.
Select Annotate .

If you annotate resulting categories, the chips switch to the green color that represents the true positive TP.

Automatically the other possible returned categories are turned into orange chips that represent false positives FP. It is possible to annotate them too if desired.

The annotation counter Ann is then updated accordingly.

Annotate all the categories in the detail view

To annotate all the categories of the document in the detail view, select Annotate all categories .

Annotate suggestions coming from the Building magic Taxonomy procedure in the Resources tab

Select Show suggested documents
Select Expand to expand the Suggested Annotations Documents panel.
Click the suggestion, a widget is displayed.
Select Annotate.

Remove an annotation or a category

To remove an annotation or a category in list or detail view:

Select the annotation.
Select Remove .

Remove all the categories in detail view

In detail view, if you want to remove all the categories from a document, select Remove all categories .

Replace an annotation

To replace an annotation:

Select the annotation.
Select Replace .
Enter the new category name, then press Enter.

Show the resources for a specific annotation or category

To show the resources for a specific annotation or category:

Select the annotation.
Select Show resources .

The Resources tab displays the category in the taxonomy.

Show the overall annotations for that specific annotation category

To show the overall annotations for that specific annotation category:

Select the annotation.
Select Show info .