Manage annotations

Introduction

Unlike in categorization and extraction projects, in thesaurus projects the training of the models is not done with annotated documents (the training library), but with the definition of the thesaurus concepts and the semantic relationships between them.
However, annotations serve to test the models, by indicating the occurrences of the concepts that are expected to be extracted.
The annotation tool also allows changing the thesaurus by adding and modifying concepts when useful expressions are found in the text.

Annotations are created and modified in the detail view and its related views, and the context view of the Documents tab.

Annotate in detail view

Annotations are mainly performed in the detail view.

To annotate a portion of text:

Select the portion of text to be annotated: a pop-up menu appears.
Select the annotation type.

You can select:
- Annotate as...
- Create Concept
- Create label

Annotate as

Annotate as displays a window with which a concept can be:

Simply annotated, if the form corresponds to the form in the thesaurus node.
Added as Alternative label.
Added as Hidden label.

If View in resources after annotation is checked, the Resources tab will be displayed and the concept context will be focused.

Create concept

Create Concept displays a window with which you can create a new concept, organize it in the taxonomy, and annotate it by selecting the toggle button Annotate concept.

If View in resources after annotation is checked, the Resources tab will be displayed and the concept context will be focused.

Note

You can also create a concept from the Entities or Tokens tab, both in list and detail views.

Select or hover over the single occurrence of an entity or a token.
Select Add new concept.

Create label

Create label displays a window with which you can add a concept as Alternative label or Hidden label.

If View in resources after annotation is checked, the Resources tab will be displayed and the concept context will be focused.

Note

Each time you annotate, delete an annotation, validate a document or remove the validation, a last activity notification appears at the bottom of the text.

When co-working on a single text, select Refresh each time a cooperator makes an action on a text you're working on.

Annotate an entity or a token

To annotate an occurrence of an entity or token:

Use the right panel to highlight occurrences of entities and/or tokens in the text, as described in this other article.
Select the single occurrence of an entity or a token.
In the text, select the related chip to be annotated: a pop-up menu appears.
Select the annotation type as described in Annotate in detail view.

Note

To remove the highlighted occurrences:

Deselect the occurrences.

Or:

Select the X button .

Annotation look

An annotation chip consists of two components, each of a specific color:

On the left: the type indicator:
On the right: the annotation with a specific background color that represents the quality of the experimental outcome:
- Violet = FN (False Negative)
- Green = TP (True Positive)
- Orange = FP (False Positive)
- Blue = ANN or EXT

The default color is violet which indicates the annotation is a false negative because it does not—or does not yet—correspond to an extraction.

This color is meaningful in the documents of test libraries and after experiments, when you check extraction results. If, after an experiment, the annotated text is extracted as expected, the outline color will become green, indicating the annotation was matched (true positive).

The orange chip indicates a false positive, that is a result that was not annotated but needs confirmation.

The blue chip represent the generic extracted or annotated value, you can filter these chips selecting the Ext or Ann buttons at the top of the document.

Annotate an extraction

To annotate an extraction in detail view, which means confirming that the prediction is correct:

Select Annotate from the pop-up toolbar.

The colored dot in the toolbar turns from orange to green.

Or, to annotate all extractions of a concept:

Select the concept from the Thesaurus tab (multiple selection with single-click is allowed).
Select Annotate selected items .

Note

Select Select all to select all the concepts from the Thesaurus tab.
Select Deselect all to deselect all the concepts from the Thesaurus tab.

Note

If you select a single concept, the Select all icon will change to this: . Click it to select all concepts.

Remove annotations

To remove an annotation from the text in the detail view, select the annotation and choose Remove from the pop-up toolbar.

To remove all annotations of a concept:

Select the concept from the Thesaurus tab (multiple selection with single-click is allowed).
Select Remove selected annotations .

Replace an annotation concept

To change the concept of an annotation:

Select the annotation in the text, then choose Replace from the pop-up toolbar.
In the pop-up menu, select the new concept.

Show annotated or extracted concepts in the resources

To show an annotated concept in the Resources tab, select an annotation—or an extraction—then select Show resources from the pop-up toolbar.

Or:

Hover over the concept from the Thesaurus tab on the right and select Show in resources .

Or:

Select the concept from the Thesaurus tab on the right and select Show in resources .

Show information about an annotated or extracted concept

To show information about an annotated concept:

Select Show info from the pop-up toolbar.

Or:

Hover over the concept from the Thesaurus tab on the right and select Show information .

Or:

Select the concept from the Thesaurus tab on the right and select Show information .

Information regarding the annotated concept, such as broader concept, preferred label and alternative label, is displayed in a pop-up window.

Annotate in immersive view

Annotation management in immersive view is the same as in detail view. Just some collateral tools are missing.

Annotate in Pdf document view

Annotation management in Pdf document view is the same as in detail view. Just there are some light graphical differences.

To display the chip quality indicator, hover over the annotation.

Annotate in context view

The context view shows phrasal contexts of extractions and annotations.

Populate the list

To populate the list:

Select the Thesaurus tab in the left panel.
Select the checkbox at the left of the values of interest.

Unpopulate the list

To unpopulate the list:

Deselect the checkboxes.

Or:

Deselect the concepts from the Context drop-down menu.

Confirm an extraction

To confirm an annotation or an extraction:
- Hover over it and select the annotation icon .
Or:
- Select it and select the annotation icon .
To confirm all annotations or extractions, select the annotation icon beside the document title.

Delete an annotation or an extraction

To delete an annotation or an extraction:
- Hover over it and select the trash can icon .
Or:
- Select it and click the trash can icon .
Or:
- Don't annotate the extraction, it will be removed in the next experiment.