Manage annotations

Introduction

Annotating documents is the core activity by which examples of desired extractions—annotations—are provided.
When generating the extraction ML model, the generation algorithm "learns" from these examples to give the model the ability to predict similar extractions.

Annotations should be created for all the information classes before generating the extraction ML model, according to the principle: "no annotations, no extractions".

Unlike categorization projects, in which you annotate entire documents with the expected categories, here you need to annotate portions of text.

Since annotation are specific selected text portions, they are created in the detail view of the Documents tab and are modified in that same view or in the context view, in the same tab.

The three views of the Documents tab

The Documents tab has three views:

The list view, which displays the list of the documents. This is the default view.
The context view, which displays the list of the documents and, for each of them, a list of textual contexts, each one centered on an annotation. You can activate this view from the list view only (see below).

Note

The context view is activated only if there are annotations.
The detail view, which shows a simplified list on the left and the detail of a specific document in the center.

Open a document from the list view

To switch from the list view to the detail view:

Double click a document in the list.

Or:

Select the file name of a document in the list.

Or:

Select a document in the list and select View document at the top left of the document list.

Switch from list view to context view

To switch from the list view to the context view:

Select Show context view from the button bar at the top of the document list.
To populate the list:
1. Select the Classes tab in the left panel and:
  - To see all the annotations and extractions of a certain class in their context, select the checkbox at the left of the class box.
  - To see all the annotations and/or extractions of the same value in their context:
    1. Expand the information class with the expanding arrow .
    2. Under Extractions or Annotations, select the checkbox at the left of the value box.
2. Repeat the steps above to display more contexts and corresponding documents.

Or:

Select the Classes tab in the left panel and:
- To see all the annotations and extractions of a certain class in their context, hover over a class and select Context view .
- To see all the annotations and/or extractions of the same value in their context:
  1. Expand the information class with the expanding arrow .
  2. Under Extractions or Annotations, select the checkbox at the left of the value box.
Repeat the steps above to display more contexts and corresponding documents.

Or:

Select the Classes tab in the left panel.
Expand a class selecting the expanding arrow .
Hover over a value under Extractions or Annotations and select Context view .
Under Extractions or Annotations, select the other checkboxes you need at the left of the value boxes.

Note

To unpopulate the list:
- Uncheck the classes boxes.
Or:
- From the Context drop-down menu, deselect the classes.
Select Show all documents to show all documents.
Use the drop-down menus to sort the documents by relevance and filter documents by language .

Switch from context view to list view

To switch from the context view to the list view, select Show documents view from the button bar at the top of the document list.

Open a document from context view

To switch from the context view to the detail view, select the file name of a document in the list.

Change document in the detail view

To change the current document in detail view:

Select the document in the left panel

Or:

Use the Next document and the Previous document buttons located at the top of the document detail panel.

Switch from detail view to list view

To switch from the detail view to the list view, select Browse documents browse from the left panel.

The basic way to create annotations

Annotations are created in the detail view. See above how to open a document in detail view.

To annotate a portion of text:

Select all the text to be annotated: a pop-up menu appears.
Select the information class from the menu.

Note

Classes marked as favorite are placed at the top of the list and they are recognizable by the star next to the class name.

To annotate an occurrence of an entity or token:

Use the right panel to highlight occurrences of entities and/or tokens in the text, as you do for corpora.
Click the single occurrence of an entity or a token: a pop-up menu appears.
Select the information class from the menu.

After an annotation or a change, an icon suggests a page refresh.

Fast annotation

To fast annotate a document in the detail view:

Select Toggle fast mode to display the fast annotation toolbar.
In the fast annotation toolbar, select the class you want to annotate.
Select portions of text or click occurrences of entities and/or tokens you have previously highlighted in the text: all your selections will be annotated in the class you selected in the fast annotation bar.

Example

If you want to fast annotate all the ingredients in a recipe:

In the fast annotation bar, select ING that is the chip that represents the Ingredients class.
Select the ingredients in the document and they will automatically be annotated as Ingredients.

Or:

In the right panel, select the Tokens tab, then Main Lemmas.
Select the lemma onion that is a potential ingredient, its occurrences will be highlighted in the document.
Select the occurrences of the lemma highlighted in the text to annotate them as Ingredients.
.

Annotate by propagation

To annotate a document by propagation in the detail view:

Select an annotation in the text: a pop-up toolbar will appear.
Select Propagate on the toolbar.

As a result, all the portions of text that are deemed similar to the annotation will tentatively be annotated and the propagation toolbar will be displayed.

Use the propagation toolbar to:

Browse tentative annotations (A)
Confirm individual tentative annotations (B)
Delete a tentative annotation (C)
Confirm all tentative annotations (D)
Close the toolbar and delete all the tentative annotations (E)

Annotation look

The annotation chip has two colors:

The background color of the short class name on the left is the class color you can set in the Resources tab.
The color of the outline of the right part represents quality, that is the outcome of experiments:
- Purplish blue = FN (False Negative).
- Green = TP (True Positive).
- Orange = FP (False Positive).

The default color is purplish blue which indicates the annotation is a false negative because it does not—or does not yet—correspond to an extraction.
This color is meaningful inside the documents of test libraries and after experiments, when you check extraction results. If, after an experiment, the annotated text is extracted as expected, the outline color will become green, indicating the annotation was matched (true positive).

Annotate extracted values

Annotations and extractions are listed in the Classes tab of the right panel.

Extractions are available after experiments.

Annotate an extracted value in the text

Show the extractions.
Select the extraction that you want to annotate.
Selct Annotate .

The chip outline color turns from FP (False Positive) to TP (True Positive).

.

Note

It is possible to use Fast annotation and Annotatation by propagation.

Annotate an extracted value in the right panel

Select the Classes tab in the right panel.
Select the extractions to annotate.
Select Annotate selected items .

The extraction indicator in the right panel and the chip color outline in the text change from FP (False Positive) to TP (True Positive).

Annotate an extracted value in Context View

You can also manage your annotations while in context view.

Select the annotation icon to confirm your annotation.
Select the annotation icon beside the text title to confirm all annotations.

Remove annotations

In detail view

To remove an annotation from the text:

Select the annotation from the Classes tab.
Select the annotation from the text, then select Remove from the pop-up toolbar.

To remove an annotation from the document:

Select the Classes tab in the right panel.
Expand the class.
Select the value.
Select Remove selected annotations from the panel toolbar.

To remove all the annotations of a class from the document:

Select the Classes tab in the right panel.
Select the class.
Select Remove selected annotations from the panel toolbar.

In context view

You can use the context view (read above how to activate it) to easily delete annotations from multiple texts. Just select the trash can icon on the right of the context.

Change the class of an annotation

To change the class of an annotation in the detail view:

Select the annotation from the Classes tab.
Select the annotation in the text and then select Replace from the pop-up toolbar.
In the pop-up menu, select the new class.

Edit the value of an annotation

To change the value of an annotation in the detail view:
1. Select the annotation from the Classes tab.
2. Select the annotation in the text and then select Edit from the pop-up toolbar.
3. Edit the value in dialog box.
4. Select Save.

This operation is also useful in case you need to change a class value. For example, if you have two different people with the same name, it will be possible to discriminate by varying the value, or to consider a different measure of the same value as above.

The change has effect on the annotation but not on the document.

To change the value of an annotation in the detail view, right panel:
1. Select the Classes tab in the right panel.
2. Select the annotation to edit.
3. Select Edit .
4. Edit the value in dialog box.
5. Select Save.
To change the value of an annotation in the context view, select the pencil icon .

Show annotation in a document

See Show specific items in the document

Normalize annotations to their base form

To normalize annotations to their base form:

Select or select Settings , then Annotation.
Use the radio button Normalize annotations to enable or disable the normalization.

Advanced annotation utilities

There are additional tools for annotations that can be activated by selecting the ellipsis and selecting Enable advanced annotations utilities.

The following buttons will appear under the Classes panel toolbar:

Shift selected class
Clone selected class

Shift selected class

Suppose you made annotations under the wrong class, like this example:

To make a class shift:

Select the wrong class to replace.
Select Shift selected class on the toolbar.
Select the right class from the drop-down menu and select Save.

You will get:

Clone selected class

Some annotated values may belong to more classes.

Consider this example:

The two values also refer to units of measure. To clone a new class (previously defined in the Resources tab):

Select the class to clone.
Select Clone selected class on the toolbar.
Select the class from the drop-down menu and select Save.

You will get:

Show and hide extractions

In the list view, you can see the number of extractions and annotations.

To hide the number of extractions, select Hide extractions .
To show the number of extractions, select Show extractions .

Note

Each time you—or a cowrker—apply changes to the document, a notification warns you.