Skip to content

Manage annotations

Introduction

Annotating documents is the core activity by which examples of desired extractions—annotations—are provided.
When generating the extraction ML model, the generation algorithm "learns" from these examples to give the model the ability to predict similar extractions.

Annotations should be created for all the information classes before generating the extraction ML model, according to the principle: "no annotations, no extractions".

Unlike categorization projects, in which you annotate entire documents with the expected categories, here you need to annotate portions of text.

Annotations should also be created in the set of test documents, especially after experiments that return extracted classes, because they are necessary for a measurement of the extraction quality, that is, if the ML model has actually learned to extract.

Since annotation are specific selected text portions, they are created in the detail view of the Documents tab and are modified in that same view or in the context view, in the same tab.

The three views of the Documents tab

The Documents tab has three views:

  • The list view, which displays the list of the documents. This is the default view.
  • The Context view, which displays the list of the documents and, for each of them, a list of textual contexts, each one centered on an annotation. You can activate this view from the list view only (see below).

    Note

    The context view is activated only if there are annotations.

  • The detail view, which shows a simplified list on the left and the detail of a specific document in the center.

Open a document from the list view

To switch from the list view to the detail view:

  • Double click a document in the list.

Or:

  • Select the file name of a document in the list.

Or:

  • Select a document in the list and select View document view at the top right of the documents list.

Switch from list view to context view

To switch from the list view to the context view:

  1. Select Show context view from the button bar at the top of the documents list.
  2. To populate the list:

    1. Select the Classes tab in the left panel and:

      • To see all the annotations of a certain class in their context, select the checkbox at the left of the class box.
      • To see all the annotations of the same value in their context:

        1. Expand the information class.
        2. Select the checkbox at the left of the value box.
    2. Repeat the steps above to display more contexts and corresponding documents.

Note

  • To unpopulate the list:

    • Uncheck the classes boxes.

    Or:

    • From the Context drop-down menu, deselect the classes.
  • Select Show all documents to show all documents.

  • Use the drop-down menus to sort the documents by language and relevance.

Switch from context view to list view

To switch from the context view to the list view, select Show documents view from the button bar at the top of the documents list.

Open a document from context view

To switch from the context view to the detail view:

  • Select the file name of a document in the list.

Or:

  • Select at least one context in the document box in the list and then select View document view at the top right of the documents list.

Change document in the detail view

To change the current document in detail view:

  • Select the document in the left panel

Or:

  • Use the Next document next and the Previous document prev buttons located at the top of the document detail panel.

Switch from detail view to list view

To switch from the detail view to the list view, select Browse documents browse from the left panel.

The basic way to create annotations

Annotations are created in the detail view. See above how to open a document in detail view.

To annotate a portion of text:

  1. Select all the text to be annotated: a pop-up menu appears.
  2. Select the information class from the menu.

Note

Classes marked as favorite are placed at the top of the list and they are recognizable by the star next to the class name.

To annotate an occurrence of an entity or token:

  1. Use the right panel to highlight occurrences of entities and/or tokens in the text, as you do for corpora.
  2. Click the single occurrence of an entity or a token: a pop-up menu appears.
  3. Select the information class from the menu.

After an annotation or a change, an icon suggests to refresh the page.

Fast annotation

To fast annotate a document in the detail view:

  1. Select Toggle fast mode to display the fast annotation toolbar.
  2. In the fast annotation toolbar, select the class you want to annotate.
  3. Select portions of text or click occurrences of entities and/or tokens you have previously highlighted in the text: all your selections will be annotated in the class you selected in the fast annotation bar.

Example

If you want to fast annotate all the ingredients in a recipe:

  1. In the fast annotation bar, select ING that is the chip that represents the Ingredients class.
  2. Select the ingredients in the document and they will automatically be annotated as ingredients and displayed in the Classes tab.

Or:

  1. In the fast annotation bar, select ING that is the chip that represents the Ingredients class.
  2. In the right panel, select Tokens then Lemmas
  3. Select onion that is a potential ingredient, its occurrences will be highlighted in the document.

  4. Select the occurrences of the lemma highlighted in the text to annotate them as ingredients

Annotate by propagation

To annotate a document by propagation in the detail view:

  1. Select an annotation in the text: a pop-up toolbar will appear.
  2. Select Propagate on the toolbar.

As a result, all the portions of text that are deemed similar to the annotation will tentatively be annotated and the propagation toolbar will be displayed.

Use the propagation toolbar to:

  • Browse tentative annotations (A)
  • Confirm individual tentative annotations (B)
  • Delete a tentative annotation (C)
  • Confirm all tentative annotations (D)
  • Close the toolbar and delete all the tentative annotations (E)

Annotation look

The annotation chip has two colors:

  • The background color of the short class name on the left is the class color you can set in the Resources tab.
  • The color of the outline of the right part represents quality, that is the outcome of experiments:

    • Purplish blue = FN (False Negative).
    • Green = TP (True Positive).
    • Orange = FP (False Positive).

The default color is purplish blue which indicates the annotation is a false negative because it does not—or does not yet—correspond to an extraction.
This color is meaningful inside the documents of test libraries and after experiments, when you check extraction results. If, after an experiment, the annotated text is extracted as expected, the outline color will become green, indicating the annotation was matched (true positive).

Annotate extracted values

Annotations and extractions are listed in the Classes tab of the right panel.

Extractions are available after experiments.

Annotate an extracted value in the text

  1. Show the extractions.
  2. Select the extraction that you want to annotate.
  3. Select Annotate .

The chip outline color turns from FP (False Positive) to TP (True Positive).

.

Note

It is possible to use Fast annotation and Annotatation by propagation.

Annotate an extracted value in the right panel

  1. Select Classes tab in the right panel.
  2. Select the extractions to annotate.
  3. Select Annotate selected item .

The extraction indicator in the right panel and the chip color outline in the text change from FP (False Positive) to TP (True Positive).

Annotate an extracted value in Context View

You can also annotate in context view.

  • Select the annotation icon to confirm your annotation.
  • Select the annotation icon beside the text title to confirm all annotations.

Remove annotations

In detail view

To remove an annotation from the text:

  1. If not displayed, select the annotation from the Class tab.
  2. Select the annotation and select Remove from the pop-up toolbar.

To remove an annotation from the document:

  1. Select the Classes tab in the right panel.
  2. Expand the class.
  3. Select the value.
  4. Select Remove selected annotations from the panel toolbar.

To remove all the annotations of a class from the document:

  1. Select the Classes tab in the right panel.
  2. Select the class.
  3. Select Remove selected annotations from the panel toolbar.

In context view

You can use the context view (read above how to activate it) to easily delete annotations from multiple texts. Just select the trash can icon on the right of the context.

Change the class of an annotation

To change the class of an annotation in the detail view:

  1. If not displayed, select the annotation from the Class tab.
  2. Select the annotation in the text and then select Replace from the pop-up toolbar.

  3. In the pop-up menu, select the new class.

Edit the value of an annotation

  • To edit the value of an annotation in the detail view:

    1. If not displayed, select the annotation from the Class tab.

    2. Select the annotation in the text and then select Edit from the pop-up toolbar.

    3. Edit the value in dialog box.

    4. Select Save.

This operation is also useful in case you need to change a class value. For example, if you have two different persons with the same name, it will be possible to discriminate by varying the value. Or to consider a different measure of the same value.

The change has effect on the annotation but not on the document.

  • To change the value of an annotation in the detail view, right panel:

    1. Select Classes tab in the right panel.
    2. Select the annotation to edit.
    3. Select Edit .
    4. Edit the value in dialog box.
    5. Select Save.
  • To change the value of an annotation in the context view, select the pencil icon .

Move annotations

To move annotations from a class to another:

  1. If the advanced annotations utilities are not enabled, select the ellipses , then Enable the advanced annotations utilities.
  2. Select the class in the right panel, Class tab.
  3. Select Shift selected class .
  4. Select the target class from the Class drop-down menu in the Shift class annotations dialog.
  5. Select Save to confirm.

Clone annotations

To move clone annotations from a class to another:

  1. If the advanced annotations utilities are not enabled, select the ellipses , then Enable the advanced annotations utilities.
  2. Select the class in the right panel, Class tab.
  3. Select Clone selected class .
  4. Select the target class from the Class drop-down menu in the Shift class annotations dialog.
  5. Select Save to confirm.

Show annotation in a document

See Show specific items in the document.

Normalize annotations to their base form

To normalize annotations to their base form:

  1. Select or select Settings , then Annotations
  2. Use the radio button Normalize annotations to enable or disable the normalization.

Match strategy for a document

It might be useful to see what a document with a different match strategy would look like without having to do an experiment all over again.

To set a match strategy for the single document, select the related drop-down menu in the bar at the bottom.