Annotate with Active learning

Active learning is a tool that helps annotate documents.

This tool generates extraction suggestions after a certain number of manual annotations—the annotation goal—has been performed.

This tool is useful especially when starting a new project with no annotations, the so-called bootstrap step.

Start an Active learning session

To start an Active learning session, select Active learning

The default environment is the Immersive view even if it works in detail view too. In the Immersive view it is easier to manage the Active learning, the Fast annotation and Propagation are already switched on by default and there are also some additional utilities (for example the Toggle suggestion mode button. See further on this page).

You can use Active learning in detail view but you have to highlight the extraction as described in Annotate in detail view.

Active learning annotation preferences

The first time you access the tool, a dialog box appears that allows you to set your preferences.

• Enter the Annotation goal, that is the manual annotation amount to perform (min 50) or the number of manual annotated documents (min 10) to start the annotation suggestion generation.

• In the Smart document suggestion section, you can set what type of document is suggested for the next annotations.

  Select:

  • Prefer richer documents if you want documents with a large number of extractions but not annotated.
  • Prefer similar documents if you want related documents.
  • Prefer recently enriched documents if you want recently analyzed documents.
  • Prefer almost completed documents if you want a non-validated document with a large number of annotations.

  If you select all options, the order of preference is as displayed in the dialog.

• In the document validation, select:

  • Automatic for the system validation procedure that is activated for all the annotated documents.
  • Manual if you want to validate the documents when you decide.

Note

It is always possible to change these settings, selecting:

• Active learning settings

Or:

• ** Project settings in the Active learning** tab .

Annotate documents

To start the annotation suggestion generation, annotating a certain number of documents or, better, reaching the annotation goal number is necessary. In this preliminary step you have to manually annotate the documents. The procedure is the same as in the detail view.

Anyway since the propagation tool is activated, if you manually annotate a portion of text, the other occurrences are automatically highlighted with a dashed underlining.

A notification showing the occurrence counter is displayed in the upper right corner of the document. This is useful to understand if there are other occurrences in parts of the document that are not displayed. For each confirmed suggested annotation, the counter decreases by one.

Note

The last annotated occurrence propagates throughout the whole set of documents as suggestion.

Access the next document with the smart document suggestion

To access the next document with the smart document suggestion, as set in the preferences, select the blue arrow in the lower bar.

Check the current annotation goal number

To check the current annotation goal number, see the annotation goal status bar in the lower side.

The status bar takes on the following colors as the annotations increase:

• Red: annotation number not enough to start generating annotation suggestions.

  

• Orange: annotation number almost enough to start generating annotation suggestions.

  

• Green: annotation number enough to start generating annotation suggestions, even if the annotation goal is not reached.

  

  

When the annotation goal is reached, the Start now button turns blue .

Note

If the annotation goal number is reached, the generated suggestions are more accurate.

Start the suggestion generation

You can start the annotation suggestion generation when the status bar is green:

• Select Start now if the annotation goal number is not reached.

Or:

• Select Start now if the annotation goal number is reached.

A notification is displayed once a user starts a suggestion generation procedure.

Info

During the extraction suggestions generation you can work on other tasks.

Check the generation process progress

To check the generation process status, watch the status bar at the bottom side.

Skip the suggestion generation

To skip the experiment, select Skip , that is available only when Start now is activated.

The annotation goal number is reset to zero.

Check the suggestions

To check the extraction suggestions:

1. Select Toggle suggestions mode .
2. Browse the documents, the extraction suggestions are displayed with a dashed underlining.

Note

Deselect Toggle suggestion mode to hide the extraction suggestions.

Confirm or reject the extraction suggestions

To confirm the extraction suggestion, select it and it will turn into a true positive.

Note

If other identical values are already true positives, they will be highlighted once you confirm the extraction suggestion.

To reject the extraction suggestion, ignore it.