Skip to content

Experiments quality

Overview

To view the quality of an experiment:

  1. Go to the experiments dashboard.
  2. Select the experiment from the Experiments panel.

Or:

  1. Select Statistics from the Experiments drop-down menu,
  2. Select the experiment from the Experiments panel.

Note

In the Experiments panel, select Expand and Collapse to expand and collapse the panel.

The Statistics sub-tab—displayed by default—shows a set of statistical information about an experiment.

Information about the experiment

In the upper side tab you can see the experiment details, such as the:

  • Experiment name
  • Model type
  • Performance date and time
  • Experiment author
  • Number of library documents
  • Analyzed documents
  • Validated documents
  • Training library
  • ML Model type in case of the Auto-ML Extraction as selected model.
  • Matching strategy

Read the quality

The quality measurements are displayed in the following metrics:

  • Micro Average
  • Macro Average
  • Sample Average
  • Weighted Average

The metric in focus is selected in the Settings window, Experiments tab. The specific metric icon is also displayed in the experiment card.

Check annotated and extracted documents and classes

In the lower part, is is reported the number of:

  • Classes and documents
  • Number of extracted and annotated classes and documents

Check the Top classes chart to see the most extracted classes. Hover over the orange line to know the count of extraction of a specific class.

Check the Most extracted and Not extracted documents to see documents with most extractions and documents with no extractions.

Hover over a document and select Open document to open a document in detail view.

Hover over a document and select Annotate document to annotate it.

Note

More details about Annotate document and Open document below.

Analytics on classes in Micro metrics

In the Resources sub-tab, for each Class, the following items are displayed in the Micro metrics sub-tab:

  • Ext. (Number of extractions)
  • Ann. (Number of annotations)
  • TP (True Positive)
  • FP (False Positive)
  • FN (False Negative)
  • Precision
  • Recall
  • F-mea. (F-Measure)

Sort the classes

To sort the classes based on a specific item, hover over the column headers and select the arrows beside them.

Check the top and worst extractions of a class

To check the top and worst extractions of a specific class, select the related expanding icon .

The orange line represents the number of documents with extractions of a value, while the violet one the number of documents with annotations of a value.

To open the search bar, hover over the class and select Search .

You will see the Documents statistics tab with the filtered list of documents based on the concept in focus.

Show the classes in resources

To show a selected class in the Resources tab, select Show in resources .

Check the document with metrics

To check the documents quality results, in the Documents sub-tab, select the Documents with metrics panel in the lower tab side.

For each document, the following items are displayed:

  • Document validated mark (if the document is validated)
  • Ext. (Number of extractions)
  • Ann. (Number of annotations)
  • TP (True Positive)
  • FP (False Positive)
  • FN (False Negative)
  • Precision
  • Recall
  • F-mea.

View and hide the annotated items

To view the annotated items of a document based on the results of the latest experiment performed, select the expanding icon .

The chip colors do not correspond to the quality of all the annotations of a class. As example, the value macaroni is reported as a false negative with a count of 7 occurrences. This does not mean that all occurrences of macaroni in the document are false negatives, but that there is at least an occurrence of macaroni that is a false negative.

Consider this other example:

The value chicken in this other document has 8 occurrences but its chip is orange. This means that there is at least a value of chicken that is a false positive, but it does not mean that all values of chicken are false positives.

One last example:

In this other case, the chip colors are green, meaning that all class values in the documents are true positives.

Sort the documents

To sort the documents based on an item, hover over the column headers and select the arrows beside them.

Open a document and add annotations

To open a document in detail view based on a selected experiment in focus, hover over it and select Open document . The document will open in the detail view.

For example:

In the selected experiment, Experiment 9, the value Rice has a single false positive occurrence in the document 06_recipe.txt.

Evidence of this is available when Open document is selected.

What you see is a record of the document before the latest experiment was performed in comparison with the selected one.

In this view, annotation is not possible. To annotate, select Annotate documents from the toolbar beside the document name. This is what you will see:

As you can see, the latest experiment—Experiment 10— is in focus and the value Rice is now a true positive. This means that after Experiment 9 and before Experiment 10 the value Rice has been turned into a true positive.

An alternative to see this view while in the experiment dashboard with an experiment in focus is:

  1. In the Documents with metrics sub-panel, hover over a document.
  2. Select Annotate document .

Filter validated documents

To filter validated documents, select Only validated documents to get the filter criteria:

  • = no filtering.
  • = only validated documents.
  • = only not validated documents.

Check the not annotated documents

To check the non-annotated documents, select the Not annotated documents sub-panel.

For each document, the extraction count is displayed.

View and hide the annotated items

To view the annotated items for a document, select the expanding icon .

Sort the documents

To sort the documents based on an item, hover over a column header and select the arrow beside it.

Add annotations

To add annotations to a document:

  1. Hover over the document of interest and select Annotate document .
  2. Follow the usual annotation procedure.

Open a document in detail view and then annotate

To open a document in detail view, hover over the document of interest and select Open document .

While in there, select Annotate documents to start annotating the document in the detail view of the Documents tab.

Filter documents with extractions or without extractions

Click the icon beside Only documents with extractions until it turns into one of the following forms:

  • = no filter.
  • = only documents with extractions.
  • = only documents with no extractions.

Edit the experiment name

To edit the experiment name, hover over the experiment name and select Edit experiment name .

Delete the experiment

To delete the experiment, select Delete .