Skip to content

Report

Overview

The Report tool window displays and allows managing the reports produced by multi-document preparation and analysis operations.
It also allows comparing the reports of two analyses to see if any improvements or regressions have occurred.

The window contains a table with these columns:

Name Description
Type Report type (A = analysis, P = preparation, C = comparison )
ID Report ID
Description Report name
Date Operation time
Duration Operation duration
Files Document count
Success Success rate expressed as a percentage
Categorization/Precision Categorization Precision expressed as a percentage
Categorization/Recall Categorization Recall expressed as a percentage
Categorization/F-Measure Categorization F-Measure expressed as a percentage
Extraction/Precision Extraction Precision expressed as a percentage
Extraction/Recall Extraction Recall expressed as a percentage
Extraction/F-Measure Extraction F-Measure expressed as a percentage

In the case of comparison reports, an icon on the left of the description indicates the qualitative trend, i.e. the difference in quality between the two analyzes compared:

Icon Description
Progress: the second analysis yielded better results, there was an improvement
Stability: overall quality was the same for both analyses, tie
Regression: the second analysis produced worse results

The percentage values ​​can be displayed in a different color in case they reach a target value. Target values and highlight color can be set in Studio Settings > Project > Quality > Target reached color.

The context menu contains the Edit Description command that allows changing the report description.

Available mouse commands are:

Command Description
Click a column header Change sort order
Double-click a row Show the Analysis Details window for A type reports, an XML file for P type reports and the Analysis Comparison window for C type reports.

The toolbar contains:

Icon Name Description
Filter list Report type filter
02_1.png Categorization Quality Data Display or hide categorization data
02_2.png Extraction Quality Data Display or hide extraction data
compare.png Compare Compare two analysis reports creating a comparison report
View Reports Display the report details in the Analysis Details window
.png Delete Reports Delete the selected report
ref.png Refresh Refresh report list

It is also possible to delete a report with the Del key.

The info bar shows the reports count.

Analysis Details

The Analysis Details window shows the details of a type A report.

The windows contains the panels described below.

Documents

This panel shows file-by-file data for the selected report, each row represents a document.

The columns are the following:

Name Description
Validation status Validated or not validated file or file not found
File File name
Size File size in bytes
Duration Analysis duration
Success Analysis outcome
Error Error
Categories Number of categories
Extractions Number of extractions
Categorization TP Categorization true positives, i.e. number of target categories matched
Categorization FP Categorization false positives, i.e. number of unexpected results
Categorization FN Categorization false negatives, i.e. number of target categories not matched
Categorization Precision Categorization Precision expressed as a percentage
Categorization Recall Categorization Recall expressed as a percentage
Categorization F-Measure Categorization F-Measure expressed as a percentage
Extraction TP Extraction true positives, i.e. number of target extractions matched
Extraction FP Extraction false positives, i.e. number of unexpected results
Extraction FN Extraction false negatives, i.e. number of target extractions not matched
Extraction Precision Extraction Precision expressed as a percentage
Extraction Recall Extraction Recall expressed as a percentage
Extraction F-Measure Extraction F-Measure F-Measure expressed as a percentage

The toolbar contains:

Icon Name Description
Filter list Analysis outcome filter (FAILURE or not)
Categorization Quality Data Display or hide categorization data
03_2.png Extraction Quality Data Display or hide extraction data
03_4.png Error Column Display or hide the Error column
03_5.png Export CSV Export all the files data in Comma-separated values (CSV) format

Available mouse commands are:

Command Description
Click a column header Change sort order
Double-click a row Display the file in the editing area

The info bar shows the files count.

Taxonomy

This panel shows the results of the categorization against the project taxonomy.

It contains two areas. The upper area shows taxonomy information and contains a table with a row for each domain.
The table is initially collapsed and can be expanded row by row with the expand and collapse commands on the left side of the row or with the toolbar commands.
The table has these columns:

Name Description
Name Domain name
Description Domain label
TP True positives, i.e. number of times the category was returned as a result and matched an annotations (matches)
FP False positives, i.e. number of times the category was returned as a result, but was not annotated as a categorization target (unexpected results)
FN False negatives, i.e. number of documents for which the category was annotated as a categorization target, but didn't come out as a result (missed matches)
Precision Categorization Precision expressed as a percentage
Recall Categorization Recall expressed as a percentage
F-Measure Categorization F-Measure expressed as a percentage

Toolbar commands are:

Icon Name Description
Expand All Expand all the tree nodes
Collapse All Collapse all the tree nodes

The info bar shows first-level nodes count.

The lower area shows data for all analyzed documents relative to the category selected in the upper area.
It contains a table with these columns:

Name Description
File Document file name
Annotations 1 if the selected category was annotated as a target categorization result for the document, 0 otherwise
Rule results 1 if the selected category was returned as a categorization result for the document, 0 otherwise
TP True positive: 1 if the selected category was annotated as a target categorization result for the document and was also returned as a categorization result for the document (match), 0 otherwise
FP False positive: 1 if the category was returned as a categorization result for the document, but was not annotated as a target categorization result for the document (unexpected result), 0 otherwise
FN False negative: 1 if the selected category was annotated as a target categorization result for the document, but was not returned as a categorization result for the document (missed match), 0 otherwise

The info bar shows the files count.

Templates

This panel shows the extraction results against the defined templates.

It contains two areas. The upper area shows templates information in a table.
The table is initially collapsed and can be expanded row by row with the expand and collapse commands on the left side of the row or with the toolbar commands. First-level rows correspond to templates, second-level rows correspond to template's fields.
The table has these columns:

Name Description
Name Template or field name
Attributes Field attributes
TP True positives, i.e. number of times actual extractions matched annotations (matches)
FP False positives, i.e. number of times actual extractions didn't match any annotation (unexpected results)
FN False negatives, i.e. number of annotations that were not matched by actual extractions (missed matches)
Precision Extraction Precision expressed as a percentage
Recall Extraction Recall expressed as a percentage
F-Measure Extraction F-Measure expressed as a percentage

The info bar shows first-level nodes count.

Toolbar commands are:

Icon Name Description
Expand All Expand all the tree nodes
Collapse All Collapse all the tree nodes

The lower area shows data for documents with annotations or actual extractions relative to the template or field selected in the upper area.
It contains a table with these columns:

Name Description
File Document file name
Annotations Number of annotations
Rule results Number of actual extractions
TP True positives: number of actual extractions that matched annotations (matches)
FP False positives: number of actual extractions that didn't match any annotations (unexpected results)
FN False negatives: number of annotations that were not matched by actual extractions (missed matches)

The info bar shows the files count.

Properties

This panel shows a lot of information about the report grouped as follows:

  • Module: details of the project module
  • Report: information on the selected report
  • Build: information about the software version and the build operation
  • Rules: number of rules per type
  • Files: number of files per type
  • Statistics: statistical information on the analysis
  • Timimgs: break-down of the times required for the various phases of the analysis

Analysis Comparison

The Analysis Comparison window shows the details of a type C report, i.e. the comparison of two analysis report.

This is the information shown:

Name Description
Module Project module name
Trend Quality trend considering the changes from the first to the second report
Analysis Date ID and time of the two analysis reports
Extraction Extraction performance metrics
Categorization Categorization performance metrics

The Details buttons open windows that show side-by-side comparison of report data. These windows are described below.

Properties

The Properties window shows a side-by-side comparison of the properties of the two reports.

The information for each report is the same as in the Properties panel of the Analysis Details window.

Extraction results

The Extraction results window shows a detailed comparison of extraction results.

It contains two areas. The upper area shows templates information in a table.
The table is initially collapsed and can be expanded row by row with the expand and collapse commands on the left side of the row or with the toolbar commands. First-level rows correspond to templates, second-level rows correspond to template's fields.
The table has these columns:

Name Description
Name Template or field name
Annotations Number of annotations
Attributes Field attribute
TP True positives counters
FP False positives counters
FN False negatives counters
Precision Precision data
Recall Recall data
F-Measure F-Measure data

By default, columns TP, FP, FN, Precision, Recall and F-Measure display only the difference or delta (Δ) between the metrics of the two reports. The delta symbol is colored to indicate quality trend:

  • Green: progress
  • Black: stability
  • Red: regression

The header of these columns act as a toggle switch to display or hide the values in addition to the difference.

The info bar shows first-level nodes count.

Toolbar commands are:

Icon Name Description
Expand All Expand all the tree nodes
Collapse All Collapse all the tree nodes
Toggle Attribute Visibility Display or hide the Attributes column

The lower area shows data for documents with annotations or actual extractions relative to the template or field selected in the upper area.
It contains a table with these columns:

Name Description
File Document file name
Annotations Number of annotations
Rule results Number of actual extractions
TP True positives: number of actual extractions that matched annotations (matches)
FP False positives: number of actual extractions that didn't match any annotations (unexpected results)
FN False negatives: number of annotations that were not matched by actual extractions (missed matches)

Numbers between brackets refer to the older report, the other numbers are from the newer report.

The info bar shows the files count.

The toolbar contains these controls:

Icon Name Description
Docs In Filter the list to show only documents that have actual categorization results as for the newer report and did not have any categorization result as for the older report
Docs Out Filter the list to show only documents that don't have actual categorization results as for the newer report, but had categorization results as for the older report
Docs Won Filter the document list to show only documents that have won true positives
Docs Lost Filter the document list to show only documents that have lost true positives
Reset filters Remove the filters and display the complete list

Categorization results

The Categorization results window shows a detailed comparison of categorization results.

It contains two areas. The upper area shows taxonomy information in a table.
The table is initially collapsed and can be expanded row by row with the expand and collapse commands on the left side of the row or with the toolbar commands. The table has these columns:

Name Description
Name Domain name
Annotations Number of documents in which the domain was annotated as a target categorization result
Attributes Domain label
TP True positives counters
FP False positives counters
FN False negatives counters
Precision Precision data
Recall Recall data
F-Measure F-Measure data

By default, columns TP, FP, FN, Precision, Recall and F-Measure display only the difference or delta (Δ) between the metrics of the two reports.
The header of these columns act as a toggle switch to display or hide the values in addition to the difference.

The info bar shows first-level nodes count.

Toolbar commands are:

Icon Name Description
Expand All Expand all the tree nodes
Collapse All Collapse all the tree nodes
Toggle Attribute Visibility Display or hide the Attributes column

The lower area shows data for documents with annotations or actual categorization results relative to the category selected in the upper area.
It contains a table with these columns:

Name Description
File Document file name
Annotations 1 if the selected category was annotated as a target categorization result for the document, 0 otherwise
Rule results 1 if the selected category was returned as a categorization result for the document, 0 otherwise
TP True positive: 1 if the selected category was annotated as a target categorization result for the document and was also returned as a categorization result for the document (match), 0 otherwise
FP False positive: 1 if the category was returned as a categorization result for the document, but was not annotated as a target categorization result for the document (unexpected result), 0 otherwise
FN False negative: 1 if the selected category was annotated as a target categorization result for the document, but was not returned as a categorization result for the document (missed match), 0 otherwise

Numbers between brackets refer to the older report, the other numbers are from the newer report.

The info bar shows the files count.

The toolbar contains these controls:

Icon Name Description
Docs In Filter the list to show only documents that have actual extractions as for the newer report and did not have any extraction as for the older report
Docs Out Filter the list to show only documents that don't have actual extractions as for the newer report, but had extractions as for the older report
Docs Won Filter the document list to show only documents that have won true positives
Docs Lost Filter the document list to show only documents that have lost true positives
Reset filters Remove the filters and display the complete list