Skip to content

Make experiments in extraction projects

Overview

Once annotations have been created for all defined information classes, both in a training library and in a test library, you can perform experiments which consist of creating the extraction model and applying it to a test library.

The training library must meet the following requirements:

  • At least ten annotated documents.
  • At least one class with ten annotations.

Platform provides the following model types for extraction projects:

  • Auto-ML Extraction.
  • Explainable Extraction.
  • Studio, that is an imported CPK.

The Auto-ML Extraction model type creates a Symbolic model.

The Explainable Extraction model type creates a Machine Learning model.

The Studio model type creates a model based on an expert.ai Studio CPK imported model.

To start an experiment:

  1. Select Start an experiment on the toolbar of the project dashboard.
    The experiment wizard starts.

  2. In the Start an experiment dialog:

    2.1. Enter the experiment name in Name or leave empty for automatic assignment.

    2.2. Select the test library in the Test library drop-down menu.

    2.3. Select an available model type:

    Or:

    Or:

    and then follow the wizard. Select Back to go back to the previous stage or Cancel to quit.

    Note

    Select Hide advanced parameters to hide the advanced parameters that are marked with a blue italic caption.

  3. In Summary select the match strategy in the Metrics Matching Strategy drop-down menu, then select Start to start the experiment.

    Note

    The match strategy affects the difference in values ​​in the result metrics.

The experiment progress window is displayed during the engine process.

To terminate the process before its end, select Delete experiment.

The process consists of six sequential stages:

  1. Initialization
  2. Model generation preparation
  3. Model generation
  4. Document analysis preparation
  5. Document analysis
  6. Experiment wrap-up

Note

If the experiment fails, the Info tab appears displaying information and the type of errors. You can also check the Activity log tab for further information.

Once the process is completed, the analytics are displayed in the Experiments tab, Statistics sub-tab, where it is possible to analyze and interpret the results.

Note

Experiment results are associated to the test libraries you chose in the experiment wizard, so it is common that the Experiments tab is disabled for other libraries.

Auto-ML Extraction engine procedure

  1. Select the training library in the Training library drop-down menu and the selection policy under Training documents selection policy, then select Next to go on.
  2. Select the Machine learning model type (multiple selection is allowed. One to have a single experiment, 2 to have experiments in parallel), then select Next to go on.
  3. Select the Feature space (advanced): which data elements to use in feature vector creation, then select Next to go on.
  4. Select the Model-specific hyperparameters parameters, then select Next to go on.
  5. Select the Precision and recall balance parameters, then select Next to go on.
  6. Select the Machine Learning automatic self-tuning process parameters, then select Next to go on.

Explainable Extraction engine procedure

  1. Select the training library in the Training library drop-down menu and the selection policy under Training documents selection policy, then select Next to go on.
  2. Select the Support, confidence and tolerance parameters, then select Next to go on.
  3. Select the Active features options, then select Next to go on.
  4. Select the Options for selecting best rules, then select Next to go on.

Studio engine procedure

  1. Select the model in Model selection, then Next to go on.
  2. Check the remap in Remapper, then Next to go on.
  3. In Summary, select the match strategy in the Matching Metrics Strategy drop-down menu and select Start.