Workflow output

Default output

A workflow can consist of a single flow or multiple, independent, flows, each branched and complex as necessary.

In the case of a single flow, the overall output of the workflow coincides with that of the final block of the flow, while with multiple flows the output is the concatenation of the output of all the flows, more precisely a JSON object with a property for each of the flows, corresponding to the output of the last block of each flow.
The property has a name that is the ID of the last block of the flow and a value that is an object containing the output of that block.

For example, if a workflow has two flows and the last blocks of the two flows have these IDs, respectively:

04b57400_d73b_4c99_1545_d9fdf3fe7ac7
e65954e7_3fc8_4fd2_cfbe_2225c1a3da2c

the overall JSON output of the workflow will look like this:

{
    "04b57400_d73b_4c99_1545_d9fdf3fe7ac7": {
        block output
    },
    "e65954e7_3fc8_4fd2_cfbe_2225c1a3da2c": {
        block output
    }
}

In this reference section of the manual you will find the description of the output for all the types of blocks (models, processors, etc.).

Error output

In case of errors, the workflow output is an object with this structure:

{
    "errors": []
}

where errors is an array of objects, each describing an error and having these properties:

message (string): error message`
node (string): ID of the block that returned the error

Verbose output

NL FLow can produce a more verbose output which provides debug information. This information is useful both in the design of complex workflows and for any troubleshooting.

Verbose output can be produced in two ways:

In interactive workflow tests, by turning on the debug option before starting a test.

Warning

Verbose output can be relatively large while the maximum size of the output the test tool can show is 5MB.
Requesting the document analysis API resource with the verboseErrorOutput=true query string parameter.

In the second case, the verbose output is returned only if there were errors, otherwise the output is the default one.

The verbose output is a JSON object with two properties:

workflowData (object): reports the workflow data.
componentsData (object): debug data of the workflow components that have been executed. Its structure varies based on whether the workflow was published in synchronous or asynchronous mode.

workflowData

The workflowData object has these properties:

input (object): workflow input
output (object): non-verbose output of the workflow
statistics (object): execution statistics. Its properties are:
- startTime (number): Unix timestamp, in milliseconds, of execution start
- endTime (number): Unix timestamp, in milliseconds, of execution end
- latencyMs (number): latency, that is duration of the execution, in milliseconds

componentsData

Synchronous mode

If the workflow is published in synchronous mode, componentsData contains a property for each block of the workflow that has been executed, with the exception of Fork—there is, however, a property for the corresponding Join—and Tunnel.

Each property is an object which name coincides with the block ID and has these properties:

executed (boolean): True if the block has been executed, False otherwise.
(if the block was executed) statistics (object): an object containing time statistics in these properties:
- startTime (number): Unix timestamp, in milliseconds, of execution start
- endTime (number): Unix timestamp, in milliseconds, of execution end
- latencyMs (number): duration of the execution in milliseconds
(if the block was executed without errors) output (object): the block output
(if the block was executed with errors) errors (array): an array of objects describing block errors; each object has the same structure of the object returned by the workflow in case of errors.

Asynchronous mode

If the workflow is published in asynchronous mode, the structure of componentsData is completely different from the case of synchronous mode.
The object contains the execution data of workflow's blocks with these peculiarities:

For Switch blocks, the data of the sole branches which were executed because of conditions that were met.
For each context corresponding to a splitter or remapper block, the items being iterated and the processing data for each of them.
For parallel flows, data is returned inside an object representing a virtual join block working as the convergence point of the flows.

The componentsData object can therefore have a nested structure when the workflow has conditional branches, splitter or remapper contexts, parallel flows.

type

The properties of the componentsData object vary depending on the value of the type property, a string that can be:

HEXAGON: for any block of a processor, of the Map operator or of a custom component
SEQUENCE: for a sequence of connected blocks
SPLITTER: for a splitter or remapper block
REDUCER: for a reducer block
SWITCH: for a Switch block
FORKJOIN: for a drawn or virtual Join or Join-Fork block. A virtual, non-configurable, Join block—invisible in the diagram—is inserted at the end of independent parallel flows, to merge their outputs.

The End Context, End Switch, Fork, and Tunnel operator blocks do not generate data in verbose output, but End Context is included in the description of components with type SPLITTER, End Switch is included in the description of components with type SWITCH and Fork is included in the description of components with type FORKJOIN.

In the following paragraphs you will find the description of the variable properties based on the value of type.

HEXAGON

An object with type property equal to HEXAGON corresponds to the execution of a block of a processor or a Map operator or a custom component. The other properties of the object are:

hexagonId (string): block identification code
serviceId (string): identifier of the software service corresponding to the block
input (object): the input received by the block
output (object): the output produced by the block
statistics (object): block execution statistics (see below)

SEQUENCE

An object with type property equal to SEQUENCE represents a sequence of consecutive blocks. The other properties of the object are:

elements (array): list of sequence components. Each item in the array is an object that represents a portion of the workflow that's been executed and the value of the type property determines the other properties of the object.

SPLITTER

An object with type property equal to SPLITTER corresponds to the execution of the block of a splitter or a remapper. The other properties of the object are:

maxAdmittedErrors (number): value of the Maximum number of split element processing that can fail without propagating the error property of the End Context block placed at the end of the splitter or remapper context
maxAdmittedErrorPercentage (number): value of the Maximum percentage of split element processing that can fail without propagating the error property of the End Context block placed at the end of the splitter or remapper context
hexagonId (string): block identification code
contextId (string): identifier of the splitter or remapper context
serviceId (string): identifier of the software service corresponding to the block
mergerId (string): identification code of the End Context block placed at the end of the splitter or remapper context
input (object): the input received by the block
statistics (object): block execution statistics (see below)
items (array): items generated - in case of splitter—or inherited from a previous context—in case of remapper
splititem (object): object representing the single item
component (object): portion of the workflow representative of the splitter or reducer context executed on the item. The value of the type property determines other object properties.

REDUCER

An object with type property equal to REDUCER corresponds to the execution of a reducer type block. The other properties of the object are:

hexagonId (string): block identification code
serviceId (string): identifier of the software service corresponding to the block
contextToReduce (string): identification code of the reference context, object of reduction
input (array): each item of the array corresponds to the output of the reference context for an item of the context itself
output (object): the output produced by the block
statistics (object): block execution statistics (see below)

SWITCH

An object with type property equal to SWITCH corresponds to the execution of a Switch block. The other properties of the object are:

switchId (string): block identification code
endSwitchId (string): identification code of the End Switch block in which the conditional branches that depart from the Switch block converge
key (string): Key property of the block
value (string): value of the key upstream of the block corresponding to key
output (string): output of executed conditional branches
maxAdmittedErrors (number): value of the Maximum number of split element processing that can fail without propagating the error property of the End Switch block in which the conditional branches departing from the Switch block converge
multiple (boolean): Multiple functional property of the block
conditions (array): Conditions functional property of the block. Every item of the array is an object corresponding to a condition and has these properties:
- name (string): name of the condition
- code (string): condition code
- operator (string): condition operator
- value (string): value of the condition
- executed (boolean): true if the condition has been satisfied by the value of key, false otherwise
- component (object): present only if the condition has been satisfied, corresponds to the workflow branch that departs from the Switch block and corresponds to the condition. The value of the type property determines the other properties of the object.

FORKJOIN

An object with type property equal to FORKJOIN corresponds to the execution of a Join, Join-Fork or virtual Join block that merges the outputs of independent flows. The other properties of the object are:

maxAdmittedErrors (number): in the case of a Join or Join-Fork block, value of the Maximum number of split element processing that can fail without propagating the error property of the block, otherwise, for virtual Join, value 0
elements (array): array containing an item for each workflow branch or independent flow that converges in the block. Each item is an object that represents a portion of the workflow upstream of the block and the value of the type property determines the other properties of the object.

statistics

The properties of the statistics object provide detailed information on the operation times of a block.

To understand this information it is necessary to consider that the functioning of the workflow in asynchronous mode is managed by a software dubbed "orchestrator" and that each processing block has input and output queues.
All the data—inputs and outputs—is kept in a shared storage. When the orchestrator finishes preparing the input for a block, it puts it in the shared storage and appends a notifying message at the end of the input queue of the block.
When the block has finished processing all the previously queued requests, it uses that notification to locate the new input in the shared storage and reads it from there.
After processing the input, the block writes its output in the shared storage and queues a notifying message in the output queue. The orchestrator then makes use of that output to craft the input for downstream blocks or to produce the overall workflow's output.

Info

All the statistics whose names ends with Time are Unix timestamps in milliseconds.

queueingTime (number): the moment the orchestator wrote the message notifying the availability of a new input in the input queue of the block
messageConsumeTime (number): the moment the block started taking the input from the shared storage to validate it
logicStartTime (number): the moment the block started processing the input
logicEndTime (number): the moment the block finished processing the input and produce the corresponding output
messageProduceTime (number): the moment the block started putting the output in the shared storage
dequeuingTime (number): the moment the orchestator finds the message notifying about the availability of the block's output
logicLatencyMs (number): difference in milliseconds between logicEndTime and logicStartTime, represents the time taken by the block to process the input and produce the output
hexagonLatencyMs (number): difference in milliseconds between messageProduceTime and messageConsumeTime, represents the time elapsed from reading the input to writing the output from and to the shared storage
infrastructuralLatencyMs (number): difference in milliseconds between dequeuingTime and queueingTime, represents the time elapsed between input queuing and output dequeuing