Extract Converter processor

Description

The Extract Converter processor recognizes blocks of text in PDF documents and returns them together with supplemental information like the table of contents, the fonts, the number of pages, the author, the creation date and any PDF metadata.

The blocks of text are extracted page by page in the same order a human would read them.
When extracting text, Extract Converter can use Optical Character Recognition (OCR) to get plain text from images that represent text.

It is used as the first block of a workflow when it is certain that only PDF documents will be analyzed, as an alternative to the Tika Converter processor. Tika Converter does not have OCR and Extract Converter, especially in case of documents with complex layouts—multiple columns, text boxes, figures—returns the text in an order more similar to that in which a human would read it, therefore more suited to being analyzed effectively with a model.

Note

Extract Converter may take much longer than Tika Converter to process a document because of OCR—which is optional and disabled by default, it can be activated by configuring the processor block—and the other sophisticated analysis performed.

Input

The processor requires the input JSON to contain these top level keys:

"path": "filePath",
"base64": "base64Encoding"

where:

filePath is the name of the file to process or its path.
base64Encoding is the Base64 encoding of the file.

The value of path is only for logging or auditing purposes, it is not used to "read" the file, which instead is completely represented by the base64 value.

Block properties

Block properties can be set by editing the block.
Extract Converter workflow blocks have the following properties:

Common:
- The unique block ID and the service version, displayed in the title bar (read only, displayed also in the block tooltip in the canvas).
- Block name: the block name, it can be edited.
- Description: the description of the processor (read only).
Type Specific:
- Timeout: execution timeout expressed in minutes (m) or seconds (s).
Deployment:
- Replicas: number of required instances.
- Memory: required memory.
- CPU: thousandths of a CPU required (for example: 1000 = 1 CPUs).

Functional:

Enable or disable table and title detection: recognition of tables and titles (enabled by default).
Enable or disable OCR extraction: Optical Character Recognition applied to document's images (disabled by default).

Specify OCR language: when OCR is enabled, the language or script to be used to determine text. Possible choices are:

Value	Description
`eng`	English
`chi_sim`	Chinese (simplified)
`chi_tra`	Chinese (traditional)
`hin`	Hindi
`spa`	Spanish
`fra`	French
`ara`	Arabic
`ben`	Bengali
`rus`	Russian
`por`	Portuguese
`ind`	Indonesian
`deu`	German
`ita`	Italian
`latn`	Latin script (any language with Latin characters)

For multi-language documents, concatenate language codes with a plus character (+), for example: eng+spa+ita.

Enable/disable table of content extraction: enable or disable the table of content extraction of the analyzed document, if any (enabled by default).
Enable/disable font extraction: the fonts document are recognized and they are also placed in a specific key in the JSON output (enabled by default).
Reading order mode: parameter to set the reading order. Possible values are:
- standard: the algorithm searches for the best possible reading order, useful when documents with two or more columns or with mixed layout are considered.
- vertical: the algotithm is forced to read whenever possible from left to right, from top to bottom.
- auto: available for future use.

Input

Used for input mapping: one property for each of the top level keys of the input JSON.
If:
- The block is the first in a flow and the workflow input contains only the expected keys.
Or:
- The previous block's output contains only the expected keys.
these properties do not need to be set.
Otherwise, the properties determine which top level keys of the overall "upstream JSON" must be mapped to the block's input keys. The values of the properties must be set choosing from the compatible keys of upstream blocks' output or, if the input format of the workflow has been defined, from the keys of the $nlflow_input pseudo block.

Output

Structure

The output of an Extract Converter block is a JSON object with the following structure:

{
    "result": {
        "fonts": [],
        "header": {},
        "layout": [],
        "tableOfContents": [],
        "timeInfo": {},
        "words": []
    }
}

where:

fonts contains a list of the fonts used in the document.
header contains general information about the document and the text extraction task.
layout contains the document's layout, that is the text organized in a hierarchical structure with pages and corresponding text blocks.
tableOfContents contains the document table of contents (TOC), if any.
timeInfo contains a list of debug information related to the algorithm and its progress execution.
words contains the document's text in words, without layout information.

fonts

This array lists the fonts used in the document's text. Each item in the array represents a font and has these properties:

Key	Description
`bold`	true if bold, false otherwise
`id`	Unique font ID assigned during the analysis
`id_name`	Original font name
`italic`	true if italic, false otherwise
`name`	Normalized font name
`ocr`	true if recognized through OCR, false otherwise

For example:

{
    "bold": false,
    "id": 1,
    "id_name": "Arial",
    "italic": false,
    "name": "Arial",
    "ocr": false
}

Note

If a font is not detected the key name is set to mix.

The header object contains information about the whole document and the extraction task. The properties are:

conversionDateTime: extraction task end date and time.
customInfo: PDF document properties:
- Author: author
- CreationDate: creation date and time¹
- Creator: creator
- ModDate: last modification date and time¹
- Producer: generator application
- Title: document title
documentName: document name.
errorPages: number of pages that could not be analyzed (present only in case of errors).
options: extraction task options, for troubleshooting only.
totPages: total number of pages.
version: software version for the Extract Converter processor.

metadata: an array of PDF metadata.
Metadata is optional data that the PDF editor can insert into pages. This data is not displayed on the page, but is associated with visible elements.

Each metadata can have these properties:

bbox: array containing the coordinates² of the metadata bounding box.
- item 0: upper left corner X
- item 1: upper left corner Y
- item 2: lower right corner X
- item 3: lower right corner Y
key: metadata key, its name.
page: number of the page where the metadata is located.
value: metadata value.

For example:

"metadata": [{
        "bbox": [146, 207, 419, 228],
        "key": "txtPolicyNumber",
        "page": 3,
        "value": "PACUIC001101-07 "
    }, {
        "bbox": [39, 426, 417, 357],
        "key": "txtNamedInsuredAndAddress",
        "page": 3,
        "value": "SWEET FRUIT ASSOCIATION INC.\r\n7100 APRICOT WAY\r\nST. PETERSBURG, FL  33706 "
   }, {
        "bbox": [421, 356, 829, 438],
        "key": "AgencyNameAndAddress",
        "page": 3,
        "value": "StaySafe Insurance Services, Inc.\r\n2502 N Rodeo Drive\r\nTampa, FL  33607 "
    }, {
        "bbox": [144, 254, 283, 275],
        "key": "txtEffectiveDate",
        "page": 3,
        "value": "4/27/2022 "
    }
]

layout

layout is an array containing all the layout elements recognized in the document.
The order of the elements inside the array reflects the sequence of pages, so all the elements of page 1 are found first, then those of page 2, and so on.
Within the elements of a page, the first element represents the page itself and the other elements are blocks of text, tables or table cells. The position of text blocks and tables in the array corresponds to what Extract Converter assumed to be the order in which a human would read them on the page.

Elements can represent:

Pages (only the bounding box)
Titles
Headers
Footers
Body-level text blocks
Tables (only the bounding box)
Table cells

The properties that each element can have are:

Element type→ Property ↓	Pages	Titles	Headers & footers	Body-level text blocks	Tables	Table cells
`id`	X	X	X	X	X	X
`type`	X	X	X	X	X	X
`page`	X	X	X	X	X	X
`children`	X				X
`parent`		X	X	X	X	X
`content`		X	X	X		X
`bbox`	X	X	X	X	X	X
`label`		X
`row`						X
`column`						X
`isHead`						X
`span`						X

Properties are:

id: element ID, every element has a unique value for this property.

type: element type. Possible values are:

Element type	`type` value	Description
Pages	`page`	The element is the "container" (it has no text of its own) of all the textual elements displayed on a page.
Body-level text blocks	`text`	The element is a block of text (e.g. a paragraph, a text box) at the body-level, i.e. not a title.
Titles	`title`	The element is a heading.
Tables	`table`	The element is the "container" (it has no text of its own) of all the element (cells) of a table.
Table cells	`cell`	The element is a table's cell.
Header	`header`	The element is a page header.
Footers	`footer`	The element is a page footer.

page: page number
children: list of child blocks' IDs, only in page and table elements. This property is an array, each item of which is the value of the id property of an element that is hierarchically a child of this element. For example, the titles in a page are children of the page element, the cells of a table are children of a table element.
parent: parent element ID. In case of table cells, the value of this property is the value of the id property of the table element, while for title, text, header & footer and table elements, it is the value of the id property of a page element. Page elements don't have this property because their "parent" is the document itself.
label: for titles, it specifies the title level as in tableOfContents.
content: text of the element, this property is absent in page and table elements, which are "containers".
bbox: array containing the coordinates² of the element's bounding box.
- Item 0: upper left corner X
- Item 1: upper left corner Y
- Item 2: lower right corner X
- Item 3: lower right corner Y
row: cell row number.
column: cell column number.
isHead: set to true if the cell is a column header.
span: cell span. It's an array of integer numbers. When present, the cell spans over more than one row and/or columns. The first item of the array is the row span, the second is the column span.

Name	Description
`score`	Item recognition confidence score
`level`	Title level on the titles' hierarchy. The value of this property coincides with the value of the `label` property of the corresponding title element in `layout` (see the `layoutId` property below)
`source`	Only for troubleshooting
`layoutId`	Cross-reference to the layout element. The value of this property coincides with the value of the `id` property of the corresponding title element in `layout`
`content`	TOC item text

For example:

{
    "score": 0.8755,
    "level": 1,
    "source": "d",
    "layoutId": 2,
    "content": "UMBRELLA LIABILITY POLICY SCHEDULE"
}

words

The words array contains one item per page and each item represents, in an encoded and compressed form, all the words present on the page.

The value of the single item is encoded in Base64.
The decoded value is a byte array in gzip format. The expanded byte array value is another byte array in which each word corresponds to a variable-length sequence of bytes with this structure:

UTF-8 encoded text0x00Parent element IDFont IDBounding box coordinates

UTF-8 encoded text is the text of the word.
Parent element ID is four bytes long and must be interpreted as a little-endian integer. The value is the ID—the value of the id property—of the layout element in which the word is located.
Font ID is four bytes long and must be interpreted as a little-endian integer. The value is the ID of the font with which the word is written in the document, so it coincides with the value of the id property of the item of the fonts array that represents the font.
Bounding box coordinates is 16 bytes long and consists of four parts of four bytes each. Each part must be interpreted as a little-endian integer. The parts are the coordinates² of the word bounding box and, taken from left to right, have this meaning.
1. upper left corner X
2. upper left corner Y
3. lower right corner X
4. lower right corner Y

Output-input mapping

The result top level key in the output JSON is compatible with the documentLayout input property of model blocks.

PDF defines a standard date format similar to the international standard Abstract Syntax Notation One (ASN.1), defined in ISO/IEC 8824. A date-time is a string with this format:
```
D:YYYYMMDDHHmmSSOHH'mm'
```
where
- YYYY is the year
- MM is the month
- DD is the day (01-31)
- HH is the hour (00-23)
- mm is the minute (00-59)
- SS is the second (00-59)
- O is the relationship of local time to Universal Time (UT), denoted by one of the characters +, -, or Z (see below)
- HH followed by ' is the absolute value of the offset from UT in hours (00-23)
- mm followed by ' is the absolute value of the offset from UT in minutes (00-59)
A plus sign (+) as the value of the O field signifies that local time is later than UT, a minus sign (-) that local time is earlier than UT, and the letter Z that local time is equal to UT. If no UT information is specified, the relationship of the specified time to UT is considered to be unknown. Whether or not the time zone is known, the rest of the date is specified in local time.
For example, December 23, 2022, at 7:52 PM, U.S. Pacific Standard Time, is represented by the string:
```
D:20221223195200-08'00'
```
OR
```
D:20220327195230+05'00'
```
↩↩
Coordinates are in pixels and referred to a 100 DPI (dots per inch) rendering of the page. The coordinates origin is at the top left corner of the rendered page. ↩↩↩