Getting Started

Introduction

The Rossum API allows you to programmatically access and manage your organization's Rossum data and account information. The API allows you to do the following programmatically:

• Manage your organization, users, workspaces and queues
• Configure captured data: select extracted fields
• Integrate Rossum with other systems: import, export, extensions

On this page, you will find an introduction to the API usage from a developer perspective, and a reference to all the API objects and methods.

Developer Resources

There are several other key resources related to implementing, integrating and extending the Rossum platform:

• Refer to the Rossum Developer Portal for guides, tutorials, news and a community Q&A section.
• Subscribe to the rossum-api-announcements group to stay up to date on the API and platform updates.
• For managing and configuring your account, use the rossum command-line tool. (Setup instructions.)
• For a management overview of Rossum implementation options, see the Rossum Integration Whitepaper.
• If you are an RPA developer, refer to our UiPath or BluePrism guides.

Quick API Tutorial

For a quick tutorial on how to authenticate, upload a document and export extracted data, see the sections below. If you want to skip this quick tutorial, continue directly to the Overview section.

It is a good idea to go through the introduction to the Rossum platform on the Developer Portal first to make sure you are up to speed on the basic Rossum concepts.

If in trouble, feel free to contact us at support@rossum.ai.

Install curl tool

Test curl is installed properly

curl https://api.elis.rossum.ai/v1

{"organizations":"https://api.elis.rossum.ai/v1/organizations","workspaces":"htt
ps://api.elis.rossum.ai/v1/workspaces","schemas":"https://api.elis.rossum.ai/v1/
schemas","connectors":"https://api.elis.rossum.ai/v1/connectors","inboxes":"http
s://api.elis.rossum.ai/v1/inboxes","queues":"https://api.elis.rossum.ai/v1/queue
s","documents":"https://api.elis.rossum.ai/v1/documents","users":"https://api.el
is.rossum.ai/v1/users","groups":"https://api.elis.rossum.ai/v1/groups","annotati
ons":"https://api.elis.rossum.ai/v1/annotations","pages":"https://api.elis.rossu
m.ai/v1/pages"}

All code samples included in this API documentation use curl, the command line data transfer tool. On MS Windows 10, MacOS X and most Linux distributions, curl should already be pre-installed. If not, please download it from curl.haxx.se).

Optionally use jq tool to pretty-print JSON output

curl https://api.elis.rossum.ai/v1 | jq

{
  "organizations": "https://api.elis.rossum.ai/v1/organizations",
  "workspaces": "https://api.elis.rossum.ai/v1/workspaces",
  "schemas": "https://api.elis.rossum.ai/v1/schemas",
  "connectors": "https://api.elis.rossum.ai/v1/connectors",
  "inboxes": "https://api.elis.rossum.ai/v1/inboxes",
  "queues": "https://api.elis.rossum.ai/v1/queues",
  "documents": "https://api.elis.rossum.ai/v1/documents",
  "users": "https://api.elis.rossum.ai/v1/users",
  "groups": "https://api.elis.rossum.ai/v1/groups",
  "annotations": "https://api.elis.rossum.ai/v1/annotations",
  "pages": "https://api.elis.rossum.ai/v1/pages"
}

You may also want to install jq tool to make curl output human-readable.

Use the API on Windows

This API documentation is written for usage in command line interpreters running on UNIX based operation systems (Linux and Mac). Windows users may need to use the following substitutions when working with API:

Example of API call on UNIX-based OS

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' -H 'Content-Type: application/json' \
  -d '{"target_queue": "https://api.elis.rossum.ai/v1/queues/8236", "target_status": "to_review"}' \
  'https://api.elis.rossum.ai/v1/annotations/315777/copy'

Examples of API call on Windows

curl -H "Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03" -H "Content-Type: application/json" ^
  -d "{""target_queue"": ""https://api.elis.rossum.ai/v1/queues/8236"", ""target_status"": ""to_review""}" ^
  "https://api.elis.rossum.ai/v1/annotations/315777/copy"


curl -H "Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03" -H "Content-Type: application/json" ^
  -d "{\"target_queue\": \"https://api.elis.rossum.ai/v1/queues/8236\", \"target_status\": \"to_review\"}" ^
  "https://api.elis.rossum.ai/v1/annotations/315777/copy"

Create an account

In order to interact with the API, you need an account. If you do not have one, you can create one via our self-service portal.

Login to the account

Fill-in your username and password (login credentials to work with API are the same as those to log into your account.). Trigger login endpoint to obtain a key (token), that can be used in subsequent calls.

curl -s -H 'Content-Type: application/json' \
  -d '{"username": "east-west-trading-co@elis.rossum.ai", "password": "aCo2ohghBo8Oghai"}' \
  'https://api.elis.rossum.ai/v1/auth/login'
{"key": "db313f24f5738c8e04635e036ec8a45cdd6d6b03"}

This key will be valid for a default expire time (currently 162 hours) or until you log out from the sessions.

Upload a document

In order to upload a document (PDF or image) through the API, you need to obtain the id of a queue first.

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03'
  'https://api.elis.rossum.ai/v1/queues?page_size=1' | jq -r .results[0].url
https://api.elis.rossum.ai/v1/queues/8199

Then you can upload document to the queue. Alternatively, you can send documents to a queue-related inbox. See upload for more information about importing files.

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  -F content=@document.pdf 'https://api.elis.rossum.ai/v1/queues/8199/upload' | jq -r .results[0].annotation
https://api.elis.rossum.ai/v1/annotations/319668

Wait for document to be ready and review extracted data

As soon as a document is uploaded, it will show up in the queue and the data extraction will begin. It may take a few seconds to several minutes to process a document. You can check status of the annotation and wait until its status is changed to to_review.

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/annotations/319668' | jq .status
"to_review"

After that, you can open the Rossum web interface elis.rossum.ai to review and confirm extracted data.

Download reviewed data

Now you can export extracted data using the export endpoint of the queue. You can select XML, CSV, XLSX or JSON format. For CSV, use URL like:

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/queues/8199/export?status=exported&format=csv&id=319668'
Invoice number,Invoice Date,PO Number,Due date,Vendor name,Vendor ID,Customer name,Customer ID,Total amount,
2183760194,2018-06-08,PO2231233,2018-06-08,Alza.cz a.s.,02231233,Rossum,05222322,500.00

Logout

Finally you can dispose token safely using logout endpoint:

curl -s -X POST -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/auth/logout'
{"detail":"Successfully logged out."}

What's new

List of recent API updates related to Elis integrations and extensions.

• Relation object was introduced to track annotation dependencies (e.g. children-parent relationship of split documents). (7/15/2020)

• Automated flag was introduced to allow easy tracking of automated documents. (7/15/2020)

• It is now possible to update/add/delete multiple datapoints using a bulk operations API. (7/6/2020)

• Autopilot configuration has been moved from Workspace to Queue. (6/5/2020)

• We are renaming webhook API objects to hook in order to generalize API to allow simple integration of third-party code and services. Please note that queue object was also updated to support hook objects. (6/1/2020)

• It is now possible to pass annotation metadata during file upload. (5/29/2020)

Overview

HTTP and REST

The Rossum API is organized around REST. Our API has predictable, resource-oriented URLs, and uses HTTP response codes to indicate API errors. We use built-in HTTP features, like HTTP authentication and HTTP verbs, which are understood by off-the-shelf HTTP clients.

HTTP Verbs

Call the API using the following standard HTTP methods:

• GET to retrieve an object or multiple objects in a specific category
• POST to create an object
• PUT to modify entire object
• PATCH to modify fields of the object
• DELETE to delete an object

We support cross-origin resource sharing, allowing you to interact securely with our API from a client-side web application. JSON is returned by API responses, including errors (except when another format is requested, e.g. XML).

Base URL

Base API endpoint URL depends on the deployment type and location, e.g. https://api.elis.rossum.ai. If you are not sure about correct URL, please contact us at support@rossum.ai.

Please note that Rossum frontend may use different API endpoints, e.g. https://elis.rossum.ai/api/, but these are not meant for third-party integrations and should not be used without consent from Rossum.

Authentication

Most of the API endpoints require user to be authenticated. To login to the Rossum API, post an object with username and password fields. Login returns new authentication key to be used in token authentication.

User may delete a token using logout endpoint or automatically after a configured time (default expire time is 162 hours). Default expire time can be lowered using max_token_lifetime_s field. When token expires, 401 status is returned. Users are expected to re-login to obtain a new token.

Login

Login user using username and password

curl -H 'Content-Type: application/json' \
  -d '{"username": "east-west-trading-co@elis.rossum.ai", "password": "aCo2ohghBo8Oghai"}' \
  'https://api.elis.rossum.ai/v1/auth/login'

{
  "key": "db313f24f5738c8e04635e036ec8a45cdd6d6b03"
}

POST /v1/auth/login

Use token key in requests

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/organizations/406'

Login user expiring after 1 hour

curl -H 'Content-Type: application/json' \
  -d '{"username": "east-west-trading-co@elis.rossum.ai", "password": "aCo2ohghBo8Oghai", "max_token_lifetime_s": 3600}' \
  'https://api.elis.rossum.ai/v1/auth/login'

{
  "key": "ltcg2p2w7o9vxju313f04rq7lcc4xu2bwso423b3"
}

Response

Status: 200

Returns object with "key", which is an actual authentication token.

Logout

Logout user

curl -X POST -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/auth/logout'

{
  "detail": "Successfully logged out."
}

POST /v1/auth/logout

Logout user, discard auth token.

Response

Status: 200

Single Sign-On (SSO)

Rossum allows customers to integrate with their own identity provider, such as Google, Azure AD or any other provider using OAuth2 OpenID Connect protocol (OIDC). Rossum then acts as a service provider.

When SSO is enabled for an organization, user is redirected to a configured identity provider login page and only allowed to access Rossum application when successfully authenticated. Identity provider user claim (e.g. email (default), sub, preferred_username, unique_name) is used to match a user account in Rossum. Please note that Rossum user accounts still must be created for every user before logging to Rossum.

Required setup of the OIDC identity provider:

• Redirect URI (also known as Reply URL): https://elis.rossum.ai/api/oauth/code

Required information to allow OIDC setup for the Rossum service provider:

• OIDC endpoint, such as https://accounts.google.com. It should support openid configuration, e.g. https://accounts.google.com/.well-known/openid-configuration
• client id
• client secret
• claim that should be matched in Rossum
• Rossum organization id

If you need to setup SSO for your organization, please contact support@rossum.ai.

Pagination

All object list operations are paged by default, so you may need several API calls to obtain all objects of given type.

(*) Maximum page size of annotation list is 1000 for CSV export, 100 for other formats.

Filters and ordering

List queues of workspace 7540, with locale en_US and order results by name.

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/queues?workspace=7540&locale=en_US&ordering=name'

Lists may be filtered using various attributes. Multiple attributes are combined with & in the URL, which results in more specific response. Please refer to the particular object description.

All filter parameters that refers to object expect only id (instead of URL) of the object.

Ordering of results may be enforced by the ordering parameter and one or more keys delimited by a comma. Preceding key with a minus sign - enforces descending order.

Metadata

Example metadata in a document object

{
  "id": 319768,
  "url": "https://api.elis.rossum.ai/v1/documents/319768",
  "s3_name": "05feca6b90d13e389c31c8fdeb7fea26",
  "annotations": [
    "https://api.elis.rossum.ai/v1/annotations/319668"
  ],
  "mime_type": "application/pdf",
  "arrived_at": "2019-02-11T19:22:33.993427Z",
  "original_file_name": "document.pdf",
  "content": "https://api.elis.rossum.ai/v1/documents/319768/content",
  "metadata": {
    "customer-id": "f205ec8a-5597-4dbb-8d66-5a53ea96cdea",
    "source": 9581,
    "authors": ["Joe Smith", "Peter Doe"]
  }
}

When working with API objects, it may be useful to attach some information to the object (e.g. customer id to a document). You can store custom JSON object in a metadata section available in most objects.

List of objects with metadata support: organization, workspace, user, queue, schema, connector, inbox, document, annotation, page.

Total metadata size may be up to 1 kB per object.

Versioning

API Version is part of the URL, e.g. https://api.elis.rossum.ai/v1/users.

To allow API progress, we consider addition of a field in a JSON object to be a backward-compatible operation that may be introduced at any time. Clients are expected to deal with such changes.

In order to be notified about future API changes and improvements, please subscribe to the rossum-api-announcements group.

Dates

All dates fields are represented as ISO 8601 formatted strings, e.g. 2018-06-01T21:36:42.223415Z. All returned dates are in UTC timezone.

Errors

Our API uses conventional HTTP response codes to indicate the success or failure of an API request.

Import and Export

Documents may be imported into Rossum using the REST API and email gateway. Supported file formats are PDF, PNG, JPEG and TIFF.`

In order to get the best results from Rossum the documents should be in A4 format of at least 150 DPI (in case of scans/photos). Read more about import recommendations.

Upload

You can upload a document to the queue using upload endpoint with one or more files to be uploaded. You can also specify additional field values in upload endpoint, e.g. your internal document id. As soon as a document is uploaded, data extraction is started.

Upload endpoint supports basic authentication to enable easy integration with third-party systems.

Import by Email

It is also possible to send documents by email using a properly configured inbox that is associated with a queue. Users then only need to know the email address to forward emails to.

For every incoming email, Rossum extracts PDF documents and images, stores them in the queue and starts data extraction process.

Small images (up to 100x100 pixels) are ignored, see inbox for reference.

You can use selected email header data (e.g. Subject) to initialize additional field values, see rir_field_names attribute description for details.

Export

In order to export extracted and confirmed data you can call export endpoint. You can specify status, time-range filters and annotation id list to limit returned results.

Export endpoint supports basic authentication to enable easy integration with third-party systems.

Auto-split of document

It is possible to process a single PDF file that contains several invoices. Just insert a special separator page between the documents. You can print this page and insert it between documents while scanning.

Rossum will recognize a QR code on the page and split the PDF into individual documents automatically. Produced documents are imported to the queue, while the original document is set to a split state.

Document Schema

Every queue has an associated schema that specifies which fields will be extracted from documents as well as the structure of the data sent to connector and exported from the platform.

Rossum schema supports data fields with single values (datapoint), fields with multiple values (multivalue) or tuples of fields (tuple). At the topmost level, each schema consists of sections, which may either directly contain actual data fields (datapoints) or use nested multivalues and tuples as containers for single datapoints.

But while schema may theoretically consist of an arbitrary number of nested containers, the Rossum UI supports only certain particular combinations of datapoint types. The supported shapes are:

• simple: atomic datapoints of type number, string, date or enum
• list: simple datapoint within a multivalue
• tabular: simple datapoint within a "multivalue tuple" (a multivalue list containing a tuple for every row)

Schema content

Schema content consists of a list of section objects.

Common attributes

The following attributes are common for all schema objects:

Section

Example of a section

{
    "category": "section",
    "id": "amounts_section",
    "label": "Amounts",
    "children": [...],
    "icon": ""
}

Section represents a logical part of the document, such as amounts or vendor info. It is allowed only at the top level. Schema allows multiple sections, and there should be at least one section in the schema.

Datapoint

A datapoint represents a single value, typically a field of a document or some global document information. Fields common to all datapoint types:

rir_field_names attribute allows to specify source of initial value of the object. List items may be:

• one of extracted field types to use the AI engine prediction value
• upload:id to identify a value specified while uploading the document
• email_header:<id> to use a value extracted from email headers. Supported email headers: from, to, subject, message-id, date.
• email_body:<id> to select email body. Supported values are text_html for HTML body or text_plain for plain text body.
• email:<id> to identify a value specified in email.received hook response

If more list items in rir_field_names are specified, the first available value will be used.

String type

Example string datapoint

{
    "category": "datapoint",
    "id": "document_id",
    "label": "Invoice ID",
    "type": "string",
    "default_value": null,
    "rir_field_names": ["document_id"],
    "constraints": {
        "length": {
            "exact": null,
            "max": 16,
            "min": null
        },
        "regexp": {
            "pattern": "^INV[0-9]+$"
        },
        "required": false
    }
}

String datapoint does not have any special attribute.

Date type

Example date datapoint

{
  "id": "item_delivered",
  "type": "date",
  "label": "Item Delivered",
  "format": "MM/DD/YYYY",
  "category": "datapoint"
}

Attributes specific to Date datapoint:

Date format supported: available tokens

Example date formats:

• D/M/YYYY: e.g. 23/1/2019
• MM/DD/YYYY: e.g. 01/23/2019
• YYYY-MM-DD: e.g. 2019-01-23 (ISO date format)

Number type

Example number datapoint

{
  "id": "item_quantity",
  "type": "number",
  "label": "Quantity",
  "format": "#,##0.#",
  "category": "datapoint"
}

Attributes specific to Number datapoint:

The following table shows numeric formats with their examples.

Aggregations

Example aggregations

{
  "id": "quantity",
  "type": "number",
  "label": "Quantity",
  "category": "datapoint",
  "aggregations": {
    "sum": {
      "label": "Total"
    }
  },
  "default_value": null,
  "rir_field_names": []
}

Aggregations allow computation of some informative values, e.g. a sum of a table column with numeric values. These are returned among messages when /annotations/{id}/validate endpoint is called. Aggregations can be computed only for tables (multivalues of tuples).

All aggregation objects can have an attribute label that will be shown in the UI.

Enum type

Example enum datapoint with options

{
  "id": "document_type",
  "type": "enum",
  "label": "Document type",
  "hidden": false,
  "category": "datapoint",
  "options": [
    {
      "label": "Invoice Received",
      "value": "21"
    },
    {
      "label": "Invoice Sent",
      "value": "22"
    },
    {
      "label": "Receipt",
      "value": "23"
    }
  ],
  "default_value": "21",
  "rir_field_names": []
}

Attributes specific to Enum datapoint:

Every option consists of an object with keys:

Enum datapoint value is matched in a case insensitive mode, e.g. EUR currency value returned by the AI Core Engine is matched successfully against {"value": "eur", "label": "Euro"} option.

Button type

Specifies a button shown in Rossum UI. For more details please refer to custom UI extension.

Example button datapoint

{
  "id": "show_email",
  "type": "button",
  "category": "datapoint",
  "popup_url": "http://example.com/show_customer_data",
}

Despite being a datapoint object, button currently cannot hold any value. Therefore, the set of available Button datapoint attributes is limited to:

Value constraints

Example value constraints

{
  "id": "document_id",
  "type": "string",
  "label": "Invoice ID",
  "category": "datapoint",
  "constraints": {
    "length": {
      "max": 32,
      "min": 5
    },
    "required": false
  },
  "default_value": null,
  "rir_field_names": [
    "document_id"
  ]
}

Constraints limit allowed values. When constraints is not satisfied, annotation is considered invalid and cannot be exported.

Multivalue

Example of a multivalue:

{
  "category": "multivalue",
  "id": "line_item",
  "label": "Line Item",
  "children": {
    ...
  },
  "show_grid_by_default": false,
  "min_occurrences": null,
  "max_occurrences": null,
  "rir_field_names": null
}

Example of a multivalue with grid row-types specification:

{
  "category": "multivalue",
  "id": "line_item",
  "label": "Line Item",
  "children": {
    ...
  },
  "grid": {
    "row_types": [
      "header", "data", "footer"
    ],
    "default_row_type": "data",
    "row_types_to_extract": [
      "data"
    ]
  },
  "min_occurrences": null,
  "max_occurrences": null,
  "rir_field_names": ["line_items"]
}

Multivalue is list of datapoints or tuples of the same type. It represents a container for data with multiple occurrences (such as line items) and can contain only objects with the same id.

Multivalue grid object

Multivalue grid object allows to specify a row type for each row of the grid. For data representation of actual grid data rows see Grid object description.

For example to distinguish two header types and a footer in the validation interface, following row types may be used: header, subsection_header, data and footer.

Currently, data extraction classifies every row as either data or header (additional row types may be introduced in the future). We remove rows returned by data extraction that are not in row_types list (e.g. header by default) and are on the top/bottom of the table. When they are in the middle of the table, we mark them as skipped (null).

There are three visual modes, based on row_types quantity:

• More than two row types defined: User selects row types freely to any non-default row type. Clearing row type resets to a default row type. We support up to 6 colors to easily distinguish row types visually.
• Two row types defined (header and default): User only marks header and skipped rows. Clearing row type resets to a default row type.
• One row type defined: User is only able to mark row as skipped (null value in data). This is also a default behavior when no grid row types configuration is specified in the schema.

Tuple

Example of a tuple:

{
  "category": "tuple",
  "id": "tax_details",
  "label": "Tax Details",
  "children": [
    ...
  ],
  "rir_field_names": [
    "tax_details"
  ]
}

Container representing tabular data with related values, such as tax details. A tuple must be nested within a multivalue object, but unlike multivalue, it may consist of objects with different ids.

Updating Schema

When project evolves, it is a common practice to enhance or change the extracted field set. This is done by updating the schema object.

By design, Rossum supports multiple schema versions at the same time. However, each document annotation is related to only one of those schemas. If the schema is updated, all related document annotations are updated accordingly. See preserving data on schema change below for limitations of schema updates.

In addition, every queue is linked to a schema, which is used for all newly imported documents.

When updating a schema, there are two possible approaches:

• Update the schema object (PUT/PATCH). All related annotations will be updated to match current schema shape (even exported and deleted documents).
• Create a new schema object (POST) and link it to the queue. In such case, only newly created objects will use the current schema. All previously created objects will remain in the shape of their linked schema.

Use case 1 - Initial setting of a schema

• Situation: User is initially setting up the schema. This might be an iterative process.
• Recommendation: Edit the existing schema by updating schema (PUT) or updating part of a schema (PATCH).

Use case 2 - Updating attributes of a field (label, constraints, options, etc.)

• Situation: User is updating field, e.g. changing label, number format, constraints, enum options, hidden flag, etc.
• Recommendation: Edit existing schema (see Use case 1).

Use case 3 - Adding new field to a schema, even for already imported documents.

• Situation: User is extending a production schema by adding a new field. Moreover, user would like to see all annotations (to_review, postponed, exported, deleted, etc. states) in the look of the newly extended schema.
• Recommendation: Edit existing schema (see Use case 1). Data of already created annotations will be transformed to the shape of the updated schema. New fields of annotations in to_review and postponed state that are linked to extracted fields types will be filled by AI Engine, if available. New fields for already exported or deleted annotations (also purged, exporting and failed_export) will be filled with empty or default values.

Use case 4 - Adding new field to schema, only for newly imported documents

• Situation: User is extending a production schema by adding a new field. However, with the intention that the user does not want to see the newly added field on previously created annotations.
• Recommendation: Create a new schema object (POST) and link it to the queue. Annotation data of previously created annotations will be preserved according to the original schema. This approach is recommended if there is an organizational need to keep different field sets before and after the schema update.

Use case 5 - Deleting schema field, even for already imported documents.

• Situation: User is changing a production schema by removing a field that was used previously. However, user would like to see all annotations (to_review, postponed, exported, deleted, etc. states) in the look of the newly extended schema. There is no need to see the original fields in already exported annotations.
• Recommendation: Edit existing schema (see Use case 1).

Use case 6 - Deleting schema field, only for newly imported documents

• Situation: User is changing a production schema by removing a field that was used previously. However, with the intention that the user will still be able to see the removed fields on previously created annotations.
• Recommendation: Create a new schema object (see Use case 4). Annotation data of previously created annotations will be preserved according to the original schema. This approach is recommended if there is an organizational need to retrieve the data in the original state.

Preserving data on schema change

In order to transfer annotation field values properly during the schema update, a datapoint's category and schema_id must be preserved.

Supported operations that preserve fields values are:

• adding a new datapoint (filled from AI Engine, if available)
• reordering datapoints on the same level
• moving datapoints from section to another section
• moving datapoints to and from a tuple
• reordering datapoints inside a tuple
• making datapoint a multivalue (original datapoint is the only value in a new multivalue container)
• making datapoint non-multivalue (only first datapoint value is preserved)

Extracted field types

AI engine currently automatically extracts the following fields at the all endpoint, subject to ongoing expansion.

Identifiers

Example of a schema with different identifiers:

[
  {
    "category": "section",
    "children": [
      {
        "category": "datapoint",
        "constraints": {
          "required": false
        },
        "default_value": null,
        "id": "document_id",
        "label": "Invoice number",
        "rir_field_names": [
          "document_id"
        ],
        "type": "string"
      },
      {
        "category": "datapoint",
        "constraints": {
          "required": false
        },
        "default_value": null,
        "format": "D/M/YYYY",
        "id": "date_issue",
        "label": "Issue date",
        "rir_field_names": [
          "date_issue"
        ],
        "type": "date"
      },
      {
        "category": "datapoint",
        "constraints": {
          "required": false
        },
        "default_value": null,
        "id": "terms",
        "label": "Terms",
        "rir_field_names": [
          "terms"
        ],
        "type": "string"
      }
    ],
    "icon": null,
    "id": "invoice_info_section",
    "label": "Basic information"
  }
]

Document attributes

Amounts

Typical relations (may depend on local laws):

amount_total = amount_total_base + amount_total_tax
amount_rounding = amount_total - round(amount_total)
amount_due = amount_total - amount_paid + amount_rounding

All amounts are in the main currency of the invoice (as identified in the currency response field). Amounts in other currencies are generally excluded.

Tables

At the moment, the AI engine automatically extracts 2 types of tables. In order to pick one of the possible choices, set rir_field_names attribute on multivalue.

Tax details

Example of a tax details table:

{
    "category": "section",
    "children": [
      {
        "category": "multivalue",
        "children": {
          "category": "tuple",
          "children": [
            {
              "category": "datapoint",
              "constraints": {
                "required": false
              },
              "default_value": null,
              "format": "# ##0.#",
              "id": "vat_detail_rate",
              "label": "VAT rate",
              "rir_field_names": [
                "tax_detail_rate"
              ],
              "type": "number",
              "width": 15
            },
            ...
          ],
          "id": "vat_detail",
          "label": "VAT detail"
        },
        "default_value": null,
        "id": "vat_details",
        "label": "VAT details",
        "max_occurrences": null,
        "min_occurrences": null,
        "rir_field_names": [
          "tax_details"
        ]
      }
    ],
    "icon": null,
    "id": "amounts_section",
    "label": "Amounts section"
}

Tax details table and breakdown by tax rates.

Line items

Example of a line items table:

    {
    "category": "section",
    "children": [
      {
        "category": "multivalue",
        "children": {
          "category": "tuple",
          "children": [
            {
              "category": "datapoint",
              "constraints": {
                "required": true
              },
              "default_value": null,
              "id": "item_desc",
              "label": "Description",
              "rir_field_names": [
                "table_column_description"
              ],
              "type": "string",
              "stretch": true
            },
            {
              "category": "datapoint",
              "constraints": {
                "required": false
              },
              "default_value": null,
              "format": "# ##0.#",
              "id": "item_quantity",
              "label": "Quantity",
              "rir_field_names": [
                "table_column_quantity"
              ],
              "type": "number",
              "width": 15
            },
            {
              "category": "datapoint",
              "constraints": {
                "required": false
              },
              "default_value": null,
              "format": "# ##0.#",
              "id": "item_amount_total",
              "label": "Price w tax",
              "rir_field_names": [
                "table_column_amount_total"
              ],
              "type": "number"
            }
          ],
          "id": "line_item",
          "label": "Line item",
          "rir_field_names": []
        },
        "default_value": null,
        "id": "line_items",
        "label": "Line item",
        "max_occurrences": null,
        "min_occurrences": null,
        "rir_field_names": [
          "line_items"
        ]
      }
    ],
    "icon": null,
    "id": "line_items_section",
    "label": "Line items"
  }

AI engine currently automatically extracts line item table content and recognizes row and column types as detailed below. Invoice line items come in a wide variety of different shapes and forms. The current implementation can deal with (or learn) most layouts, with borders or not, different spacings, header rows, etc. We currently make two further assumptions:

• The table generally follows a grid structure - that is, columns and rows may be represented as rectangle spans. In practice, this means that we may currently cut off text that overlaps from one cell to the next column. We are also not optimizing for table rows that are wrapped to multiple physical lines.
• The table contains just a flat structure of line items, without subsection headers, nested tables, etc.

We plan to gradually remove both assumptions in the future.

Annotation Lifecycle

When a document is submitted to Rossum within a given queue, an annotation object is assigned to it. An annotation goes through a variety of states as it is processed, and eventually exported.

Usage report

In order to obtain an overview of the Rossum usage, you can download Excel file with basic Rossum statistics.

The statistics contains following attributes:

• Username (may be empty in case document was not modified by any user)
• Workspace name
• Queue name
• Imported: count of all documents that were imported during the time period
• Deleted: count of documents that were deleted during the time period
• Exported: count of documents that were successfully exported during the time period
• Net time: total time spent by a user validating the successfully exported documents

Download usage statistics (January 2019).

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/manage/usage-report?from=2019-01-01&to=2019-01-31'

Excel file (xlsx) may be downloaded from https://api.elis.rossum.ai/manage/usage-report.

You may specify date range using from and to parameters (inclusive). If not specified, a report for last 12 months is generated.

New API

A new API, which contains more information, is being introduced. Eventually, it will replace the original https://api.elis.rossum.ai/manage/usage-report usage report.

Download usage report (December 2019 - February 2020).

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' -H 'Content-Type: application/json' \
  'https://api.elis.rossum.ai/v1/annotations/usage-report' -d '{"filter": {"begin_date": "2019-12-01", "end_date": "2020-01-31"}}'

Request

POST /v1/annotations/usage-report

{
  "filter": {
    "users": [
      "https://api.elis.rossum.ai/v1/users/173"
    ],
    "queues": [
      "https://api.elis.rossum.ai/v1/queues/8199"
    ],
    "begin_date": "2019-12-01",
    "end_date": "2020-01-31"
  },
  "exported_on_time_threshold_s": 86400,
  "group_by": [
    "user",
    "workspace",
    "queue",
    "month"
  ]
}

Response

Status: 200

{
  "series": [
    {
      "begin_date": "2019-12-01",
      "end_date": "2020-01-01",
      "queue": "https://api.elis.rossum.ai/v1/queues/8199",
      "workspace": "https://api.elis.rossum.ai/v1/workspaces/7540",
      "values": {
        "imported_count": 2,
        "deleted_count": null,
        "exported_count": null,
        "turnaround_avg_s": null,
        "corrections_per_document_avg": null,
        "exported_on_time_count": null,
        "exported_late_count": null,
        "time_per_document_avg_s": null,
        "time_and_corrections_per_field": []
      }
    },
    {
      "begin_date": "2020-01-01",
      "end_date": "2020-02-01",
      "queue": "https://api.elis.rossum.ai/v1/queues/8199",
      "workspace": "https://api.elis.rossum.ai/v1/workspaces/7540",
      "user": "https://api.elis.rossum.ai/v1/users/173",
      "values": {
        "imported_count": null,
        "deleted_count": 2,
        "exported_count": 2,
        "turnaround_avg_s": 1341000,
        "corrections_per_document_avg": 1.0,
        "exported_on_time_count": 1,
        "exported_late_count": 1,
        "time_per_document_avg_s": 70.0,
        "time_and_corrections_per_field": [
          {
            "schema_id": "date_due",
            "label": "Date due",
            "total_count": 1,
            "corrected_ratio": 0.0,
            "time_spent_avg_s": 0.0
          },
          ...
        ]
      }
    },
    ...
  ],
  "totals": {
    "imported_count": 7,
    "deleted_count": 2,
    "exported_count": 3,
    "turnaround_avg_s": 894000,
    "corrections_per_document_avg": 1.0,
    "exported_on_time_count": 2,
    "exported_late_count": 1,
    "time_per_document_avg_s": 70.0
  }
}

The response consists of two parts: totalsand series.

Totals

Totals contain summary information for the whole period (between begin_date and end_date).

Series

Series contain information grouped by fields defined in group_by. The data (see above) are wrapped in values object, and accompanied by the values of of attributes that were used for grouping.

In addition to the totals data, series contain time_and_corrections_per_field list that provides detailed data about statistics on each field.

Series details

The detail object contains statistics grouped per field (schema_id).

Extensions

The Rossum platform may be extended via third-party, externally running services or custom functions. These extensions are registered to receive callbacks from the Rossum platform on various occasions and allow to modify the platform behavior. Currently we support these callback extensions: Webhooks, Serverless Functions, and Connectors.

Webhooks and connectors require a third-party service accessible through a HTTP endpoint. This may incur additional operational and implementation costs. User-defined serverless functions, on the contrary, are executed within Rossum platform and no additional setup is necessary. They share the interface (input and output data format, error handling) with webhooks.

See the Building Your Own Extension set of guides in Rossum's developer portal for an introduction to Rossum extensions.

Webhook Extension

The webhook component is the most flexible extension. It covers all the most frequent use cases:

• displaying messages to users in validation screen,
• applying custom business rules to check, change, or autofill missing values,
• reacting to document status change in your workflow,
• sending reviewed data to an external systems.

Implement a webhook

Webhooks are designed to be implemented using a push-model using common HTTPS protocol. When an event is triggered, the webhook endpoint is called with a relevant request payload. The webhook must be deployed with a public IP address so that the Rossum platform can call its endpoints; for testing, a middleware like ngrok or serveo may come useful.

Webhook vs. Connector

Webhook extensions are similar to connectors, but they are more flexible and easier to use. A webhook is notified when a defined type of webhook event occurs for a related queue.

Advantages of webhooks over connectors:

• There may be multiple webhooks defined for a single queue
• No hard-coded endpoint names (/validate, /save)
• Robust retry mechanism in case of webhook failure

Webhooks are defined using a hook object of type webhook. For a description how to create and manage hooks, see the Hook API.

Webhook Events

Example data sent for annotation_status event to the hook.config.url when status of the annotation changes

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5514b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/789",
  "action": "changed",
  "event": "annotation_status",
  "annotation": {
    "document": "https://api.elis.rossum.ai/v1/documents/314621",
    "id": 314521,
    "queue": "https://api.elis.rossum.ai/v1/queues/8236",
    "schema": "https://api.elis.rossum.ai/v1/schemas/223",
    "pages": [
      "https://api.elis.rossum.ai/v1/pages/551518"
    ],
    "modifier": null,
    "modified_at": null,
    "confirmed_at": null,
    "exported_at": null,
    "assigned_at": null,
    "status": "to_review",
    "previous_status": "importing",
    "rir_poll_id": "54f6b91cfb751289e71ddf12",
    "messages": null,
    "url": "https://api.elis.rossum.ai/v1/annotations/314521",
    "content": "https://api.elis.rossum.ai/v1/annotations/314521/content",
    "time_spent": 0,
    "metadata": {}
  },
  "document": {
    "id": 314621,
    "url": "https://api.elis.rossum.ai/v1/documents/314621",
    "s3_name": "272c2f41ae84a4f19a422cb432a490bb",
    "mime_type": "application/pdf",
    "arrived_at": "2019-02-06T23:04:00.933658Z",
    "original_file_name": "test_invoice_1.pdf",
    "content": "https://api.elis.rossum.ai/v1/documents/314621/content",
    "metadata": {}
  }
}

Example data sent for annotation_content event to the hook.config.url when user updates a value in UI

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5214b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/781",
  "action": "user_update",
  "event": "annotation_content",
  "annotation": {
    "document": "https://api.elis.rossum.ai/v1/documents/314621",
    "id": 314521,
    "queue": "https://api.elis.rossum.ai/v1/queues/8236",
    "schema": "https://api.elis.rossum.ai/v1/schemas/223",
    "pages": [
      "https://api.elis.rossum.ai/v1/pages/551518"
    ],
    "modifier": null,
    "modified_at": null,
    "confirmed_at": null,
    "exported_at": null,
    "assigned_at": null,
    "status": "to_review",
    "previous_status": "importing",
    "rir_poll_id": "54f6b91cfb751289e71ddf12",
    "messages": null,
    "url": "https://api.elis.rossum.ai/v1/annotations/314521",
    "content": [
        {
            "id": 1123123,
            "url": "https://api.elis.rossum.ai/v1/annotations/314521/content/1123123",
            "schema_id": "basic_info",
            "category": "section",
            "children": [
              {
                "id": 20456864,
                "url": "https://api.elis.rossum.ai/v1/annotations/1/content/20456864",
                "content": {
                  "value": "18 492.48",
                  "normalized_value": "18492.48",
                  "page": 2,
                  ...
              },
            "category": "datapoint",
            "schema_id": "number",
            "validation_sources": [
              "checks",
              "score"
            ],
            "time_spent": 0
              }
            ]
        }
    ],
    "time_spent": 0,
    "metadata": {}
  },
  "document": {
    "id": 314621,
    "url": "https://api.elis.rossum.ai/v1/documents/314621",
    "s3_name": "272c2f41ae84a4f19a422cb432a490bb",
    "mime_type": "application/pdf",
    "arrived_at": "2019-02-06T23:04:00.933658Z",
    "original_file_name": "test_invoice_1.pdf",
    "content": "https://api.elis.rossum.ai/v1/documents/314621/content",
    "metadata": {}
  },
  "updated_datapoints": [11213211]
}

Example of a response for annotation_content hook

{
  "messages": [
    {
      "content": "Invalid invoice number format",
      "id": 197467,
      "type": "error"
    }
  ],
  "operations": [
    {
      "op": "replace",
      "id": 198143,
      "value": {
        "content": {
          "value": "John",
          "position": [103, 110, 121, 122],
          "page": 1
        },
        "hidden": false,
        "options": [],
        "validation_sources": ["human"]
      }
    },
    {
      "op": "remove",
      "id": 884061
    },
    {
      "op": "add",
      "id": 884060,
      "value": [
        {
          "schema_id": "item_description",
          "content": {
            "page": 1,
            "position": [162, 852, 371, 875],
            "value": "Bottle"
          }
        }
      ]
    }
  ]
}

Example data sent for email event to the hook.config.url when email is received by Rossum mail server

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5214b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/781",
  "action": "received",
  "event": "email",
  "email": "https://api.elis.rossum.ai/v1/emails/987",
  "files": [
    {
      "id": "1",
      "filename": "image.png",
      "mime_type": "image/png",
      "n_pages": 1,
      "height_px": 100.0,
      "width_px": 150.0,
      "document": "https://api.elis.rossum.ai/v1/documents/427"
    },
    {
      "id": "2",
      "filename": "MS word.docx",
      "mime_type": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
      "n_pages": 1,
      "height_px": null,
      "width_px": null,
      "document": "https://api.elis.rossum.ai/v1/documents/428"
    },
    {
      "id": "3",
      "filename": "A4 pdf.pdf",
      "mime_type": "application/pdf",
      "n_pages": 3,
      "height_px": 3510.0,
      "width_px": 2480.0,
      "document": "https://api.elis.rossum.ai/v1/documents/429"
    },
    {
      "id": "4",
      "filename": "unknown_file",
      "mime_type": "text/xml",
      "n_pages": 1,
      "height_px": null,
      "width_px": null,
      "document": "https://api.elis.rossum.ai/v1/documents/430"
    }
  ],
  "headers": {
    "from": "test@example.com",
    "to": "east-west-trading-co-a34f3a@elis.rossum.ai",
    "subject": "Some subject",
    "date": "Mon, 04 May 2020 11:01:32 +0200",
    "message-id": "15909e7e68e4b5f56fd78a3b4263c4765df6cc4d"
  },
  "body": {
    "body_text_plain": "Some body",
    "body_text_html": "<div dir=\"ltr\">Some body</div>"
  }
}

Example of a response for email hook

{
  "files": [
    {
      "id": "3",
      "values": [
        {
          "id": "email:invoice_id",
          "value": "INV001234"
        },
        {
          "id": "email:customer_name",
          "value": "John Doe"
        }
      ]
    }
  ]
}

Webhook events specify when the hook should be notified. They can be defined as following:

• either as whole event containing all supported actions for its type (for example annotation_status)
• or as separately named actions for specified event (for example annotation_status.changed)

Supported events and their actions

• When annotation_content.export action fails, annotation is switched to the failed_export state.
• In case a non-interactive webhook call fails (HTTP status >= 400), it is retried within 30 seconds. There are up to 10 attempts performed.
• updated_datapoints list is always empty for annotation_content.initialize and annotation_content.export actions.
• updated_datapoints list may also be empty for annotation_content.user_update in case of an action triggered interactively by a user, but with no data changes (e.g. first validation after opening a validation UI or changes confirmation).
• Retry mechanism is temporarily disabled for email.received action. Meanwhile, we are working on a more robust and reliable way of processing email webhooks.

Annotation status event data format

annotation_status event contains following additional event specific attributes.

^* Attribute is only included in the request when specified in hook.sideload.

Example data sent to hook with sideloaded queue objects

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5214b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/781",
  "action": "changed",
  "event": "annotation_status",
  ...
  "queues": [
    {
      "id": 8198,
      "name": "Received invoices",
      "url": "https://api.elis.rossum.ai/v1/queues/8198",
      ...
      "metadata": {},
      "use_confirmed_state": false,
      "settings": {}
    }
  ]
}

Annotation content event data format

annotation_content event contains following additional event specific attributes.

^* Attribute is only included in the request when specified in hook.sideload.

Annotation content event response format

All of the annotation_content events expect a JSON object with the following optional lists in the response: messages and operations

The message object contains attributes:

For example, you may use error for fatals like a missing required field, whereas info is suitable to decorate a supplier company id with its name as looked up in the suppliers database.

The operations object describes operation to be performed on the annotation data (replace, add, remove). Format of the operations key is the same as for bulk update of annotations, please refer to the annotation data API for complete description.

Email received event data format

email event contains following additional event specific attributes.

The files object contains attributes:

The headers object contains the same values as are available for initialization of values in email_header:<id> (namely: from, to, subject, message-id, date).

The body object contains the body_text_plain and body_text_html.

Email received event response format

All of the email events expect a JSON object with the following list in the response: files

The files object contains attributes:

The values object consists of following:

This is useful for filtering out unwanted files by some measures that are not available in Rossum by default.

Validating payloads from Rossum

Example of hook receiver, which verifies the validity of Rossum request

import hashlib
import hmac

from flask import Flask, request, abort

app = Flask(__name__)

SECRET_KEY = "<Your secret key stored in hook.config.secret>"  # never store this in code

@app.route("/test_hook", methods=["POST"])
def test_hook():
    digest = hmac.new(SECRET_KEY.encode(), request.data, hashlib.sha1).hexdigest()
    try:
        prefix, signature = request.headers["X-Elis-Signature"].split("=")
    except ValueError:
        abort(401, "Incorrect header format")

    if not (prefix == "sha1" and hmac.compare_digest(signature, digest)):
        abort(401, "Authorization failed.")
    return

For authorization of payloads, the shared secret method is used. When a secret token is set in hook.config.secret, Rossum uses it to create a hash signature with each payload. This hash signature is passed along with each request in the headers as X-Elis-Signature.

The goal is to compute a hash using hook.config.secret and the request body, and ensure that the signature produced by Rossum is the same. Rossum uses HMAC SHA1 signature.

Webhook requests may be autenticated using a client SSL certificate, see Hook API for reference.

Access to Rossum API

You can access Rossum API from the Webhook. Each execution gets unique API key valid for 10 minutes. Rossum API key is passed to webhooks as a first-level attribute rossum_authorization_token within the webhook payload.

Serverless Function Extension

Serverless functions allows to extend Rossum functionality without setup and maintenance of additional services.

Webhooks and Serverless functions share a basic setup: input and output data format and error handling. They are both configured using a hook API object.

Unlike webhooks, serverless functions do not send the event and action notifications to a specific URL. Instead, the function's code snippet is executed within the Rossum platform. See function API description for details about how to setup a serverless function and connect it to the queue.

Supported events and their actions

For description of supported events, actions and input/output data examples, please refer to Webhook Extensions section.

Supported runtimes

Currently Rossum supports NodeJS 12 runtime (nodejs12.x) to execute JavaScript functions and (python3.8) to execute Python. If you would like to use another runtime, please let us know at product@rossum.ai.

Please be aware that we may eventually deprecate and remove runtimes in the future (deprecation will be announced at least 6 months before the deprecation date).

Limitations

Serverless functions do not have internet access by default. If your functions require internet access to work properly, e.g. when exporting data over API to ERP system, please let us know at product@rossum.ai.

Implementation

Example serverless function usable for annotation_content event (JavaScript implementation).

// This serverless function example can be used for annotation_content events
// (e.g. user_update action). annotation_content events provide annotation
// content tree as the input.
//
// The function below shows how to:
// 1. Display a warning message to the user if "item_amount_base" field of
//    a line item exceeds a predefined threshold
// 2. Removes all dashes from the "invoice_id" field
//
// item_amount_base and invoice_id should be fields defined in a schema.

// --- ROSSUM HOOK REQUEST HANDLER ---

// The rossum_hook_request_handler is an obligatory main function that accepts
// input and produces output of the rossum serverless function hook. Currently,
// the only available programming language is Javascript executed on Node.js 12 environment.
// @param {Object} annotation - see https://api.elis.rossum.ai/docs/#annotation-content-event-data-format
// @returns {Object} - the messages and operations that update the annotation content

exports.rossum_hook_request_handler = ({
  annotation: {
    content
  }
}) => {
  try {
    // Values over the threshold trigger a warning message
    const TOO_BIG_THRESHOLD = 1000000;

    // List of all datapoints of item_amount_base schema id
    const amountBaseColumnDatapoints = findBySchemaId(
      content,
      'item_amount_base',
    );

    messages = [];
    for (var i = 0; i < amountBaseColumnDatapoints.length; i++) {

      // Use normalized_value for comparing values of Date and Number fields (https://api.elis.rossum.ai/docs/#content-object)
      if (amountBaseColumnDatapoints[i].content.normalized_value >= TOO_BIG_THRESHOLD) {
        messages.push(
          createMessage(
            'warning',
            'Value is too big',
            amountBaseColumnDatapoints[i].id,
          ),
        );
      }
    }

    // There should be only one datapoint of invoice_id schema id
    const [invoiceIdDatapoint] = findBySchemaId(content, 'invoice_id');

    // "Replace" operation is returned to update the invoice_id value
    const operations = [
      createReplaceOperation(
        invoiceIdDatapoint,
        invoiceIdDatapoint.content.value.replace(/-/g, ''),
      ),
    ];

    // Return messages and operations to be used to update current annotation data
    return {
      messages,
      operations,
    };
  } catch (e) {
    // In case of exception, create and return error message. This may be useful for debugging.
    const messages = [
      createMessage('error', 'Serverless Function: ' + e.message)
    ];
    return {
      messages,
    };
  }
};

// --- HELPER FUNCTIONS ---

// Return datapoints matching a schema id.
// @param {Object} content - the annotation content tree (see https://api.elis.rossum.ai/docs/#annotation-data)
// @param {string} schemaId - the field's ID as defined in the extraction schema(see https://api.elis.rossum.ai/docs/#document-schema)
// @returns {Array} - the list of datapoints matching the schema ID

const findBySchemaId = (content, schemaId) =>
  content.reduce(
    (results, dp) =>
    dp.schema_id === schemaId ? [...results, dp] :
    dp.children ? [...results, ...findBySchemaId(dp.children, schemaId)] :
    results,
    [],
  );

// Create a message which will be shown to the user
// @param {number} datapointId - the id of the datapoint where the message will appear (null for "global" messages).
// @param {String} messageType - the type of the message, any of {info|warning|error}. Errors prevent confirmation in the UI.
// @param {String} messageContent - the message shown to the user
// @returns {Object} - the JSON message definition (see https://api.elis.rossum.ai/docs/#annotation-content-event-response-format)

const createMessage = (type, content, datapointId = null) => ({
  content: content,
  type: type,
  id: datapointId,
});

// Replace the value of the datapoint with a new value.
// @param {Object} datapoint - the content of the datapoint
// @param {string} - the new value of the datapoint
// @return {Object} - the JSON replace operation definition (see https://api.elis.rossum.ai/docs/#annotation-content-event-response-format)

const createReplaceOperation = (datapoint, newValue) => ({
  op: 'replace',
  id: datapoint.id,
  value: {
    content: {
      value: newValue,
    },
  },
});

To implement a serverless function, create a hook object of type function. Use code object config attribute to specify a serialized source code.

Connector Extension

The connector component is aimed at two main use-cases: applying custom business rules during data validation, and direct integration of Rossum with downstream systems.

The connector component receives two types of callbacks - an on-the-fly validation callback on every update of captured data, and an on-export save callback when the document capture is finalized.

The custom business rules take use chiefly of the on-the-fly validation callback. The connector can auto-validate and transform both the initial AI-based extractions and each user operator edit within the validation screen; based on the input, it can push user-visible messages and value updates back to Rossum. This allows for both simple tweaks (like verifying that two amounts sum together or transforming decimal points to thousand separators) and complex functionality like intelligent PO match.

The integration with downstream systems on the other hand relies mainly on the save callback. At the same moment a document is exported from Rossum, it can be imported to a downstream system. Since there are typically constraints on the captured data, these constraints can be enforced even within the validation callback.

Implement a connector

Connectors are designed to be implemented using a push-model using common HTTPS protocol. When annotation data is changed, or when data export is triggered, specific connector endpoint is called with annotation data as a request payload. The connector must be deployed with a public IP address so that the Rossum platform can call its endpoints; for testing, a middleware like ngrok or serveo may come useful.

Example of a valid no-op (empty) validate response

{"messages": [], "updated_datapoints": []}

Example of a valid no-op (empty) save response

{}

The connector API consists of two endpoints, validate and save, described below. A connector must always implement both endpoints (though they may not necessarily perform a function in a particular connector - see the right column for an empty reply example), the platform raises an error if it is not able to run a endpoint.

Setup a connector

The next step after implementing the first version of a connector is configuring it in the Rossum platform.

In Rossum, a connector object defines service_url and params for construction of HTTPS requests and authorization_token that is passed in every request to authenticate the caller as the actual Rossum server. It may also uniquely identify the organization when multiple Rossum organizations share the same connector server.

To set-up a connector for a queue, create a connector object using either our API or the rossum tool – follow these instructions. A connector object may be associated with one or more queues. One queue can only have one connector object associated with it.

Connector API

Example data sent to connector (validate, save)

{
  "meta": {
    "document_url": "https://api.elis.rossum.ai/v1/documents/6780",
    "arrived_at": "2019-01-30T07:55:13.208304Z",
    "original_file": "https://api.elis.rossum.ai/v1/original/bf0db41937df8525aa7f3f9b18a562f3",
    "original_filename": "Invoice.pdf",
    "queue_name": "Invoices",
    "workspace_name": "EU",
    "organization_name": "East West Trading Co",
    "annotation": "https://api.elis.rossum.ai/v1/annotations/4710",
    "queue": "https://api.elis.rossum.ai/v1/queues/63",
    "workspace": "https://api.elis.rossum.ai/v1/workspaces/62",
    "organization": "https://api.elis.rossum.ai/v1/organizations/1",
    "modifier": "https://api.elis.rossum.ai/v1/users/27",
    "updated_datapoint_ids": ["197468"],
    "modifier_metadata": {},
    "queue_metadata": {},
    "annotation_metadata": {},
    "rir_poll_id": "54f6b9ecfa751789f71ddf12",
    "automated": false
  },
  "content": [
    {
      "id": "197466",
      "category": "section",
      "schema_id": "invoice_info_section",
      "children": [
        {
          "id": "197467",
          "category": "datapoint",
          "schema_id": "invoice_number",
          "page": 1,
          "position": [916, 168, 1190, 222],
          "rir_position": [916, 168, 1190, 222],
          "rir_confidence": 0.97657,
          "value": "FV103828806S",
          "validation_sources": ["score"],
          "type": "string"
        },
        {
          "id": "197468",
          "category": "datapoint",
          "schema_id": "date_due",
          "page": 1,
          "position": [938, 618, 1000, 654],
          "rir_position": [940, 618, 1020, 655],
          "rir_confidence": 0.98279,
          "value": "12/22/2018",
          "validation_sources": ["score"],
          "type": "date"
        },
        {
          "id": "197469",
          "category": "datapoint",
          "schema_id": "amount_due",
          "page": 1,
          "position": [1134, 1050, 1190, 1080],
          "rir_position": [1134, 1050, 1190, 1080],
          "rir_confidence": 0.74237,
          "value": "55.20",
          "validation_sources": ["human"],
          "type": "number"
        }
      ]
    },
    {
      "id": "197500",
      "category": "section",
      "schema_id": "line_items_section",
      "children": [
        {
          "id": "197501",
          "category": "multivalue",
          "schema_id": "line_items",
          "children": [
            {
              "id": "198139",
              "category": "tuple",
              "schema_id": "line_item",
              "children": [
                {
                  "id": "198140",
                  "category": "datapoint",
                  "schema_id": "item_desc",
                  "page": 1,
                  "position": [173, 883, 395, 904],
                  "rir_position": null,
                  "rir_confidence": null,
                  "value": "Red Rose",
                  "validation_sources": [],
                  "type": "string"
                },
                {
                  "id": "198142",
                  "category": "datapoint",
                  "schema_id": "item_net_unit_price",
                  "page": 1,
                  "position": [714, 846, 768, 870],
                  "rir_position": null,
                  "rir_confidence": null,
                  "value": "1532.02",
                  "validation_sources": ["human"],
                  "type": "number"
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

All connector endpoints, representing particular points in the document lifetime, are simple verbs that receive a JSON POSTed and potentially expect a JSON returned in turn.

The authorization type and authorization token is passed as an Authorization HTTP header. Authorization type may be secret_key (shared secret) or Basic for HTTP basic authentication.

Please note that for Basic authentication, authorization_token is passed as-is, therefore it must be set to a correct base64 encoded value. For example username connector and password secure123 is encoded as Y29ubmVjdG9yOnNlY3VyZTEyMw== authorization token.

Connector requests may be autenticated using a client SSL certificate, see Connector API for reference.

Errors

If a connector does not implement an endpoint, it may return HTTP status 404. An endpoint may fail, returning either HTTP 4xx or HTTP 5xx; for some endpoints (like validate and save), this may trigger a user interface message; either the error key of a JSON response is used, or the response body itself in case it is not JSON. The connector endpoint save can be called in asynchronous (default) as well as synchronous mode (useful for embedded mode).

Data format

The received JSON object contains two keys, meta carrying the metadata and content carrying endpoint-specific content.

The metadata identify the concerned document, containing attributes:

A common class of content is the annotation tree, which is a JSON object that can contain nested datapoint objects, and matches the schema datapoint tree.

Intermediate nodes have the following structure:

Datapoint (leaf) nodes structure contains actual data:

Annotation lifecycle with a connector

If an asynchronous connector is deployed to a queue, an annotation status will change from reviewing to exporting and subsequently to exported or failed_export. If no connector extension is deployed to a queue or if the attribute asynchronous is set to false, an annotation status will change from reviewing to exported (or failed_export) directly.

Endpoint: validate

This endpoint is called after the document processing has finished, when operator opens a document in the Rossum verification interface and then every time after operator updates a field. After the processing is finished, the initial validate call is marked with initial=true URL parameter. For the other calls, only /validate without the parameter is called.

The request path is fixed to /validate and cannot be changed.

It may:

• validate the given annotation tree and return a list of messages commenting on it (e.g. pointing out errors or showing matched suppliers).
• update the annotation tree by returning a list of replace, add and remove operations

Both the messages and the updated data are shown in the verification interface. Moreover, the messages may block confirmation in the case of errors.

This endpoint should be fast as it is part of an interactive workflow.

Receives an annotation tree as content.

Returns a JSON object with the lists: messages, operations and updated_datapoints.

Keys messages, operations (optional)

The description of these keys was moved to the Hook Extension. See the description here.

Key updated_datapoints (optional, deprecated)

We also support a simplified version of updates using updated_datapoints response key. It only supports updates (no add or remove operations) and is now deprecated. The updated datapoint object contains attributes:

Validate endpoint should always return 200 OK status.

An error message returned from the connector prevents user from confirming the document.

Endpoint: save

This endpoint is called when the invoice transitions to the exported state. Connector may process the final document annotation and save it to the target system. It receives an annotation tree as content. The request path is fixed to /save and cannot be changed.

The save endpoint is called asynchronously (unless synchronous mode is set in related connector object. Timeout of the save endpoint is 60 seconds.

For successful export, the request should have 2xx status.

Example of successful save response without messages in UI

HTTP/1.1 204 No Content

HTTP/1.1 200 OK
Content-Type: text/plain

this response body is ignored

HTTP/1.1 200 OK
Content-Type: application/json

{
  "messages": []
}

When messages are expected to be displayed in the UI, they should be sent in the same format as in validate endpoint.

Example of successful save response with messages in UI

HTTP/1.1 200 OK
Content-Type: application/json

{
  "messages": [
    {
      "content": "Everything is OK.",
      "id": null,
      "type": "info"
    }
  ]
}

If the endpoint fails with an HTTP error and/or message of type error is received, the document transitions to the failed_export state - it is then available to the operators for manual review and re-queuing to the to_review state in the user interface. Re-queuing may be done also programmatically via the API using a PATCH call to set to_review annotation status. Patching annotation status to exporting state triggers an export retry.

Example of unsuccessful save response with messages in UI

HTTP/1.1 422 Unprocessable Entity
Content-Type: application/json

{
  "messages": [
    {
      "content": "Even though this message is info, the export will fail due to the status code.",
      "id": null,
      "type": "info"
    }
  ]
}

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain

An errror message "Export failed." will show up in the UI

HTTP/1.1 200 OK
Content-Type: application/json

{
  "messages": [
    {
      "content": "Proper status code could not be set.",
      "id": null,
      "type": "error"
    }
  ]
}

Custom UI Extension

Sometimes users might want to extend the behavior of UI validation view with something special. That should be the goal of custom UI extensions.

Buttons

Currently, there are two different ways of using a custom button:

1. Popup Button - opens a specific URL in the web browser
2. Validate Button - triggers a standard validate call to connector

If you would like to read more about how to create a button, see the Button schema.

Popup Button

Popup Button opens a website completely managed by the user in a separate tab. It runs in parallel to the validation interface session in the app. Such website can be used for any interface that will assist operators in the reviewing process.

Example Use Cases of Popup Button:

1. opening an email linked to the annotated document
2. creating a new item in external database according to extracted data

Communication with the Validation Interface

You can communicate with the validation interface directly using standard browser API of window.postMessage. You will need to use window.addEventListeners in order to receive messages from the validation interface:

The shape of the result key is the same as the top level content attribute of the annotation data response.

Once the listener is in place, you can post one of supported message types:

• GET_DATAPOINTS - returns the same tree structure you’d get by requesting annotation data

Now you can manage your own business logic. To apply the results of the logic to datapoints in Rossum, you can use the following message type:

• UPDATE_DATAPOINT - sends updated value to a Rossum datapoint. Only one datapoint value can be updated at a time.

• FINISH - informs the Rossum app that the popup process is ready to be closed. After this message is posted, popup will be closed and Rossum app will trigger a validate call.

Providing message type to postMessage lets Rossum interface know what operation user requests and determines the type of the answer which could be used to match appropriate response.

Validate button

If popup_url key is missing in button's schema, clicking the button will trigger a standard validate call to connector. In such call, updated_datapoint_ids will contain the ID of the pressed button.

Note: if you’re missing some annotation data that you’d like to receive in a similar way, do contact our support team. We’re collecting feedback to further expand this list.

Automation

All imported documents are processed by the data extraction process to obtain values of fields specified in the schema. Extracted values are then available for validation in the UI.

Using per-queue automation settings, it is possible to skip manual UI validation step and automatically switch document to confirmed state or proceed with the export of the document. Decision to export document or switch it to confirmed state is based on Queue settings.

Currently, there are three levels of automation:

• No automation: User has to review all documents in the UI to validate extracted data (default).

• Confidence automation: User only reviews documents with low data extraction confidence ("tick" icon is not displayed for one or more fields) or validation errors. By default, we automate documents that duplicates and do not automate documents that edits (split) is proposed. You can change this in per-queue automation settings

• Full automation: All documents with no validation errors are exported or switched to confirmed state only if they do not contain a suggested edit (split). You can change this in per-queue automation settings
  
  An error triggered by a schema field constraint or connector validation blocks auto-export even in full-automation level. In such case, non-required fields with validation errors are cleared and validation is performed again. In case the error persists, the document must be reviewed manually, otherwise it is exported or switched to confirmed state.

Read more about the Automation framework on our developer hub.

Sources of field validation

Low-confidence fields are marked in the UI by an "eye" icon, we consider them to be not validated. On the API level they have an empty validation_sources list.

Validation of a field may be introduced by various sources: data extraction confidence above a threshold, computation of various checksums (e.g. VAT rate, net amount and gross amount) or a human review. These validations are recorded in the validation_source list. The data extraction confidence threshold may be adjusted, see validation sources for details.

AI Confidence Scores

While there are multiple ways to automatically pre-validate fields, the most prominent one is score-based validation based on AI Core Engine confidence scores.

The confidence score predicted for each AI-extractd field is stored in the rir_confidence attribute. The score is a number between 0 and 1, and is calibrated in such a way that it corresponds to the probability of a given value to be correct. In other words, a field with score 0.80 is expected to be correct 4 out of 5 times.

The value of the score_threshold (can be set on queue, or individually per datapoint in schema; default is 0.975) attribute represents the minimum score that triggers automatic validation. Because of the score meaning, this directly corresponds to the achieved accuracy. For example, if a score threshold for validation is set at 0.975, that gives an expected error rate of 2.5% for that field.

Autopilot

Autopilot is a automatic process removing "eye" icon from fields. This process is based on past occurrence of field value on documents which has been already processed in the same queue.

Read more about this Automation component on our developer hub.

Autopilot configuration

Default Autopilot configuration

{
  "autopilot": {
    "enabled": true,

    "search_history":{
        "rir_field_names": ["sender_ic", "sender_dic", "account_num", "iban", "sender_name"],
        "matching_fields_threshold": 2
    },
    "automate_fields":{
        "rir_field_names": [
            "account_num",
            "bank_num",
            "iban",
            "bic",
            "sender_dic",
            "sender_ic",
            "recipient_dic",
            "recipient_ic",
            "const_sym"
        ],
        "field_repeated_min": 3
    }
  }
}

Autopilot configuration can be modified in Queue.settings where you can set rules for each queue. If Autopilot is not explicitly disabled by switch enabled set to false, Autopilot is enabled.

Configuration is divided into two sections:

History search

This section configures process of finding documents from the same sender as the document which is currently being processed. Annotation is considered from the same sender if it contains fields with same rir_field_name and value as the current document.

When at least two fields listed in rir_field_names match values of the current document, document is is considered to have same sender

{
    "search_history":{
        "rir_field_names": ["sender_ic", "sender_dic", "account_num"],
        "matching_fields_threshold": 2
    }
}

Automate fields

This section describes rules which will be applied on annotations found in previous step History search. Field will have "eye" icon removed, if we have found at least field_repeated_min fields with same rir_field_name and value on documents found in step History search.

If any config section is missing, default value which you can see on the right side is applied.

Embedded Mode

In some use-cases, it is desirable to use only the per-annotation validation view of the Rossum application. Rossum may be integrated with other systems using so-called embedded mode.

In embedded mode, special URL is constructed and then used in iframe or popup browser window to show Rossum annotation view. Some view navigation widgets are hidden (such as home, postpone and delete buttons), so that user is only allowed to update and confirm all field values.

Embedded mode can be used to view annotations only in status to_review, reviewing, postponed, or confirmed.

Embedded mode workflow

The host application first uploads a document using standard Rossum API. During this process, an annotation object is created. It is possible to obtain a status of the annotation object and wait for the status to became to_review (ready for checking) using annotation endpoint.

As soon as importing of the annotation object has finished, an authenticated user may call start_embedded endpoint to obtain a URL that is to be included in iframe or popup browser window of the host application. Parameters of the call are return_url and cancel_url that are used to redirect to in a browser when user finishes the annotation.

The URL contains security token that is used by embedded Rossum application to access Rossum API. When the checking of the document has finished, user clicks on done button and host application is notified about finished annotation through save endpoint of the connector HTTP API. By default, this call is made asynchronously, which causes a lag (up to a few seconds) between the click on done button and the call to save endpoint. However, it is possible to switch the calls to synchronous mode by switching the connector asynchronous toggle to false (see connector for reference).

API Reference

For introduction to the Rossum API, see Overview

Most of the API endpoints require user to be authenticated, see Authentication for details.

Organization

Example organization object

{
  "id": 406,
  "url": "https://api.elis.rossum.ai/v1/organizations/406",
  "name": "East West Trading Co",
  "workspaces": [
    "https://api.elis.rossum.ai/v1/workspaces/7540"
  ],
  "users": [
    "https://api.elis.rossum.ai/v1/users/10775"
  ],
  "organization_group": "https://api.elis.rossum.ai/v1/organization_groups/17",
  "ui_settings": {},
  "metadata": {},
  "trial_expires_at": "2020-09-02T14:28:11.000000Z",
  "is_trial": true,
  "oidc_provider": "some_oidc_provider"
}

Organization is a basic unit that contains all objects that are required to fully use Rossum platform.

Create new organization

curl -s -X POST -H 'Content-Type: application/json' \
  -d '{"template_name": "UK Demo Template", "organization_name": "East West Trading Co", "user_fullname": "John Doe", "user_email": "john@east-west-trading.com", "user_password": "owo1aiG9ua9Aihai", "user_ui_settings": { "locale": "en" }, "create_key": "13156106d6f185df24648ac7ff20f64f1c5c06c144927be217189e26f8262c4a"}' \
  'https://api.elis.rossum.ai/v1/organizations/create'

{
  "organization": {
    "id": 105,
    "url": "https://api.elis.rossum.ai/v1/organizations/105",
    "name": "East West Trading Co",
    "workspaces": [
      "https://api.elis.rossum.ai/v1/workspaces/160"
    ],
    "users": [
      "https://api.elis.rossum.ai/v1/users/173"
    ],
    "organization_group": "https://api.elis.rossum.ai/v1/organization_groups/17",
    "ui_settings": {},
    "metadata": {},
    "trial_expires_at": "2020-09-02T14:28:11.000000Z",
    "is_trial": true,
    "oidc_provider": null
  }
}

POST /v1/organizations/create

Create new organization and related objects (workspace, queue, user, schema, inbox).

You need a create_key in order to create an organization. Please contact support@rossum.ai to obtain one.

Selected template_name affects default schema and extracted fields. Please note that the demo templates may be updated as new features are introduced.

Available templates:

Response

Status: 200

Returns object with organization key and organization object value.

Billing for organization

In order to obtain an overview of the billed items, you can get basic billing statistics.

Download billing report (February 1, 2019 - February 5, 2019).

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' -H 'Content-Type: application/json' \
  'https://api.elis.rossum.ai/v1/organizations/789/billing' -d '{"filter": {"begin_date": "2019-02-01", "end_date": "2020-02-05"}}'

POST /v1/organizations/{id}/billing

{
  "filter": {
    "queues": [
      "https://api.elis.rossum.ai/v1/queues/8199"
    ],
    "begin_date": "2019-02-01",
    "end_date": "2019-02-03"
  },
  "group_by": [
    "queue",
  ]
}