Getting Started

Introduction

The Rossum API allows you to programmatically access and manage your organization's Rossum data and account information. The API allows you to do the following programmatically:

• Manage your organization, users, workspaces and queues
• Configure captured data: select extracted fields
• Integrate Rossum with other systems: import, export, extensions

On this page, you will find an introduction to the API usage from a developer perspective, and a reference to all the API objects and methods.

Developer Resources

There are several other key resources related to implementing, integrating and extending the Rossum platform:

• Refer to the Rossum Developer Portal for guides, tutorials, news and a community Q&A section.
• Subscribe to the rossum-api-announcements group to stay up to date on the API and platform updates.
• For managing and configuring your account, use the rossum command-line tool. (Setup instructions.)
• For a management overview of Rossum implementation options, see the Rossum Integration Whitepaper.
• If you are an RPA developer, refer to our UiPath or BluePrism guides.

Quick API Tutorial

For a quick tutorial on how to authenticate, upload a document and export extracted data, see the sections below. If you want to skip this quick tutorial, continue directly to the Overview section.

It is a good idea to go through the introduction to the Rossum platform on the Developer Portal first to make sure you are up to speed on the basic Rossum concepts.

If in trouble, feel free to contact us at support@rossum.ai.

Install curl tool

Test curl is installed properly

curl https://api.elis.rossum.ai/v1

{"organizations":"https://api.elis.rossum.ai/v1/organizations","workspaces":"htt
ps://api.elis.rossum.ai/v1/workspaces","schemas":"https://api.elis.rossum.ai/v1/
schemas","connectors":"https://api.elis.rossum.ai/v1/connectors","inboxes":"http
s://api.elis.rossum.ai/v1/inboxes","queues":"https://api.elis.rossum.ai/v1/queue
s","documents":"https://api.elis.rossum.ai/v1/documents","users":"https://api.el
is.rossum.ai/v1/users","groups":"https://api.elis.rossum.ai/v1/groups","annotati
ons":"https://api.elis.rossum.ai/v1/annotations","pages":"https://api.elis.rossu
m.ai/v1/pages"}

All code samples included in this API documentation use curl, the command line data transfer tool. On MS Windows 10, MacOS X and most Linux distributions, curl should already be pre-installed. If not, please download it from curl.haxx.se).

Optionally use jq tool to pretty-print JSON output

curl https://api.elis.rossum.ai/v1 | jq

{
  "organizations": "https://api.elis.rossum.ai/v1/organizations",
  "workspaces": "https://api.elis.rossum.ai/v1/workspaces",
  "schemas": "https://api.elis.rossum.ai/v1/schemas",
  "connectors": "https://api.elis.rossum.ai/v1/connectors",
  "inboxes": "https://api.elis.rossum.ai/v1/inboxes",
  "queues": "https://api.elis.rossum.ai/v1/queues",
  "documents": "https://api.elis.rossum.ai/v1/documents",
  "users": "https://api.elis.rossum.ai/v1/users",
  "groups": "https://api.elis.rossum.ai/v1/groups",
  "annotations": "https://api.elis.rossum.ai/v1/annotations",
  "pages": "https://api.elis.rossum.ai/v1/pages"
}

You may also want to install jq tool to make curl output human-readable.

Use the API on Windows

This API documentation is written for usage in command line interpreters running on UNIX based operation systems (Linux and Mac). Windows users may need to use the following substitutions when working with API:

Example of API call on UNIX-based OS

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' -H 'Content-Type: application/json' \
  -d '{"target_queue": "https://api.elis.rossum.ai/v1/queues/8236", "target_status": "to_review"}' \
  'https://api.elis.rossum.ai/v1/annotations/315777/copy'

Examples of API call on Windows

curl -H "Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03" -H "Content-Type: application/json" ^
  -d "{""target_queue"": ""https://api.elis.rossum.ai/v1/queues/8236"", ""target_status"": ""to_review""}" ^
  "https://api.elis.rossum.ai/v1/annotations/315777/copy"


curl -H "Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03" -H "Content-Type: application/json" ^
  -d "{\"target_queue\": \"https://api.elis.rossum.ai/v1/queues/8236\", \"target_status\": \"to_review\"}" ^
  "https://api.elis.rossum.ai/v1/annotations/315777/copy"

Create an account

In order to interact with the API, you need an account. If you do not have one, you can create one via our self-service portal.

Login to the account

Fill-in your username and password (login credentials to work with API are the same as those to log into your account.). Trigger login endpoint to obtain a key (token), that can be used in subsequent calls.

curl -s -H 'Content-Type: application/json' \
  -d '{"username": "east-west-trading-co@elis.rossum.ai", "password": "aCo2ohghBo8Oghai"}' \
  'https://api.elis.rossum.ai/v1/auth/login'
{"key": "db313f24f5738c8e04635e036ec8a45cdd6d6b03"}

This key will be valid for a default expire time (currently 162 hours) or until you log out from the sessions.

Upload a document

In order to upload a document (PDF or image) through the API, you need to obtain the id of a queue first.

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03'
  'https://api.elis.rossum.ai/v1/queues?page_size=1' | jq -r .results[0].url
https://api.elis.rossum.ai/v1/queues/8199

Then you can upload document to the queue. Alternatively, you can send documents to a queue-related inbox. See upload for more information about importing files.

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  -F content=@document.pdf 'https://api.elis.rossum.ai/v1/queues/8199/upload' | jq -r .results[0].annotation
https://api.elis.rossum.ai/v1/annotations/319668

Wait for document to be ready and review extracted data

As soon as a document is uploaded, it will show up in the queue and the data extraction will begin. It may take a few seconds to several minutes to process a document. You can check status of the annotation and wait until its status is changed to to_review.

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/annotations/319668' | jq .status
"to_review"

After that, you can open the Rossum web interface elis.rossum.ai to review and confirm extracted data.

Download reviewed data

Now you can export extracted data using the export endpoint of the queue. You can select XML, CSV, XLSX or JSON format. For CSV, use URL like:

curl -s -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/queues/8199/export?status=exported&format=csv&id=319668'
Invoice number,Invoice Date,PO Number,Due date,Vendor name,Vendor ID,Customer name,Customer ID,Total amount,
2183760194,2018-06-08,PO2231233,2018-06-08,Alza.cz a.s.,02231233,Rossum,05222322,500.00

Logout

Finally you can dispose token safely using logout endpoint:

curl -s -X POST -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/auth/logout'
{"detail":"Successfully logged out."}

Overview

HTTP and REST

The Rossum API is organized around REST. Our API has predictable, resource-oriented URLs, and uses HTTP response codes to indicate API errors. We use built-in HTTP features, like HTTP authentication and HTTP verbs, which are understood by off-the-shelf HTTP clients.

HTTP Verbs

Call the API using the following standard HTTP methods:

• GET to retrieve an object or multiple objects in a specific category
• POST to create an object
• PUT to modify entire object
• PATCH to modify fields of the object
• DELETE to delete an object

We support cross-origin resource sharing, allowing you to interact securely with our API from a client-side web application. JSON is returned by API responses, including errors (except when another format is requested, e.g. XML).

Base URL

Base API endpoint URL depends on the deployment type and location, e.g. https://api.elis.rossum.ai. If you are not sure about correct URL, please contact us at support@rossum.ai.

Please note that Rossum frontend may use different API endpoints, e.g. https://elis.rossum.ai/api/, but these are not meant for third-party integrations and should not be used without consent from Rossum.

Authentication

Most of the API endpoints require user to be authenticated. To login to the Rossum API, post an object with username and password fields. Login returns new authentication key to be used in token authentication.

User may delete a token using logout endpoint or automatically after a configured time (default expire time is 162 hours). Default expire time can be lowered using max_token_lifetime_s field. When token expires, 401 status is returned. Users are expected to re-login to obtain a new token.

Login

Login user using username and password

curl -H 'Content-Type: application/json' \
  -d '{"username": "east-west-trading-co@elis.rossum.ai", "password": "aCo2ohghBo8Oghai"}' \
  'https://api.elis.rossum.ai/v1/auth/login'

{
  "key": "db313f24f5738c8e04635e036ec8a45cdd6d6b03"
}

POST /v1/auth/login

Use token key in requests

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/organizations/406'

Login user expiring after 1 hour

curl -H 'Content-Type: application/json' \
  -d '{"username": "east-west-trading-co@elis.rossum.ai", "password": "aCo2ohghBo8Oghai", "max_token_lifetime_s": 3600}' \
  'https://api.elis.rossum.ai/v1/auth/login'

{
  "key": "ltcg2p2w7o9vxju313f04rq7lcc4xu2bwso423b3"
}

Response

Status: 200

Returns object with "key", which is an actual authentication token.

Logout

Logout user

curl -X POST -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/auth/logout'

{
  "detail": "Successfully logged out."
}

POST /v1/auth/logout

Logout user, discard auth token.

Response

Status: 200

Single Sign-On (SSO)

Rossum allows customers to integrate with their own identity provider, such as Google, Azure AD or any other provider using OAuth2 OpenID Connect protocol (OIDC). Rossum then acts as a service provider.

When SSO is enabled for an organization, user is redirected to a configured identity provider login page and only allowed to access Rossum application when successfully authenticated. Identity provider user claim (e.g. email (default), sub, preferred_username, unique_name) is used to match a user account in Rossum. Please note that Rossum user accounts still must be created for every user before logging to Rossum.

Required setup of the OIDC identity provider:

• Redirect URI (also known as Reply URL): https://elis.rossum.ai/api/oauth/code

Required information to allow OIDC setup for the Rossum service provider:

• OIDC endpoint, such as https://accounts.google.com. It should support openid configuration, e.g. https://accounts.google.com/.well-known/openid-configuration
• client id
• client secret
• claim that should be matched in Rossum
• Rossum organization id

If you need to setup SSO for your organization, please contact support@rossum.ai.

Pagination

All object list operations are paged by default, so you may need several API calls to obtain all objects of given type.

(*) Maximum page size of annotation list is 1000 for CSV export, 100 for other formats.

Filters and ordering

List queues of workspace 7540, with locale en_US and order results by name.

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/v1/queues?workspace=7540&locale=en_US&ordering=name'

Lists may be filtered using various attributes. Multiple attributes are combined with & in the URL, which results in more specific response. Please refer to the particular object description.

All filter parameters that refers to object expect only id (instead of URL) of the object.

Ordering of results may be enforced by the ordering parameter and one or more keys delimited by a comma. Preceding key with a minus sign - enforces descending order.

Metadata

Example metadata in a document object

{
  "id": 319768,
  "url": "https://api.elis.rossum.ai/v1/documents/319768",
  "s3_name": "05feca6b90d13e389c31c8fdeb7fea26",
  "annotations": [
    "https://api.elis.rossum.ai/v1/annotations/319668"
  ],
  "mime_type": "application/pdf",
  "arrived_at": "2019-02-11T19:22:33.993427Z",
  "original_file_name": "document.pdf",
  "content": "https://api.elis.rossum.ai/v1/documents/319768/content",
  "metadata": {
    "customer-id": "f205ec8a-5597-4dbb-8d66-5a53ea96cdea",
    "source": 9581,
    "authors": ["Joe Smith", "Peter Doe"]
  }
}

When working with API objects, it may be useful to attach some information to the object (e.g. customer id to a document). You can store custom JSON object in a metadata section available in most objects.

List of objects with metadata support: organization, workspace, user, queue, schema, connector, inbox, document, annotation, page.

Total metadata size may be up to 1 kB per object.

Versioning

API Version is part of the URL, e.g. https://api.elis.rossum.ai/v1/users.

To allow API progress, we consider addition of a field in a JSON object to be a backward-compatible operation that may be introduced at any time. Clients are expected to deal with such changes.

In order to be notified about future API changes and improvements, please subscribe to the rossum-api-announcements group.

Dates

All dates fields are represented as ISO 8601 formatted strings, e.g. 2018-06-01T21:36:42.223415Z. All returned dates are in UTC timezone.

Errors

Our API uses conventional HTTP response codes to indicate the success or failure of an API request.

Import and Export

Documents may be imported into Rossum using the REST API and email gateway. Supported file formats are PDF, PNG, JPEG and TIFF.`

In order to get the best results from Rossum the documents should be in A4 format of at least 150 DPI (in case of scans/photos). Read more about import recommendations.

Upload

You can upload a document to the queue using upload endpoint with one or more files to be uploaded. You can also specify additional field values in upload endpoint, e.g. your internal document id. As soon as a document is uploaded, data extraction is started.

Upload endpoint supports basic authentication to enable easy integration with third-party systems.

Import by Email

It is also possible to send documents by email using a properly configured inbox that is associated with a queue. Users then only need to know the email address to forward emails to.

For every incoming email, Rossum extracts PDF documents and images, stores them in the queue and starts data extraction process.

Small images (up to 100x100 pixels) are ignored, see inbox for reference.

You can use selected email header data (e.g. Subject) to initialize additional field values, see rir_field_names attribute description for details.

Export

In order to export extracted and confirmed data you can call export endpoint. You can specify status, time-range filters and annotation id list to limit returned results.

Export endpoint supports basic authentication to enable easy integration with third-party systems.

Auto-split of document

It is possible to process a single PDF file that contains several invoices. Just insert a special separator page between the documents. You can print this page and insert it between documents while scanning.

Rossum will recognize a QR code on the page and split the PDF into individual documents automatically. Produced documents are imported to the queue, while the original document is set to a split state.

Document Schema

Every queue has an associated schema that specifies which fields will be extracted from documents as well as the structure of the data sent to connector and exported from the platform.

Rossum schema supports data fields with single values (datapoint), fields with multiple values (multivalue) or tuples of fields (tuple). At the topmost level, each schema consists of sections, which may either directly contain actual data fields (datapoints) or use nested multivalues and tuples as containers for single datapoints.

But while schema may theoretically consist of an arbitrary number of nested containers, the Rossum UI supports only certain particular combinations of datapoint types. The supported shapes are:

• simple: atomic datapoints of type number, string, date or enum
• list: simple datapoint within a multivalue
• tabular: simple datapoint within a "multivalue tuple" (a multivalue list containing a tuple for every row)

Schema content

Schema content consists of a list of section objects.

Common attributes

The following attributes are common for all schema objects:

Section

Example of a section

{
    "category": "section",
    "id": "amounts_section",
    "label": "Amounts",
    "children": [...],
    "icon": ""
}

Section represents a logical part of the document, such as amounts or vendor info. It is allowed only at the top level. Schema allows multiple sections, and there should be at least one section in the schema.

Datapoint

A datapoint represents a single value, typically a field of a document or some global document information. Fields common to all datapoint types:

rir_field_names attribute allows to specify source of initial value of the object. List items may be:

• one of extracted field types to use the AI engine prediction value
• upload:id to identify a value specified while uploading the document
• email_header:<id> to use a value extracted from email headers. Supported email headers: from, to, subject, message-id, date.
• email_body:<id> to select email body. Supported values are text_html for HTML body or text_plain for plain text body.
• email:<id> to identify a value specified in email.received hook response

If more list items in rir_field_names are specified, the first available value will be used.

String type

Example string datapoint

{
    "category": "datapoint",
    "id": "document_id",
    "label": "Invoice ID",
    "type": "string",
    "default_value": null,
    "rir_field_names": ["document_id"],
    "constraints": {
        "length": {
            "exact": null,
            "max": 16,
            "min": null
        },
        "regexp": {
            "pattern": "^INV[0-9]+$"
        },
        "required": false
    }
}

String datapoint does not have any special attribute.

Date type

Example date datapoint

{
  "id": "item_delivered",
  "type": "date",
  "label": "Item Delivered",
  "format": "MM/DD/YYYY",
  "category": "datapoint"
}

Attributes specific to Date datapoint:

Date format supported: available tokens

Example date formats:

• D/M/YYYY: e.g. 23/1/2019
• MM/DD/YYYY: e.g. 01/23/2019
• YYYY-MM-DD: e.g. 2019-01-23 (ISO date format)

Number type

Example number datapoint

{
  "id": "item_quantity",
  "type": "number",
  "label": "Quantity",
  "format": "#,##0.#",
  "category": "datapoint"
}

Attributes specific to Number datapoint:

The following table shows numeric formats with their examples.

Aggregations

Example aggregations

{
  "id": "quantity",
  "type": "number",
  "label": "Quantity",
  "category": "datapoint",
  "aggregations": {
    "sum": {
      "label": "Total"
    }
  },
  "default_value": null,
  "rir_field_names": []
}

Aggregations allow computation of some informative values, e.g. a sum of a table column with numeric values. These are returned among messages when /annotations/{id}/validate endpoint is called. Aggregations can be computed only for tables (multivalues of tuples).

All aggregation objects can have an attribute label that will be shown in the UI.

Enum type

Example enum datapoint with options

{
  "id": "document_type",
  "type": "enum",
  "label": "Document type",
  "hidden": false,
  "category": "datapoint",
  "options": [
    {
      "label": "Invoice Received",
      "value": "21"
    },
    {
      "label": "Invoice Sent",
      "value": "22"
    },
    {
      "label": "Receipt",
      "value": "23"
    }
  ],
  "default_value": "21",
  "rir_field_names": []
}

Attributes specific to Enum datapoint:

Every option consists of an object with keys:

Enum datapoint value is matched in a case insensitive mode, e.g. EUR currency value returned by the AI Core Engine is matched successfully against {"value": "eur", "label": "Euro"} option.

Button type

Specifies a button shown in Rossum UI. For more details please refer to custom UI extension.

Example button datapoint

{
  "id": "show_email",
  "type": "button",
  "category": "datapoint",
  "popup_url": "http://example.com/show_customer_data",
}

Despite being a datapoint object, button currently cannot hold any value. Therefore, the set of available Button datapoint attributes is limited to:

Value constraints

Example value constraints

{
  "id": "document_id",
  "type": "string",
  "label": "Invoice ID",
  "category": "datapoint",
  "constraints": {
    "length": {
      "max": 32,
      "min": 5
    },
    "required": false
  },
  "default_value": null,
  "rir_field_names": [
    "document_id"
  ]
}

Constraints limit allowed values. When constraints is not satisfied, annotation is considered invalid and cannot be exported.

Multivalue

Example of a multivalue:

{
  "category": "multivalue",
  "id": "line_item",
  "label": "Line Item",
  "children": {
    ...
  },
  "show_grid_by_default": false,
  "min_occurrences": null,
  "max_occurrences": null,
  "rir_field_names": null
}

Example of a multivalue with grid row-types specification:

{
  "category": "multivalue",
  "id": "line_item",
  "label": "Line Item",
  "children": {
    ...
  },
  "grid": {
    "row_types": [
      "header", "data", "footer"
    ],
    "default_row_type": "data",
    "row_types_to_extract": [
      "data"
    ]
  },
  "min_occurrences": null,
  "max_occurrences": null,
  "rir_field_names": ["line_items"]
}

Multivalue is list of datapoints or tuples of the same type. It represents a container for data with multiple occurrences (such as line items) and can contain only objects with the same id.

Multivalue grid object

Multivalue grid object allows to specify a row type for each row of the grid. For data representation of actual grid data rows see Grid object description.

For example to distinguish two header types and a footer in the validation interface, following row types may be used: header, subsection_header, data and footer.

Currently, data extraction classifies every row as either data or header (additional row types may be introduced in the future). We remove rows returned by data extraction that are not in row_types list (e.g. header by default) and are on the top/bottom of the table. When they are in the middle of the table, we mark them as skipped (null).

There are three visual modes, based on row_types quantity:

• More than two row types defined: User selects row types freely to any non-default row type. Clearing row type resets to a default row type. We support up to 6 colors to easily distinguish row types visually.
• Two row types defined (header and default): User only marks header and skipped rows. Clearing row type resets to a default row type.
• One row type defined: User is only able to mark row as skipped (null value in data). This is also a default behavior when no grid row types configuration is specified in the schema.

Tuple

Example of a tuple:

{
  "category": "tuple",
  "id": "tax_details",
  "label": "Tax Details",
  "children": [
    ...
  ],
  "rir_field_names": [
    "tax_details"
  ]
}

Container representing tabular data with related values, such as tax details. A tuple must be nested within a multivalue object, but unlike multivalue, it may consist of objects with different ids.

Updating Schema

When project evolves, it is a common practice to enhance or change the extracted field set. This is done by updating the schema object.

By design, Rossum supports multiple schema versions at the same time. However, each document annotation is related to only one of those schemas. If the schema is updated, all related document annotations are updated accordingly. See preserving data on schema change below for limitations of schema updates.

In addition, every queue is linked to a schema, which is used for all newly imported documents.

When updating a schema, there are two possible approaches:

• Update the schema object (PUT/PATCH). All related annotations will be updated to match current schema shape (even exported and deleted documents).
• Create a new schema object (POST) and link it to the queue. In such case, only newly created objects will use the current schema. All previously created objects will remain in the shape of their linked schema.

Use case 1 - Initial setting of a schema

• Situation: User is initially setting up the schema. This might be an iterative process.
• Recommendation: Edit the existing schema by updating schema (PUT) or updating part of a schema (PATCH).

Use case 2 - Updating attributes of a field (label, constraints, options, etc.)

• Situation: User is updating field, e.g. changing label, number format, constraints, enum options, hidden flag, etc.
• Recommendation: Edit existing schema (see Use case 1).

Use case 3 - Adding new field to a schema, even for already imported documents.

• Situation: User is extending a production schema by adding a new field. Moreover, user would like to see all annotations (to_review, postponed, exported, deleted, etc. states) in the look of the newly extended schema.
• Recommendation: Edit existing schema (see Use case 1). Data of already created annotations will be transformed to the shape of the updated schema. New fields of annotations in to_review and postponed state that are linked to extracted fields types will be filled by AI Engine, if available. New fields for already exported or deleted annotations (also purged, exporting and failed_export) will be filled with empty or default values.

Use case 4 - Adding new field to schema, only for newly imported documents

• Situation: User is extending a production schema by adding a new field. However, with the intention that the user does not want to see the newly added field on previously created annotations.
• Recommendation: Create a new schema object (POST) and link it to the queue. Annotation data of previously created annotations will be preserved according to the original schema. This approach is recommended if there is an organizational need to keep different field sets before and after the schema update.

Use case 5 - Deleting schema field, even for already imported documents.

• Situation: User is changing a production schema by removing a field that was used previously. However, user would like to see all annotations (to_review, postponed, exported, deleted, etc. states) in the look of the newly extended schema. There is no need to see the original fields in already exported annotations.
• Recommendation: Edit existing schema (see Use case 1).

Use case 6 - Deleting schema field, only for newly imported documents

• Situation: User is changing a production schema by removing a field that was used previously. However, with the intention that the user will still be able to see the removed fields on previously created annotations.
• Recommendation: Create a new schema object (see Use case 4). Annotation data of previously created annotations will be preserved according to the original schema. This approach is recommended if there is an organizational need to retrieve the data in the original state.

Preserving data on schema change

In order to transfer annotation field values properly during the schema update, a datapoint's category and schema_id must be preserved.

Supported operations that preserve fields values are:

• adding a new datapoint (filled from AI Engine, if available)
• reordering datapoints on the same level
• moving datapoints from section to another section
• moving datapoints to and from a tuple
• reordering datapoints inside a tuple
• making datapoint a multivalue (original datapoint is the only value in a new multivalue container)
• making datapoint non-multivalue (only first datapoint value is preserved)

Extracted field types

AI engine currently automatically extracts the following fields at the all endpoint, subject to ongoing expansion.

Identifiers

Example of a schema with different identifiers:

[
  {
    "category": "section",
    "children": [
      {
        "category": "datapoint",
        "constraints": {
          "required": false
        },
        "default_value": null,
        "id": "document_id",
        "label": "Invoice number",
        "rir_field_names": [
          "document_id"
        ],
        "type": "string"
      },
      {
        "category": "datapoint",
        "constraints": {
          "required": false
        },
        "default_value": null,
        "format": "D/M/YYYY",
        "id": "date_issue",
        "label": "Issue date",
        "rir_field_names": [
          "date_issue"
        ],
        "type": "date"
      },
      {
        "category": "datapoint",
        "constraints": {
          "required": false
        },
        "default_value": null,
        "id": "terms",
        "label": "Terms",
        "rir_field_names": [
          "terms"
        ],
        "type": "string"
      }
    ],
    "icon": null,
    "id": "invoice_info_section",
    "label": "Basic information"
  }
]

Document attributes

Amounts

Typical relations (may depend on local laws):

amount_total = amount_total_base + amount_total_tax
amount_rounding = amount_total - round(amount_total)
amount_due = amount_total - amount_paid + amount_rounding

All amounts are in the main currency of the invoice (as identified in the currency response field). Amounts in other currencies are generally excluded.

Tables

At the moment, the AI engine automatically extracts 2 types of tables. In order to pick one of the possible choices, set rir_field_names attribute on multivalue.

Tax details

Example of a tax details table:

{
    "category": "section",
    "children": [
      {
        "category": "multivalue",
        "children": {
          "category": "tuple",
          "children": [
            {
              "category": "datapoint",
              "constraints": {
                "required": false
              },
              "default_value": null,
              "format": "# ##0.#",
              "id": "vat_detail_rate",
              "label": "VAT rate",
              "rir_field_names": [
                "tax_detail_rate"
              ],
              "type": "number",
              "width": 15
            },
            ...
          ],
          "id": "vat_detail",
          "label": "VAT detail"
        },
        "default_value": null,
        "id": "vat_details",
        "label": "VAT details",
        "max_occurrences": null,
        "min_occurrences": null,
        "rir_field_names": [
          "tax_details"
        ]
      }
    ],
    "icon": null,
    "id": "amounts_section",
    "label": "Amounts section"
}

Tax details table and breakdown by tax rates.

Line items

Example of a line items table:

    {
    "category": "section",
    "children": [
      {
        "category": "multivalue",
        "children": {
          "category": "tuple",
          "children": [
            {
              "category": "datapoint",
              "constraints": {
                "required": true
              },
              "default_value": null,
              "id": "item_desc",
              "label": "Description",
              "rir_field_names": [
                "table_column_description"
              ],
              "type": "string",
              "stretch": true
            },
            {
              "category": "datapoint",
              "constraints": {
                "required": false
              },
              "default_value": null,
              "format": "# ##0.#",
              "id": "item_quantity",
              "label": "Quantity",
              "rir_field_names": [
                "table_column_quantity"
              ],
              "type": "number",
              "width": 15
            },
            {
              "category": "datapoint",
              "constraints": {
                "required": false
              },
              "default_value": null,
              "format": "# ##0.#",
              "id": "item_amount_total",
              "label": "Price w tax",
              "rir_field_names": [
                "table_column_amount_total"
              ],
              "type": "number"
            }
          ],
          "id": "line_item",
          "label": "Line item",
          "rir_field_names": []
        },
        "default_value": null,
        "id": "line_items",
        "label": "Line item",
        "max_occurrences": null,
        "min_occurrences": null,
        "rir_field_names": [
          "line_items"
        ]
      }
    ],
    "icon": null,
    "id": "line_items_section",
    "label": "Line items"
  }

AI engine currently automatically extracts line item table content and recognizes row and column types as detailed below. Invoice line items come in a wide variety of different shapes and forms. The current implementation can deal with (or learn) most layouts, with borders or not, different spacings, header rows, etc. We currently make two further assumptions:

• The table generally follows a grid structure - that is, columns and rows may be represented as rectangle spans. In practice, this means that we may currently cut off text that overlaps from one cell to the next column. We are also not optimizing for table rows that are wrapped to multiple physical lines.
• The table contains just a flat structure of line items, without subsection headers, nested tables, etc.

We plan to gradually remove both assumptions in the future.

Annotation Lifecycle

When a document is submitted to Rossum within a given queue, an annotation object is assigned to it. An annotation goes through a variety of states as it is processed, and eventually exported.

Usage report

In order to obtain an overview of the Rossum usage, you can download Excel file with basic Rossum statistics.

The statistics contains following attributes:

• Username (may be empty in case document was not modified by any user)
• Workspace name
• Queue name
• Imported: count of all documents that were imported during the time period
• Deleted: count of documents that were deleted during the time period
• Exported: count of documents that were successfully exported during the time period
• Net time: total time spent by a user validating the successfully exported documents

Download usage statistics (January 2019).

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' \
  'https://api.elis.rossum.ai/manage/usage-report?from=2019-01-01&to=2019-01-31'

Excel file (xlsx) may be downloaded from https://api.elis.rossum.ai/manage/usage-report.

You may specify date range using from and to parameters (inclusive). If not specified, a report for last 12 months is generated.

New API

A new API, which contains more information, is being introduced. Eventually, it will replace the original https://api.elis.rossum.ai/manage/usage-report usage report.

Download usage report (December 2019 - February 2020).

curl -H 'Authorization: token db313f24f5738c8e04635e036ec8a45cdd6d6b03' -H 'Content-Type: application/json' \
  'https://api.elis.rossum.ai/v1/annotations/usage-report' -d '{"filter": {"begin_date": "2019-12-01", "end_date": "2020-01-31"}}'

Request

POST /v1/annotations/usage-report

{
  "filter": {
    "users": [
      "https://api.elis.rossum.ai/v1/users/173"
    ],
    "queues": [
      "https://api.elis.rossum.ai/v1/queues/8199"
    ],
    "begin_date": "2019-12-01",
    "end_date": "2020-01-31"
  },
  "exported_on_time_threshold_s": 86400,
  "group_by": [
    "user",
    "workspace",
    "queue",
    "month"
  ]
}

Response

Status: 200

{
  "series": [
    {
      "begin_date": "2019-12-01",
      "end_date": "2020-01-01",
      "queue": "https://api.elis.rossum.ai/v1/queues/8199",
      "workspace": "https://api.elis.rossum.ai/v1/workspaces/7540",
      "values": {
        "imported_count": 2,
        "deleted_count": null,
        "exported_count": null,
        "turnaround_avg_s": null,
        "corrections_per_document_avg": null,
        "exported_on_time_count": null,
        "exported_late_count": null,
        "time_per_document_avg_s": null,
        "time_and_corrections_per_field": []
      }
    },
    {
      "begin_date": "2020-01-01",
      "end_date": "2020-02-01",
      "queue": "https://api.elis.rossum.ai/v1/queues/8199",
      "workspace": "https://api.elis.rossum.ai/v1/workspaces/7540",
      "user": "https://api.elis.rossum.ai/v1/users/173",
      "values": {
        "imported_count": null,
        "deleted_count": 2,
        "exported_count": 2,
        "turnaround_avg_s": 1341000,
        "corrections_per_document_avg": 1.0,
        "exported_on_time_count": 1,
        "exported_late_count": 1,
        "time_per_document_avg_s": 70.0,
        "time_and_corrections_per_field": [
          {
            "schema_id": "date_due",
            "label": "Date due",
            "total_count": 1,
            "corrected_ratio": 0.0,
            "time_spent_avg_s": 0.0
          },
          ...
        ]
      }
    },
    ...
  ],
  "totals": {
    "imported_count": 7,
    "deleted_count": 2,
    "exported_count": 3,
    "turnaround_avg_s": 894000,
    "corrections_per_document_avg": 1.0,
    "exported_on_time_count": 2,
    "exported_late_count": 1,
    "time_per_document_avg_s": 70.0
  }
}

The response consists of two parts: totalsand series.

Totals

Totals contain summary information for the whole period (between begin_date and end_date).

Series

Series contain information grouped by fields defined in group_by. The data (see above) are wrapped in values object, and accompanied by the values of of attributes that were used for grouping.

In addition to the totals data, series contain time_and_corrections_per_field list that provides detailed data about statistics on each field.

Series details

The detail object contains statistics grouped per field (schema_id).

Extensions

The Rossum platform may be extended via third-party, externally running services or custom functions. These extensions are registered to receive callbacks from the Rossum platform on various occasions and allow to modify the platform behavior. Currently we support these callback extensions: Webhooks, Serverless Functions, and Connectors.

Webhooks and connectors require a third-party service accessible through a HTTP endpoint. This may incur additional operational and implementation costs. User-defined serverless functions, on the contrary, are executed within Rossum platform and no additional setup is necessary. They share the interface (input and output data format, error handling) with webhooks.

See the Building Your Own Extension set of guides in Rossum's developer portal for an introduction to Rossum extensions.

Webhook Extension

The webhook component is the most flexible extension. It covers all the most frequent use cases:

• displaying messages to users in validation screen,
• applying custom business rules to check, change, or autofill missing values,
• reacting to document status change in your workflow,
• sending reviewed data to an external systems.

Implement a webhook

Webhooks are designed to be implemented using a push-model using common HTTPS protocol. When an event is triggered, the webhook endpoint is called with a relevant request payload. The webhook must be deployed with a public IP address so that the Rossum platform can call its endpoints; for testing, a middleware like ngrok or serveo may come useful.

Webhook vs. Connector

Webhook extensions are similar to connectors, but they are more flexible and easier to use. A webhook is notified when a defined type of webhook event occurs for a related queue.

Advantages of webhooks over connectors:

• There may be multiple webhooks defined for a single queue
• No hard-coded endpoint names (/validate, /save)
• Robust retry mechanism in case of webhook failure

Webhooks are defined using a hook object of type webhook. For a description how to create and manage hooks, see the Hook API.

Webhook Events

Example data sent for annotation_status event to the hook.config.url when status of the annotation changes

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5514b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/789",
  "action": "changed",
  "event": "annotation_status",
  "annotation": {
    "document": "https://api.elis.rossum.ai/v1/documents/314621",
    "id": 314521,
    "queue": "https://api.elis.rossum.ai/v1/queues/8236",
    "schema": "https://api.elis.rossum.ai/v1/schemas/223",
    "pages": [
      "https://api.elis.rossum.ai/v1/pages/551518"
    ],
    "modifier": null,
    "modified_at": null,
    "confirmed_at": null,
    "exported_at": null,
    "assigned_at": null,
    "status": "to_review",
    "previous_status": "importing",
    "rir_poll_id": "54f6b91cfb751289e71ddf12",
    "messages": null,
    "url": "https://api.elis.rossum.ai/v1/annotations/314521",
    "content": "https://api.elis.rossum.ai/v1/annotations/314521/content",
    "time_spent": 0,
    "metadata": {}
  },
  "document": {
    "id": 314621,
    "url": "https://api.elis.rossum.ai/v1/documents/314621",
    "s3_name": "272c2f41ae84a4f19a422cb432a490bb",
    "mime_type": "application/pdf",
    "arrived_at": "2019-02-06T23:04:00.933658Z",
    "original_file_name": "test_invoice_1.pdf",
    "content": "https://api.elis.rossum.ai/v1/documents/314621/content",
    "metadata": {}
  }
}

Example data sent for annotation_content event to the hook.config.url when user updates a value in UI

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5214b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/781",
  "action": "user_update",
  "event": "annotation_content",
  "annotation": {
    "document": "https://api.elis.rossum.ai/v1/documents/314621",
    "id": 314521,
    "queue": "https://api.elis.rossum.ai/v1/queues/8236",
    "schema": "https://api.elis.rossum.ai/v1/schemas/223",
    "pages": [
      "https://api.elis.rossum.ai/v1/pages/551518"
    ],
    "modifier": null,
    "modified_at": null,
    "confirmed_at": null,
    "exported_at": null,
    "assigned_at": null,
    "status": "to_review",
    "previous_status": "importing",
    "rir_poll_id": "54f6b91cfb751289e71ddf12",
    "messages": null,
    "url": "https://api.elis.rossum.ai/v1/annotations/314521",
    "content": [
        {
            "id": 1123123,
            "url": "https://api.elis.rossum.ai/v1/annotations/314521/content/1123123",
            "schema_id": "basic_info",
            "category": "section",
            "children": [
              {
                "id": 20456864,
                "url": "https://api.elis.rossum.ai/v1/annotations/1/content/20456864",
                "content": {
                  "value": "18 492.48",
                  "normalized_value": "18492.48",
                  "page": 2,
                  ...
              },
            "category": "datapoint",
            "schema_id": "number",
            "validation_sources": [
              "checks",
              "score"
            ],
            "time_spent": 0
              }
            ]
        }
    ],
    "time_spent": 0,
    "metadata": {}
  },
  "document": {
    "id": 314621,
    "url": "https://api.elis.rossum.ai/v1/documents/314621",
    "s3_name": "272c2f41ae84a4f19a422cb432a490bb",
    "mime_type": "application/pdf",
    "arrived_at": "2019-02-06T23:04:00.933658Z",
    "original_file_name": "test_invoice_1.pdf",
    "content": "https://api.elis.rossum.ai/v1/documents/314621/content",
    "metadata": {}
  },
  "updated_datapoints": [11213211]
}

Example of a response for annotation_content hook

{
  "messages": [
    {
      "content": "Invalid invoice number format",
      "id": 197467,
      "type": "error"
    }
  ],
  "operations": [
    {
      "op": "replace",
      "id": 198143,
      "value": {
        "content": {
          "value": "John",
          "position": [103, 110, 121, 122],
          "page": 1
        },
        "hidden": false,
        "options": [],
        "validation_sources": ["human"]
      }
    },
    {
      "op": "remove",
      "id": 884061
    },
    {
      "op": "add",
      "id": 884060,
      "value": [
        {
          "schema_id": "item_description",
          "content": {
            "page": 1,
            "position": [162, 852, 371, 875],
            "value": "Bottle"
          }
        }
      ]
    }
  ]
}

Example data sent for email event to the hook.config.url when email is received by Rossum mail server

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5214b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/781",
  "action": "received",
  "event": "email",
  "email": "https://api.elis.rossum.ai/v1/emails/987",
  "queue": "https://api.elis.rossum.ai/v1/queues/41",
  "files": [
    {
      "id": "1",
      "filename": "image.png",
      "mime_type": "image/png",
      "n_pages": 1,
      "height_px": 100.0,
      "width_px": 150.0,
      "document": "https://api.elis.rossum.ai/v1/documents/427"
    },
    {
      "id": "2",
      "filename": "MS word.docx",
      "mime_type": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
      "n_pages": 1,
      "height_px": null,
      "width_px": null,
      "document": "https://api.elis.rossum.ai/v1/documents/428"
    },
    {
      "id": "3",
      "filename": "A4 pdf.pdf",
      "mime_type": "application/pdf",
      "n_pages": 3,
      "height_px": 3510.0,
      "width_px": 2480.0,
      "document": "https://api.elis.rossum.ai/v1/documents/429"
    },
    {
      "id": "4",
      "filename": "unknown_file",
      "mime_type": "text/xml",
      "n_pages": 1,
      "height_px": null,
      "width_px": null,
      "document": "https://api.elis.rossum.ai/v1/documents/430"
    }
  ],
  "headers": {
    "from": "test@example.com",
    "to": "east-west-trading-co-a34f3a@elis.rossum.ai",
    "subject": "Some subject",
    "date": "Mon, 04 May 2020 11:01:32 +0200",
    "message-id": "15909e7e68e4b5f56fd78a3b4263c4765df6cc4d"
  },
  "body": {
    "body_text_plain": "Some body",
    "body_text_html": "<div dir=\"ltr\">Some body</div>"
  }
}

Example of a response for email hook

{
  "files": [
    {
      "id": "3",
      "values": [
        {
          "id": "email:invoice_id",
          "value": "INV001234"
        },
        {
          "id": "email:customer_name",
          "value": "John Doe"
        }
      ]
    }
  ]
}

Webhook events specify when the hook should be notified. They can be defined as following:

• either as whole event containing all supported actions for its type (for example annotation_status)
• or as separately named actions for specified event (for example annotation_status.changed)

Supported events and their actions

• When annotation_content.export action fails, annotation is switched to the failed_export state.
• In case a non-interactive webhook call fails (HTTP status >= 400), it is retried within 30 seconds. There are up to 10 attempts performed.
• updated_datapoints list is always empty for annotation_content.initialize and annotation_content.export actions.
• updated_datapoints list may also be empty for annotation_content.user_update in case of an action triggered interactively by a user, but with no data changes (e.g. first validation after opening a validation UI or changes confirmation).
• Retry mechanism is temporarily disabled for email.received action. Meanwhile, we are working on a more robust and reliable way of processing email webhooks.

Annotation status event data format

annotation_status event contains following additional event specific attributes.

^* Attribute is only included in the request when specified in hook.sideload.

Example data sent to hook with sideloaded queue objects

{
  "request_id": "ae7bc8dd-73bd-489b-a3d2-f5214b209591",
  "timestamp": "2020-01-01T00:00:00.000000Z",
  "hook": "https://api.elis.rossum.ai/v1/hooks/781",
  "action": "changed",
  "event": "annotation_status",
  ...
  "queues": [
    {
      "id": 8198,
      "name": "Received invoices",
      "url": "https://api.elis.rossum.ai/v1/queues/8198",
      ...
      "metadata": {},
      "use_confirmed_state": false,
      "settings": {}
    }
  ]
}

Annotation content event data format

annotation_content event contains following additional event specific attributes.

^* Attribute is only included in the request when specified in hook.sideload.

Annotation content event response format

All of the annotation_content events expect a JSON object with the following optional lists in the response: messages and operations

The message object contains attributes:

For example, you may use error for fatals like a missing required field, whereas info is suitable to decorate a supplier company id with its name as looked up in the suppliers database.

The operations object describes operation to be performed on the annotation data (replace, add, remove). Format of the operations key is the same as for bulk update of annotations, please refer to the annotation data API for complete description.

Email received event data format

email event contains following additional event specific attributes.

The files object contains attributes:

The headers object contains the same values as are available for initialization of values in email_header:<id> (namely: from, to, subject, message-id, date).

The body object contains the body_text_plain and body_text_html.

Email received event response format

All of the email events expect a JSON object with the following list in the response: files

The files object contains attributes:

The values object consists of following:

This is useful for filtering out unwanted files by some measures that are not available in Rossum by default.

Validating payloads from Rossum

Example of hook receiver, which verifies the validity of Rossum request

import hashlib
import hmac

from flask import Flask, request, abort

app = Flask(__name__)

SECRET_KEY = "<Your secret key stored in hook.config.secret>"  # never store this in code

@app.route("/test_hook", methods=["POST"])
def test_hook():
    digest = hmac.new(SECRET_KEY.encode(), request.data, hashlib.sha1).hexdigest()
    try:
        prefix, signature = request.headers["X-Elis-Signature"].split("=")
    except ValueError:
        abort(401, "Incorrect header format")

    if not (prefix == "sha1" and hmac.compare_digest(signature, digest)):
        abort(401, "Authorization failed.")
    return

For authorization of payloads, the shared secret method is used. When a secret token is set in hook.config.secret, Rossum uses it to create a hash signature with each payload. This hash signature is passed along with each request in the headers as X-Elis-Signature.

The goal is to compute a hash using hook.config.secret and the request body, and ensure that the signature produced by Rossum is the same. Rossum uses HMAC SHA1 signature.

Webhook requests may be autenticated using a client SSL certificate, see Hook API for reference.

Access to Rossum API

You can access Rossum API from the Webhook. Each execution gets unique API key valid for 10 minutes. Rossum API key is passed to webhooks as a first-level attribute rossum_authorization_token within the webhook payload.

Serverless Function Extension

Serverless functions allows to extend Rossum functionality without setup and maintenance of additional services.

Webhooks and Serverless functions share a basic setup: input and output data format and error handling. They are both configured using a hook API object.

Unlike webhooks, serverless functions do not send the event and action notifications to a specific URL. Instead, the function's code snippet is executed within the Rossum platform. See function API description for details about how to setup a serverless function and connect it to the queue.

Supported events and their actions

For description of supported events, actions and input/output data examples, please refer to Webhook Extensions section.

Supported runtimes

Currently Rossum supports NodeJS 12 runtime (nodejs12.x) to execute JavaScript functions and (python3.8) to execute Python. If you would like to use another runtime, please let us know at product@rossum.ai.

Please be aware that we may eventually deprecate and remove runtimes in the future (deprecation will be announced at least 6 months before the deprecation date).

Implementation

Example serverless function usable for annotation_content event (Python implementation).

'''
This custom function example can be used for showing custom messages to the
user on the validation screen or for updating values of specific fields.
(annotation_content event and user_update action which provides annotation
content tree as an input). The function below shows how to:
1. Display a warning message to the user if "item_amount_base" field of
a line item exceeds a predefined threshold
2. Removes all dashes from the "document_id" field

item_amount_base and document_id should be fields defined in a schema.

More about custom functions - https://developers.rossum.ai/docs/how-to-use-serverless-functions
'''

'''
The rossum_hook_request_handler is a mandatory main function that accepts
input and produces output of the rossum custom function hook.
:param payload: see https://api.elis.rossum.ai/docs/#annotation-content-event-data-format
:return: messages and operations that update the annotation content or show messages
'''


def rossum_hook_request_handler(payload):

    if payload['event'] == 'annotation_content' and payload['action'] == 'user_update':

        try:

            messages, operations = example_main_function(payload)

        except Exception as e:
            messages = [create_message('error', 'Serverless Function: ' + str(e))]
            return {"messages": messages}

        return {"messages": messages, "operations": operations}


'''
Main function that implements the custom logic for messages and operations on datapoints.
:param payload: see https://api.elis.rossum.ai/docs/#annotation-content-event-data-format
:return: tuple - messages and operations to be returned from the hook
'''


def example_main_function(payload):

    messages = []
    operations = []

    content = payload['annotation']['content']

    # Values over the threshold trigger a warning message
    TOO_BIG_THRESHOLD = 1000000;

    # List of all datapoints of item_amount_base schema id
    amount_base_column_datapoints = find_by_schema_id(content, 'item_amount_base');

    for amount_base_column_datapoint in amount_base_column_datapoints:

        # Use normalized_value for comparing values of
        # Date and Number fields (https://api.elis.rossum.ai/docs/#content-object)
        value = float(amount_base_column_datapoint['content']['normalized_value'] or 0)
        if value >= TOO_BIG_THRESHOLD:
            messages.append(create_message('warning', 'Value is too big', amount_base_column_datapoint['id']))

        # There should be only one datapoint of document_id schema id
        document_id_datapoint = find_by_schema_id(content, 'document_id')[0]

        if document_id_datapoint:
            operations = [
                create_replace_operation(
                    document_id_datapoint,
                    document_id_datapoint['content']['value'].replace('-', ''),
                )];

    return messages, operations


'''
Return datapoints matching a schema id.
:param content: annotation content tree (see https://api.elis.rossum.ai/docs/#annotation-data)
:param schema_id: field's ID as defined in the extraction schema(see https://api.elis.rossum.ai/docs/#document-schema)
:param accumulator: list for accumulating values with the same schema_id (f.e. values from same table column)
:return: the list of datapoints matching the schema ID
'''


def find_by_schema_id(content, schema_id: str):
    accumulator = []
    for node in content:
        if node["schema_id"] == schema_id:
            accumulator.append(node)
        elif "children" in node:
            accumulator.extend(find_by_schema_id(node["children"], schema_id))

    return accumulator


'''
Create a message which will be shown to the user
:param message_type: type of the message, any of {info|warning|error}. Errors prevent confirmation in the UI.
:param message_content: message shown to the user
:param datapoint_id: id of the datapoint where the message will appear (None for "global" messages).
:return: dict with the message definition (see https://api.elis.rossum.ai/docs/#annotation-content-event-response-format)
'''


def create_message(message_type, message_content, datapoint_id=None):
    return {
        "content": message_content,
        "type": message_type,
        "id": datapoint_id,
    }


'''
 Replace the value of the datapoint with a new value.
:param datapoint: content of the datapoint
:param new_value: new value of the datapoint
:return: dict with replace operation definition (see https://api.elis.rossum.ai/docs/#annotation-content-event-response-format)
'''


def create_replace_operation(datapoint, new_value):
    return {
        "op": 'replace',
        "id": datapoint['id'],
        "value": {
            "content": {
                "value": new_value,
            },
        },
    }

Example serverless function usable for annotation_content event (JavaScript/NodeJS implementation).

// This serverless function example can be used for annotation_content events
// (e.g. user_update action). annotation_content events provide annotation
// content tree as the input.
//
// The function below shows how to:
// 1. Display a warning message to the user if "item_amount_base" field of
//    a line item exceeds a predefined threshold
// 2. Removes all dashes from the "invoice_id" field
//
// item_amount_base and invoice_id should be fields defined in a schema.

// --- ROSSUM HOOK REQUEST HANDLER ---

// The rossum_hook_request_handler is an mandatory main function that accepts
// input and produces output of the rossum serverless function hook.
// @param {Object} annotation - see https://api.elis.rossum.ai/docs/#annotation-content-event-data-format
// @returns {Object} - the messages and operations that update the annotation content

exports.rossum_hook_request_handler = ({
  annotation: {
    content
  }
}) => {
  try {
    // Values over the threshold trigger a warning message
    const TOO_BIG_THRESHOLD = 1000000;

    // List of all datapoints of item_amount_base schema id
    const amountBaseColumnDatapoints = findBySchemaId(
      content,
      'item_amount_base',
    );

    messages = [];
    for (var i = 0; i < amountBaseColumnDatapoints.length; i++) {

      // Use normalized_value for comparing values of Date and Number fields (https://api.elis.rossum.ai/docs/#content-object)
      if (amountBaseColumnDatapoints[i].content.normalized_value >= TOO_BIG_THRESHOLD) {
        messages.push(
          createMessage(
            'warning',
            'Value is too big',
            amountBaseColumnDatapoints[i].id,
          ),
        );
      }
    }

    // There should be only one datapoint of invoice_id schema id
    const [invoiceIdDatapoint] = findBySchemaId(content, 'invoice_id');

    // "Replace" operation is returned to update the invoice_id value
    const operations = [
      createReplaceOperation(
        invoiceIdDatapoint,
        invoiceIdDatapoint.content.value.replace(/-/g, ''),
      ),
    ];

    // Return messages and operations to be used to update current annotation data
    return {
      messages,
      operations,
    };
  } catch (e) {
    // In case of exception, create and return error message. This may be useful for debugging.
    const messages = [
      createMessage('error', 'Serverless Function: ' + e.message)
    ];
    return {
      messages,
    };
  }
};

// --- HELPER FUNCTIONS ---

// Return datapoints matching a schema id.
// @param {Object} content - the annotation content tree (see https://api.elis.rossum.ai/docs/#annotation-data)
// @param {string} schemaId - the field's ID as defined in the extraction schema(see https://api.elis.rossum.ai/docs/#document-schema)
// @returns {Array} - the list of datapoints matching the schema ID

const findBySchemaId = (content, schemaId) =>
  content.reduce(
    (results, dp) =>
    dp.schema_id === schemaId ? [...results, dp] :
    dp.children ? [...results, ...findBySchemaId(dp.children, schemaId)] :
    results,
    [],
  );

// Create a message which will be shown to the user
// @param {number} datapointId - the id of the datapoint where the message will appear (null for "global" messages).
// @param {String} messageType - the type of the message, any of {info|warning|error}. Errors prevent confirmation in the UI.
// @param {String} messageContent - the message shown to the user
// @returns {Object} - the JSON message definition (see https://api.elis.rossum.ai/docs/#annotation-content-event-response-format)

const createMessage = (type, content, datapointId = null) => ({
  content: content,
  type: type,
  id: datapointId,
});

// Replace the value of the datapoint with a new value.
// @param {Object} datapoint - the content of the datapoint
// @param {string} - the new value of the datapoint
// @return {Object} - the JSON replace operation definition (see https://api.elis.rossum.ai/docs/#annotation-content-event-response-format)

const createReplaceOperation = (datapoint, newValue) => ({
  op: 'replace',
  id: datapoint.id,
  value: {
    content: {
      value: newValue,
    },
  },
});

To implement a serverless function, create a hook object of type function. Use code object config attribute to specify a serialized source code. You can use a code editor built-in to the Rossum UI, which also allows to test and debug the function before updating the code of the function itself.

See Python and NodeJS examples of a serverless function implementation.

Limitations

By default, no internet access is allowed from a serverless function, except the Rossum API. If your functions require internet access to work properly, e.g. when exporting data over API to ERP system, please let us know at product@rossum.ai.

Access Rossum API

The access to the Rossum API is granted through a proxy server, HTTPS_PROXY environment variable should be used to get its URL. See examples below to see how to access Rossum API from a serverless function. Python's urllib.request can handle HTTPS proxy from environment variable on its own.

Python code snippet to access Rossum API to get a list of queue names

import json
import urllib.request

def rossum_hook_request_handler(payload):
    request = urllib.request.Request(
      "https://api.elis.rossum.ai/v1/queues",
      headers={"Authorization": "token " + payload["rossum_authorization_token"]}
    )
    with urllib.request.urlopen(request) as response:
        queues = json.loads(response.read())
    queue_names = (q["name"] for q in queues["results"])
    return {"messages": [{"type": "info", "content": ", ".join(queue_names)}]}

NodeJS code snippet to access Rossum API to get a list of queue names

exports.rossum_hook_request_handler = async ({
  annotation: {
    content
  },
  rossum_authorization_token: token
}) => {
  queues = JSON.parse(await getFromRossumApi("https://api.elis.rossum.ai/v1/queues", token));
  queue_names = queues.results.map(q => q.name).join(", ")
  return { "messages": [{"type": "info", "content": queue_names}] };
}

const getFromRossumApi = async (url, token) => {
  var http = require('http');
  const proxy = new URL(process.env.HTTPS_PROXY);
  const options = {
    hostname: proxy.hostname,
    port: proxy.port,
    path: url,
    method: 'GET',
    headers: {
      'Authorization': 'token ' + token,
    },
  };
  const response = await new Promise((resolve, reject) => {
    let dataString = '';
    const req = http.request(options, function(res) {
      res.on('data', chunk => {
        dataString += chunk;
      });
      res.on('end', () => {
        resolve({
          statusCode: 200,
          body: dataString
        });
      });
    });
    req.on('error', (e) => {
      reject({
        statusCode: 500,
        body: 'Something went wrong!'
      });
    });
    req.end()
  });
  return response.body
}

Connector Extension

The connector component is aimed at two main use-cases: applying custom business rules during data validation, and direct integration of Rossum with downstream systems.

The connector component receives two types of callbacks - an on-the-fly validation callback on every update of captured data, and an on-export save callback when the document capture is finalized.

The custom business rules take use chiefly of the on-the-fly validation callback. The connector can auto-validate and transform both the initial AI-based extractions and each user operator edit within the validation screen; based on the input, it can push user-visible messages and value updates back to Rossum. This allows for both simple tweaks (like verifying that two amounts sum together or transforming decimal points to thousand separators) and complex functionality like intelligent PO match.

The integration with downstream systems on the other hand relies mainly on the save callback. At the same moment a document is exported from Rossum, it can be imported to a downstream system. Since there are typically constraints on the captured data, these constraints can be enforced even within the validation callback.

Implement a connector

Connectors are designed to be implemented using a push-model using common HTTPS protocol. When annotation data is changed, or when data export is triggered, specific connector endpoint is called with annotation data as a request payload. The connector must be deployed with a public IP address so that the Rossum platform can call its endpoints; for testing, a middleware like ngrok or serveo may come useful.

Example of a valid no-op (empty) validate response

{"messages": [], "updated_datapoints": []}

Example of a valid no-op (empty) save response

{}

The connector API consists of two endpoints, validate and save, described below. A connector must always implement both endpoints (though they may not necessarily perform a function in a particular connector - see the right column for an empty reply example), the platform raises an error if it is not able to run a endpoint.

Setup a connector

The next step after implementing the first version of a connector is configuring it in the Rossum platform.

In Rossum, a connector object defines service_url and params for construction of HTTPS requests and authorization_token that is passed in every request to authenticate the caller as the actual Rossum server. It may also uniquely identify the organization when multiple Rossum organizations share the same connector server.

To set-up a connector for a queue, create a connector object using either our API or the rossum tool – follow these instructions. A connector object may be associated with one or more queues. One queue can only have one connector object associated with it.

Connector API

Example data sent to connector (validate, save)

{
  "meta": {
    "document_url": "https://api.elis.rossum.ai/v1/documents/6780",
    "arrived_at": "2019-01-30T07:55:13.208304Z",
    "original_file": "https://api.elis.rossum.ai/v1/original/bf0db41937df8525aa7f3f9b18a562f3",
    "original_filename": "Invoice.pdf",
    "queue_name": "Invoices",
    "workspace_name": "EU",
    "organization_name": "East West Trading Co",
    "annotation": "https://api.elis.rossum.ai/v1/annotations/4710",
    "queue": "https://api.elis.rossum.ai/v1/queues/63",
    "workspace": "https://api.elis.rossum.ai/v1/workspaces/62",
    "organization": "https://api.elis.rossum.ai/v1/organizations/1",
    "modifier": "https://api.elis.rossum.ai/v1/users/27",
    "updated_datapoint_ids": ["197468"],
    "modifier_metadata": {},
    "queue_metadata": {},
    "annotation_metadata": {},
    "rir_poll_id": "54f6b9ecfa751789f71ddf12",
    "automated": false
  },
  "content": [
    {
      "id": "197466",
      "category": "section",
      "schema_id": "invoice_info_section",
      "children": [
        {
          "id": "197467",
          "category": "datapoint",
          "schema_id": "invoice_number",
          "page": 1,
          "position": [916, 168, 1190, 222],
          "rir_position": [916, 168, 1190, 222],
          "rir_confidence": 0.97657,
          "value": "FV103828806S",
          "validation_sources": ["score"],
          "type": "string"
        },
        {
          "id": "197468",
          "category": "datapoint",
          "schema_id": "date_due",
          "page": 1,
          "position": [938, 618, 1000, 654],
          "rir_position": [940, 618, 1020, 655],
          "rir_confidence": 0.98279,
          "value": "12/22/2018",
          "validation_sources": ["score"],
          "type": "date"
        },
        {
          "id": "197469",
          "category": "datapoint",
          "schema_id": "amount_due",
          "page": 1,
          "position": [1134, 1050, 1190, 1080],
          "rir_position": [1134, 1050, 1190, 1080],
          "rir_confidence": 0.74237,
          "value": "55.20",
          "validation_sources": ["human"],
          "type": "number"
        }
      ]
    },
    {
      "id": "197500",
      "category": "section",
      "schema_id": "line_items_section",
      "children": [
        {
          "id": "197501",
          "category": "multivalue",
          "schema_id": "line_items",
          "children": [
            {
              "id": "198139",
              "category": "tuple",
              "schema_id": "line_item",
              "children": [
                {
                  "id": "198140",
                  "category": "datapoint",
                  "schema_id": "item_desc",
                  "page": 1,
                  "position": [173, 883, 395, 904],
                  "rir_position": null,
                  "rir_confidence": null,
                  "value": "Red Rose",
                  "validation_sources": [],
                  "type": "string"
                },
                {
                  "id": "198142",
                  "category": "datapoint",
                  "schema_id": "item_net_unit_price",
                  "page": 1,
                  "position": [714, 846, 768, 870],
                  "rir_position": null,
                  "rir_confidence": null,
                  "value": "1532.02",
                  "validation_sources": ["human"],
                  "type": "number"
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

All connector endpoints, representing particular points in the document lifetime, are simple verbs that receive a JSON POSTed and potentially expect a JSON returned in turn.

The authorization type and authorization token is passed as an Authorization HTTP header. Authorization type may be secret_key (shared secret) or Basic for HTTP basic authentication.

Please note that for Basic authentication, authorization_token is passed as-is, therefore it must be set to a correct base64 encoded value. For example username connector and password secure123 is encoded as Y29ubmVjdG9yOnNlY3VyZTEyMw== authorization token.

Connector requests may be autenticated using a client SSL certificate, see Connector API for reference.

Errors

If a connector does not implement an endpoint, it may return HTTP status 404. An endpoint may fail, returning either HTTP 4xx or HTTP 5xx; for some endpoints (like validate and save), this may trigger a user interface message; either the error key of a JSON response is used, or the response body itself in case it is not JSON. The connector endpoint save can be called in asynchronous (default) as well as synchronous mode (useful for embedded mode).

Data format

The received JSON object contains two keys, meta carrying the metadata and content carrying endpoint-specific content.

The metadata identify the concerned document, containing attributes:

A common class of content is the annotation tree, which is a JSON object that can contain nested datapoint objects, and matches the schema datapoint tree.

Intermediate nodes have the following structure:

Datapoint (leaf) nodes structure contains actual data:

Annotation lifecycle with a connector

If an asynchronous connector is deployed to a queue, an annotation status will change from reviewing to exporting and subsequently to exported or failed_export. If no connector extension is deployed to a queue or if the attribute asynchronous is set to false, an annotation status will change from reviewing to exported (or failed_export) directly.

Endpoint: validate

This endpoint is called after the document processing has finished, when operator opens a document in the Rossum verification interface and then every time after operator updates a field. After the processing is finished, the initial validate call is marked with initial=true URL parameter. For the other calls, only /validate without the parameter is called.

The request path is fixed to /validate and cannot be changed.

It may:

• validate the given annotation tree and return a list of messages commenting on it (e.g. pointing out errors or showing matched suppliers).
• update the annotation tree by returning a list of replace, add and remove operations

Both the messages and the updated data are shown in the verification interface. Moreover, the messages may block confirmation in the case of errors.

This endpoint should be fast as it is part of an interactive workflow.

Receives an annotation tree as content.

Returns a JSON object with the lists: messages, operations and updated_datapoints.

Keys messages, operations (optional)

The description of these keys was moved to the Hook Extension. See the description here.

Key updated_datapoints (optional, deprecated)

We also support a simplified version of updates using updated_datapoints response key. It only supports updates (no add or remove operations) and is now deprecated. The updated datapoint object contains attributes:

Validate endpoint should always return 200 OK status.

An error message returned from the connector prevents user from confirming the document.

Endpoint: save

This endpoint is called when the invoice transitions to the exported state. Connector may process the final document annotation and save it to the target system. It receives an annotation tree as content. The request path is fixed to /save and cannot be changed.

The save endpoint is called asynchronously (unless synchronous mode is set in related connector object. Timeout of the save endpoint is 60 seconds.

For successful export, the request should have 2xx status.

Example of successful save response without messages in UI

HTTP/1.1 204 No Content

HTTP/1.1 200 OK
Content-Type: text/plain

this response body is ignored

HTTP/1.1 200 OK
Content-Type: application/json

{
  "messages": []
}

When messages are expected to be displayed in the UI, they should be sent in the same format as in validate endpoint.

Example of successful save response with messages in UI

HTTP/1.1 200 OK
Content-Type: application/json

{
  "messages": [
    {
      "content": "Everything is OK.",
      "id": null,
      "type": "info"
    }
  ]
}

If the endpoint fails with an HTTP error and/or message of type error is received, the document transitions to the failed_export state - it is then available to the operators for manual review and re-queuing to the to_review state in the user interface. Re-queuing may be done also programmatically via the API using a PATCH call to set to_review annotation status. Patching annotation status to exporting state triggers an export retry.

Example of unsuccessful save response with messages in UI

HTTP/1.1 422 Unprocessable Entity
Content-Type: application/json

{
  "messages": [
    {
      "content": "Even though this message is info, the export will fail due to the status code.",
      "id": null,
      "type": "info"
    }
  ]
}

HTTP/1.1 500 Internal Server Error
Content-Type: text/plain

An errror message "Export failed." will show up in the UI

HTTP/1.1 200 OK
Content-Type: application/json

{
  "messages": [
    {
      "content": "Proper status code could not be set.",
      "id": null,
      "type": "error"
    }
  ]
}

Custom UI Extension

Sometimes users might want to extend the behavior of UI validation view with something special. That should be the goal of custom UI extensions.

Buttons

Currently, there are two different ways of using a custom button:

1. Popup Button - opens a specific URL in the web browser
2. Validate Button - triggers a standard validate call to connector

If you would like to read more about how to create a button, see the Button schema.

Popup Button

Popup Button opens a website completely managed by the user in a separate tab. It runs in parallel to the validation interface session in the app. Such website can be used for any interface that will assist operators in the reviewing process.

Example Use Cases of Popup Button:

1. opening an email linked to the annotated document
2. creating a new item in external database according to extracted data

Communication with the Validation Interface

You can communicate with the validation interface directly using standard browser API of window.postMessage. You will need to use window.addEventListeners in order to receive messages from the validation interface:

The shape of the result key is the same as the top level content attribute of the annotation data response.

Once the listener is in place, you can post one of supported message types:

• GET_DATAPOINTS - returns the same tree structure you’d get by requesting annotation data