Export – byNeuron | Documentation

Documents can be used to from the database. This is configured by document definitions.

Prerequisites

To fully understand this documentation, you need to have prior knowledge about definitions in general.

Introduction

To configure an export you need a document definition.

Following steps are executed when comparing a input file to the database:

The user creates a document with the definition for the export

In the background a service is started to execute query requests

When all data is queried, it transforms the data according the configured mapping

And then uploads the file containing the data

The user can download or preview the data on the document created.

Export Definition

This document definition defines how the export file document is created.

The definition contains following properties.

Property	Description
inputDefinitions	Contains the required inputs when creating the document from a definition
document	Contains the template for creating the document, using the inputs as variables

Until now it is the same as any other definition, but for export there is more configuration required on the document template.

Document template configuration

The document template contains following additional properties in order to configure export.

Property	Description
type	Must be be `EXPORT`
mimeType	Determines the export file type: `text/csv` or `application/json`
processActions	Contains an api request to the document export service with an identification of the document.
data	Contains the database request, mapping and export settings.

Process Actions

Example process actions

"processActions": {
	"apiRequest": {
		"api": "documentdatabaseexport",
		"method": "POST",
		"body": {
			"identification": {
				"id": "{{response.identification.id}}"
			}
		}
	}
}

Data

The data part contains the configuration for the export.

It contains the following properties.

Property	Description
api	Determines which API to use for export.
database.request	`required` Contains the request to be send. Pagination and sorting are important here, because the request will be send in iterations. To optimise queries, you can provide metadata with a custom nuql query to retrieve the data. But the regular request properties are also possible.
database.mappingtype	The type of mapping to use. For now `handlebars` is supported.
database.mapping	`required` Contains the mapping of the request response (dbEntries) into a flat list.
export	Contains the settings for exporting to a CSV file: - `separator` (character) - `quoted` (true of false) - `headers` (true or false)

Example data configuration

"data": {
  "api": "accesspoints",
  "database": {
    "request": {
      "pagination": {
        "offset": 0,
        "limit": 1000
      },
      "sorting": {
        "order": [
          "publicId-"
        ]
      },
      "responseFilter": {
        "links": [
          {
            "matches": "details"
          }
        ],
        "type": "include"
      }
    },
    "mappingtype": "handlebars",
    "mapping": "[<%#each dbEntries%>{\"key\": \"<%@identification.externalId%>\",\"externalId\": \"<%@identification.externalId%>\",\"productType\": \"<%@type%>\",\"marketSegment\": \"<%@details.0.extensions.tenantCodes.edsnMarketSegment%>\"}<%#unless @last%>,<%/unless%><%/each%>]"
  },
  "export": {
    "separator": ";",
    "quoted": true,
    "headers": true
  }
}

Optimise queries with custom nuQL

In order to optimise queries custom nuQL queries can be written to get only the response required and also to order the nuQL statements in specific order to improve query performance for your use case.

A couple of things are important to keep in mind for query optimisation:

Limited the nuQL query as much as possible, only query the data which is really necessary.

For example: only retrieve entities where attribute externalId has a value by using this statement: attribute:externalId value:any.

For example: limiting number of relations by filtering on fromDate, toDate and roles.

‘Throw away’ temporary filter variables which are not necessary for the response, once the constraint is applied by using this statement: if:exists reset. This prevents to keep unnecessary data all the time in memory.

The variables used in the nuQL query need to comply with a convention to be able to translate the result in an API response.

You can execute a ‘regular’ query request on the API with the postfix /nuql in the URL to see the generated nuQL with variables:

For example: /accesspoints/_query/nuql

The sorting and pagination are included as dynamic parameters such that requests can be iterated per page.

Using this syntax: %%sorting%% %%pagination%%