Dataset editor

The dataset editor is the starting point of the Analysis module. This editor allows you to enter data for the Charts When configuring, different data sources can be selected and variables added.

The data is modified using separate sequential blocks (modifiers) that can be connected in series. This allows the data set to be narrowed down to the desired end result in the best possible way. At the same time, the overview of the individual scripts is maintained even in complex processes.

The most important steps when creating and editing a data set are:

Create/edit/delete data records

In this area, new data records can be added or existing data records can be edited or deleted.
The data records are automatically grouped according to the data source.

Buttons and icons
Button/icon Description Function

Add data record

Add

Creates a new dataset

Duplicate

Copies the selected dataset including all configurations and adds the copy to the end of the list or to the end of the group . to the end of the group

Delete

Removes the selected data set

Open group

Displays all entries assigned to this data source type

Close group

Minimizes all entries assigned to this data source type

Sorting data record entries

It is possible to sort both grouped and ungrouped data record entries in ascending or descending order.
Grouped entries are sorted within the individual group.
Consequently, all entries are sorted for non-grouped entries.

The Parameters used for sorting are the columns (see Show and hide columns.

Show and hide columns

The following columns are available for the data records:

  • Name: Name entered by the user

  • Type: Type of data source selected in the configuration

  • ID: Identification number automatically generated by the application

  • Actions: Function icons for duplicating or deleting an entry

Configuration of data records

The view is divided into the following areas:

Overview of data set configuration

  • Area 1: Configurations

  • Area 2: Preview

Configurations

In the configurations, there are three different sub-areas in which settings can be made:

The button shows whether all data and Parameters entered are permitted and valid.
If incorrect entries are made, the button turns orange. The missing or incorrect entries are also displayed when the mouse pointer is moved over them.

General settings

This group contains basic configurations of a data record.

Name

The name of the data record.
Possible entries are all characters from a-z, numbers from 0-9 and special characters.

Data source

Source from which the data is retrieved.
The selection is made from the drop-down list.
Depending on the data source, specific configurations must be set (see also Specific settings).

The following sources are currently supported:

  • OpenSearch: Data from the OpenSearch instance that serves as the BPC’s database

  • JSON: Static data set used for demonstration purposes

  • HTTP: JSON data from an external source that can be retrieved via a URL

  • INUBIT: Reports generated in INUBIT about your own performance (report data)

The data source preselected when adding a new data set is the OpenSearch that is delivered with the BPC.

Variables

Variables represent dynamic parameters when querying the data source. For example, you can limit the time periods to which the data relates.
The variables can be used when visualizing the Charts visualization and optionally in Reports for the user. The user can then adjust the variable(s) as required.

Currently, variables are only supported for OpenSearch as a data source.

A variable must be given at least one name. Furthermore, the variable should also be assigned a type. 5 variable types are currently available: Text, Boolean, number, list and date. The variable types and configuration options are shown in the following table.

Variable types
Type Possible configuration Description Example

text

Default value

If no value is set in the special context, this value is used.

"Test variable"

boolean

Default value

If no value is set in the special context, this value is used.

Checkbox with checkmark (true) or
without checkmark (false)

number

Default value

If no value is set in the special context, this value is used.

"10"

Value range

Only values in this range are valid.

Min: "1",
Max: "999"

list

Value list

Values that are offered for selection in the chart/report.
The individual values are separated by commas when they are entered.

"1,2,3,4,5"

Default value

A value from the value list.
If no value is set in the special context, this value is used.

"3"

date

Date format

Date values are converted to this format before they are sent to the data source.
See possible date formats.

"Y-m-d"

Default value

If no value is set in the special context, this value is used.
The date can be entered in the format "DD.MM.YYYY" or selected via the symbol.

"22.02.2022"

Value range

Only values in this range are valid.
Both values can be entered in the format "DD.MM.YYYY" or selected via the respective symbol.

Min: "01.01.2022",
Max: "31.12.2022"

Specific settings

These are special settings for a specific type of data source.
[#data_source]The following table provides an overview of the specific configuration options for the individual data sources:

Data source Specific setting Description Example

OpenSearch

Index

Name of the OpenSearch index containing the desired data.

"bpc-auditlog"

Query

OpenSearch query in JSON format, see also the official OpenSearch documentation.

The configured Variables can be used in this query.
Values in the JSON object can be set as placeholders {X}.
If a placeholder appears in the query, this value is replaced by the value of the variable (if this has been configured).

For storage space reasons, the result data does not contain any fields with binary data.

 
{"JSON_KEY" : "{NAME_DER_VARIABLE}"}

JSON

JSON Data

static data record in JSON format (usually object or array).

 
{
  "name" : "Max",
  "age" : 43
}

HTTP

URL

External URL address from which JSON data is retrieved.
By default, an HTTP GET request is sent by the BPC.

https://example.com/

INUBIT

INUBIT Connection

In Backend Connections preconfigured connection (type: http_proxy) to an inubit server.

Selection from the drop-down list,
A click on takes you to the corresponding administration area.

Report ID

ID of a report that was configured in INUBIT.

Selection from the drop-down list

Preview of data

Once the configuration is complete and the data set has been executed and saved, a preview of the data is offered. In this preview, a context variable that was created in the configurator can also be set or adjusted, see Variables. The context variable can be used to test the behavior of the data in relation to changed values of this variable.

Modification of the data

After retrieving and configuring the data, it may be that this data is not yet available in the appropriate format to generate charts from it. Depending on the desired chart type, data should be formatted accordingly. This task is performed when the data set is modified.

A modifier is a section of code that manipulates input data. The code section is usually executed in the backend and the result is sent back to the frontend.
Modifiers are executed in sandbox mode. This means that the modifiers do not have access to critical resources (e.g. file system, network, etc.).

The view is divided into the following areas:

Overview of data set modification

  • Area 1: Modifier steps

  • Area 2: Configuration

  • Area 3: Preview

Modifier steps

This area lists all the individual steps created for the modification.
Here you have the option of creating, deleting or executing a modifier or executing all modifiers.

The order of the modifier steps can be changed using drag & drop.

Icon/button Description Function

Add function

Add

Creates a new modifier step

Execute up to this step

Executes all previous (above) modifier steps, including the selected step.

Delete

Removes the selected modifier step

Execute all

Execute all

Executes all existing modifier steps one after the other (from top to bottom).

Configuration

This area contains basic configurations of a modifier.

Name

The name of the modifier.

Type

Type of modifier.
The following types are currently supported:

  • Javascriptdata of the OpenSearch instance, which serves as the database of the BPC

  • Groovy: static data set used for demonstration purposes

The type preselected when adding a new modifier is Groovy.

Description

Additional information about the modifier.

Modifier

Modification rules as source code.
Depending on the type, either Javascript code or Groovy code must be used.

Modifier type Javascript

With this type, a modifier is defined as Javascript source code.
The entry point of this modifier is the method: modify(data, vars). This can also be understood as a main method.
The return value of this method provides the input data for the next modifier step or for a Chart.

The method has two Parameters:

data

Input data.
Input data is usually the result of the last modifier. If the current modifier is the first, the raw data of the current data set is used.

vars

When accessing this object, you have access to the values of the data setVariables that were created during the data set configuration.

Code example
function modify (data, vars) {
   const result = helper(data);
   return result;
}

function helper (data) {
   return data;
}

Modifier type Groovy

Only the following libraries are integrated for Groovy execution:
groovy (groovy-core), groovy-datetime, groovy-json

With this type, a modifier is defined as Groovy source code.
In contrast to the Javascript modifier, there is no entry method here. The modifier rules can be defined directly in the code block.
The return value of this method provides the input data for the next modifier step or for a Chart.

Two objects are provided in this code section:

data

Input data.
Input data is usually the result of the last modifier. If the current modifier is the first, the raw data of the current data set is used.

vars

When accessing this object, you have access to the values of the data setVariables that were created during the data set configuration.

Code example
data.collect { el -> return el * el }
return data

Application examples

In the following examples, data sets are created and configured that originate from different data sources. Two different modifier types are also configured.

Create and configure data set

  1. Select the data set editor in the Analysis module.

  2. Add a new data set.
    Add data source

  3. Enter a name.

  4. select "OpenSearch" as the data source.

  5. Enter a description if necessary.

  6. select "Index".

    Elasticsearch as data source

  7. Enter query.

  8. Add variable(s) if necessary.

    1. In the Variables area, click on the Add button.

    2. Enter the variable name.

    3. Select variable type (see Variables).

    4. Enter default values if necessary

    5. Enter minimum and maximum values if necessary.

      Add variable

    6. Click the Save button.

  9. Click on Execute.
    The data entered is automatically saved when the program is executed.
    The raw data is displayed in the preview window.

    Elasticsearch raw data

In the preview window, the variables can be expanded and adjusted for testing purposes (see Preview of data).

If you make changes to the configurations, always click on the Save button at the end!

Modify data set

As the various Chart types require specific values, the data sets should be modified accordingly:

Chart type Expected attribute value

Gauge (pointer diagram)

number

Pie (pie chart)

string for the Labels,
number for the Y-axis

XY chart (line and bar charts)

string for the text axis,
number for the numerical axis,
JavaScript timestamp for the date axis

Edit/remove variable

Variables that have already been created can also be edited or deleted later.

Edit or remove variables
Keywords:
Press Esc to exit full screen
Modal image placeholder
Press Esc to exit full screen