BPC connection tutorial

Introduction

The Virtimo Business Process Center (BPC) is an application framework for creating web-based specialist applications and portal solutions.

The BPC and IGUASU work together seamlessly and are fully integrated. On the IGUASU side, there are several Processors and Services that enable a connection.

In order to focus on IGUASU and avoid extensive further configurations in the BPC, the required BPC elements that can be imported are listed below: BPC-Export.json.

A preconfigured process monitor can be used for the overview of the created data and for the process starter. A ProcessLogger, which is also preconfigured in the export file, is required for forwarding the data from IGUASU back to the BPC. There is also a Forms module in the export, which can be used for the input in the BPC.

An example of the completed tutorial can also be downloaded using the following link: BPC-Tutorial.json.

The individual steps for linking a BPC instance and an example flow are described below.

Basic configuration

Setting up the necessary Services

Two Services are required to enable a connection to the BPC from IGUASU.

The HybridRESTServerController is used to address IGUASU from the BPC and thus process messages from the BPC in flows (see Configuration of the ListenBPCFlowStarter Processor).
The HybridRESTClientController is used to send messages to the BPC (see Forwarding messages from the IGUASU to the BPC)

Configuration of the HybridRESTServerController service

The HybridRESTServerController acts as a REST endpoint with which the IGUASU can be addressed from the BPC. To do this, the service must first be created and configured in the corresponding Process Group.

A listening port and authentication details are required for configuration. For example, 9010 can be entered as the Port, while the authentication username and password can be selected as desired. Furthermore, the SSL Context Service should be set to the Virtimo preconfigured SSL service for internal and incomming HTTP requests.

The selected details must then be stored in the BPC to enable the connection to the IGUASU. Instructions for configuring the connection in the BPC can be found in the BPC documentation.

Configuration of the HybridRESTClientController Service

The HybridRESTClientController is used to address the BPC via the REST interface and is required for all BPC Processors. To do this, the service must first be created and configured in the corresponding Process Group.

The BPC URL must first be entered for configuration. If IGUASU and the BPC are located in the same cloud environment, the URL is https://bpc-karaf:8282 by default. In addition, the API key created in the BPC instance must be entered API key must be stored.

The API key created must be assigned the appropriate rights in order to be able to make the various API calls. The required rights are mentioned below for the corresponding calls.

To enable secure communication, a SSL Context Service should also be selected. If none is stored, this must be taken into account for the protocol in the URL stored in the BPC and the target URL must begin with http://.

The other setting options would also make it possible to define timeout criteria or options for authentication. However, these settings are not required for the tutorial and can therefore be left at the default value.

Configuration in the BPC

The BPC must also be configured so that BPC and IGUASU can exchange messages.

API Key

In order to establish a connection from the IGUASU to the BPC, a corresponding API key must first be created in the configuration area (Core Services/API Keys) of the BPC. This API key must have the corresponding roles and rights. Here is a selection of relevant roles and rights and their meaning (please refer to the BPC documentation for more information):

Roles:

LOG_SERVICE_USER: Enables the use of log services (see:).
NOTIFICATION_ADMIN: Enables the sending of notifications.

Rights:

LOG_SERVICE_WRITE_DATA: Enables writing to a log service (required for process monitoring).
LOG_SERVICE_READ_DATA: Enables the reading of entries from a log service.
: Required for process monitoring LOG_SERVICE_CONFIG_GET_INSTANCES is required to obtain a list of available log services in the IGUASU.
NOTIFICATION_ADD: Is required to be able to trigger notifications in the BPC from the IGUASU.
AUDIT_LOG_CREATE_ENTRY: Is required to be able to write audit logs.

Backend Connection

A corresponding Backend Connection must be created in the BPC so that actions can be triggered from the BPC in the IGUASU via the Process starter. To do this, create a new Backend Connection of the type http_proxy in the BPC settings under Backen Connections. The following properties must be set:

Connection_Username: the backend connection defined in the HybridRestServerController
Connection_Password: the user name set in the HybridRestServerController
Target_BaseURL: the URL under which the IGUASU system can be reached (typically https://iguasu-nifi, including the password set in the HybridRestServerController set in the HybridRestServerController.
In this case, e.g. https://iguasu-nifi:9010/
Connection_CheckCsrfToken: To receive messages in the IGUASU via the Forms module in the BPC, this option must not (!) be set.

Log service

If data is to be sent to BPC Process Monitoring, a corresponding log service must be created in BPC. Please refer to the BPC log service documentation for the configuration of the log service.

Receive BPC input in IGUASU

There are two ways to send input from BPC to IGUASU.

Via BPC Forms.
Via Process Actions in Process Monitoring.

For both options, the ListenBPCFlowStarter Processor is required to receive the messages in IGUASU.

Configuration of the ListenBPCFlowStarter Processor

In order to be able to process data from the BPC instance, it must first be transferred to IGUASU.
The ListenBPCFlowStarter Processor acts as an interface for receiving and further processing the transmitted information. The Processor is used as the starting point of a flow.

To use the Processor, some properties must first be set:

Listener Identifier: a unique identifier that can be used to address the Listener.
BPC Listener Controller: a hybrid BPC Listener Controller BPC Listener Controller: A HybridRESTServerController that is addressed from the BPC to enable the connection.
BPC Controller: A HybridRESTClientController that is used to establish the connection to the BPC.
Name: The display name of the endpoint.
Description: A description of the endpoint Description: A description of the endpoint.
Load the BPC Session: If this option is selected, the BPC session is attached to the FlowFile when the process is triggered.

In this example, it should be possible to enter customer data in the BPC, which is processed in IGUASU and transmitted back to the BPC to provide an overview of the customers created. We therefore select distinct information for the configuration that distinguishes the listener with the planned function from other future listeners. Information on the BPC session in which the data was created should also be processed. The option Load the BPC Session must therefore be selected.

BPC Forms

For the planned implementation with the Forms module in the context of this tutorial, the standard form, which is automatically available when a new form is created, is sufficient.

To enable a successful connection, the submitURL must be adapted so that the interaction with the submit button sends the data to the desired Processor. The submit url must point to the previously created HttpProxy backend connection this is done by calling the http-proxy API. In this case, the corresponding submitUrl is therefore:

/bpc/cxf/bpc-httpproxy/httpProxy/iguasu/submit-customer

After completing the adjustments and entering the SubmitURL, which contains the Flow ID and Processor ID, the configuration of the desired form is complete.

When the configuration of all elements has been successfully completed, the input is successfully submitted to IGUASU by pressing the Formular abschicken button. This process can be checked by routing the success relation of the ListenBPCFlowStarter into a Funnels.

Process starter

The second option for forwarding data to IGUASU, which is described in this tutorial, is input via a process starter. To do this, a corresponding process starter must first be configured in the BPC. This is done according to the process starter instructions in the BPC documentation. The following attributes must be set in the process monitoring configuration in the BPC:

Function_InubitBackendConnection: The previously created backend connection must be set here Backend Connection must be set here.
Function_InubitBaseURL: Must be set to the value "/".
Function_ProcessStarterEndpoint: The ID of the corresponding ListenBPCFlowStarter that is to be addressed must be set here.

In this case, a second ListenBPCFlowStarter is created and configured with a different identifier as described above.

Once the ListenBPCFlowStarter Processor and the Processor starter have been configured, the transmission of the data can also be tested. The data should then be visible in IGUASU as shown in the following figure.

Process Actions

BPC Process Monitoring can also be used to trigger entry-specific process actions and thus send messages to IGUASU. In order for these actions to be received in IGUASU, some attributes must be set in Process Monitoring:

Function_InubitBackendConnection: The previously created Backend Connection must be set here.
Function_InubitBaseURL: Must be set to the value "/".
Function_ProcessActionsEndpoint: The ID of the corresponding ListenBPCFlowStarter that is to be addressed must be set here.

Processing the BPC data

Once the data has been successfully received in the IGUASU, it can now be processed.

Processing the BPC Forms data

A user-defined JSON structure in which additional information is entered is to be created as a result of saving the input. As the content of the Forms FlowFiles received in IGUASU are also already in JSON format, the JsonataTransformJson processor is a good choice.

The JsonataTransformJson processor can be used to customize both the attributes and the content of a FlowFile.
As the content is to be changed, Content is selected as the input data. In addition, the JSONata Transformation specification must be filled in to finally edit the Processor. The editor column is available in the configuration area for this purpose, in which the desired structure can be stored.

As the data is to be entered in the BPC via a process logger later in the tutorial, the corresponding structure must be taken into account for the JSON content. The basic structure provides for the information of the parent and child elements to be subdivided.

For this example, the general information about the customers should be stored in the parent element and possible processing steps, which will be adapted in future iterations, in the child element. If the JSON structure of the Forms input is taken into account, this results in the following structure for the planned JSON data:

{
	"entries": [
		{
		"parent": {
			"customerid": $nfGetAttribute('filename'),
			"fullname": form.state.data.firstName & " " & form.state.data.lastName,
			"address": form.state.data.address.street & ", " & form.state.data.address.postalCode & " " & form.state.data.address.city,
			"language": "German",
			"created": $now(),
			"updated": $now()
		},
		"childs": [
			{
			"customerid": $nfGetAttribute('filename'),
			"orderid": $nfGetAttribute('filename'),
			"created": $now(),
			"order": "",
			"status": "Neuer Kunde"
			}
		]
		}
	]
}

The existing information can be re-sorted in the JSON structure to create a new object. data.firstName & data.lastName For example, the originally distinct first and last name details can be combined to form fullname. It is also possible to access attributes and integrate them into the JSON structure. This is used here for the attribute filename in order to adopt individual IDs, as the file name in IGUASU is always unique. It is also possible to call functions such as $now(), which in this case is used for the creation date.

This step converts the data of the BPC form for the process logger.
The same procedure must now also be carried out for the process starter.

Processing the process starter data

The process starter data must first be adapted, as it is not transmitted to IGUASU as JSON. The data received is in URL-encoded XML format and is introduced with the addition request=, which must also be removed.

The ReplaceText Processor can be used to remove the addition. request= is entered as the Search Value and the Replacement Value can be left empty.

This leaves the URL-encoded XML file.

The NiFi Expression Language function urlDecode() can be used for decoding, but this can only be applied to attributes. The content must therefore first be converted into an attribute in order to use the function afterwards.
For this purpose, the ExtractText Processor can be used to save the content in an attribute. For this purpose, a dynamic property to define the name for the attribute (for this example content) and the value. As the entire content of the content is to be transferred, the value (?s)(^.*$) is inserted.

A second ReplaceText Processor can now be used to replace the encoded content with the decoded content. As everything can be replaced, Always Replace is selected as the Replacement Strategy and Entire Text is selected as the Evaluation Mode.

The Expression Language can be used to use an attribute as content and perform decoding at the same time. In the Replacement Value, the decoded value of the attribute can be transferred by entering ${content:urlDecode()}. With the completed configuration, the Processor should look as follows:

Finally, the XML file must be converted into JSON to enable subsequent processing. The ConvertRecord Processor is used for this purpose.

A Record Reader and a Record Writer are required as a Service for the configuration, with which the desired formats can be read and created. As the Processor is to convert XML files into JSON format, the XMLReader service is required as a reader and the JSONRecordSetWriter service as a writer. Both Services can be activated without further configuration and used with the ConvertRecord Processor.

As a final step for processing, the JSON structure must be adapted, similar to the processing of the BPC forms data, so that the data can be forwarded to the process monitor. The JSONataTransformJSON Processor is used again for this, whereby the following data structure can be used:

{
	"entries": [
		{
		"parent": {
			"customerid": $nfGetAttribute('filename'),
			"fullname": data.item.firstName & " " & data.item.lastName,
			"address": data.item.street & ", " & data.item.postalCode & " " & data.item.city,
			"language": "German",
			"created": $now(),
			"updated": $now()
		},
		"childs": [
			{
			"customerid": $nfGetAttribute('filename'),
			"orderid": $nfGetAttribute('filename'),
			"created": $now(),
			"order": "",
			"status": "Neuer Kunde"
			}
		]
		}
	]
}

With this final step, both the data from the BPC forms and from the process starter are pre-processed for forwarding to the BPC. The flow created should look like this at this point:

Forwarding messages from the IGUASU to the BPC

Messages and data can be sent from the IGUASU to the BPC using the following methods:

Process-relevant data and logs can be sent to a log service using the PutBPCProcessLog Processors Log service in the BPC and can be displayed there in Process Monitoring.
The PutBPCAuditLog Processor can be used for internal log data.
The PutBPCNotification Processor can be used to send smaller push messages to the BPC, e.g. to provide immediate feedback on running processes.

Process logs

In order to be able to display information from IGUASU in the BPC Process Monitor, it must be sent to a log service using the PutBPCProcessLog Processor Log service in the BPC.

In the configuration of the Processors, the previously configured HybridRESTClientController is selected, via which a connection to the BPC is established. Furthermore, a Log service should already have been created in the BPC.

If the controller and the log service are configured correctly, the loggers created in the BPC are available at BPC Logger.

The Input Type property can also be used to specify which data is to be sent to the BPC. The BPC Entries JSON property could be used for this purpose, or the content of a FlowFile. As the generated data in this tutorial is contained in the FlowFile content, the latter is used for the configuration.

The JSON structure of the logger specified by the BPC must be adhered to in IGUASU so that the data can be successfully transmitted.

Once the configuration has been successfully completed, the data should be displayed in the BPC’s Process Monitor after the Processor has been executed. An example of the completed configuration can be seen in the following figure.

Audit logs

In order to provide a better overview of the customers created, it should be documented when a new customer has been generated. Audit logs can be used for this purpose, which generate a message in the Audit Monitor of the BPC.

The PutBPCAuditLogs processor can be used to create audit logs.
The HybridRESTClientController Service must also be defined first for configuration in order to enable communication with the BPC.

The API key of the BPC Controller requires the AUDIT_LOG_CREATE_ENTRY right in order to be able to write audit logs.

The log level of the generated messages must also be specified. You can choose from DEBUG, INFO, WARNING and ERROR.

To provide a better overview of the generated data, a INFO message should first be added when a user creates a new customer. The BPC Audit Originator can be defined so that it is also clear which user created the customer.

The fact that a new customer was created during the configuration of the Configuration of the ListenBPCFlowStarter the option Load the BPC Session was selected, this information is already in the bpc.session attribute of the FlowFile. The information on the user can be retrieved via a JSONPath using the NiFi Expression Language. ${bpc.session:jsonPath('$.loginName')} is therefore noted as the originator at this point so that the user can be determined dynamically.

Additional information that should be included in the message can then be defined under the points BPC Action and BPC Audit Description. An exemplary configuration is shown in the following figure.

It would also make sense to note the failure of the process logger in the audit logs.
A second PutBPCAuditLogs Processor can be created for this purpose, which is linked to the PutBPCProcessLogger Processor with the failure relation. WARNING or ERROR can also be selected as the audit level to additionally indicate the failure.

Additional information on the BPC Audit Logger can also be found in the BPC documentation.

BPC Notifications

The PutBPCNotification-Processor can be used to send a notification to the BPC.
For configuration, the BPC Controller Service must also be defined first in order to enable communication with the BPC.

The Processor can then be configured according to the use case. The following configuration options are available:

Priority: You can determine whether the notification is silent in the BPC, or whether it is displayed as a toast (push message) or as a pop-up.
Subject and Message: This allows you to define the priority of the notification Subject and Message: This allows you to specify the title and content of the notification.
Recipients Type and Recipients: This allows you to specify who receives the notification in the BPC.
Notification Type: This allows you to specify the type or severity of the notification.
For example, Info, Warning or Error can be selected.

ListenBPCResponder

To successfully complete a request from the BPC, a response must be sent back from the IGUASU. Similar to the HandleHTTPResponse Processor, a HTTP Status Code and a Response Header can be configured.

If two PutBPCAuditLog Processors have previously been created, the success or failure can also be communicated to the BPC via the ListenBPCResponder.
The HTTP status 200 can be used for success, while 400 is selected for failure. A short description can also be added via the header.

The BPC tutorial is completed with successful configuration and enables customer data to be entered in the BPC via forms or the process starter so that this is documented in the process monitor. The finished data flow could look as shown in the following figure.

Log IGUASU data to BPC with the BPCRecordSink service

To send large amounts of data that are in a flat structure (e.g. in JSON, CSV or XML format) in the content of a FlowFile to BPC, it may be better to use the BPCRecordSink service instead of the PutBPCProcessLog Processors. To do this, you can configure a PutRecord-Processor so that the content of the FlowFiles is read out as records with any record reader service (depending on the data format) and the records are sent with a BPCRecordSink. The BPCRecordSink uses the values and schema of the incoming records to automatically prepare or forward the corresponding requests for BPC.

While the PutBPCProcessLog Processor can send any child elements in its data, the BPCRecordSink service only sends each record as a parent element.

A BPCRecordSink can also be used by a QueryNiFiReportingTask. QueryNiFiReportingTasks reads status information from IGUASU using SQL queries, which can then be forwarded to a RecordSink as records. This allows status information to be sent directly to BPC.

QueryNifiReportingTasks can be created under "Reporting Tasks" in the Management menu.

Configuration of the BPCRecordSink Service

Like the PutBPCProcessLog, the BPCRecordSink Service is configured using a BPC Controller to establish a connection with the BPC and by selecting a BPC Logger.

Unlike the PutBPCProcessLog Processor, the data to be sent does not have to be converted into the appropriate JSON structure for the BPC Logging Service, but is converted automatically by the BPCRecordSink. Only the parent key must be specified for this, which must match the configurations of the BPC Logging Services.

BPC Log Service Parent Key Configuration

Illustration 1. For this BPC Log Service, the parent key for the BPCRecordService must be "PROCESSID", which corresponds to the default for both the BPC and the IGUASU.

It is also possible to assign a date and timestamp to each logged record in the BPC. To do this, the SYSDATE field name property in the BPCRecordSinkService must be configured with a corresponding field name, under which a timestamp is then also sent. The Fields Configuration in the BPC Log Service must then be configured with the timestamp type for the field.

Illustration 2. If the "SYSDATE field name" of the BPCRecordSink is "timeAndDate", this field should be added.