Log Service
This module offers, among other things, a REST endpoint to receive monitor data and write it to OpenSearch and optionally to a database. The relevant monitor "clients" are immediately informed of new data via a websocket event.
|
If you also want to write to a database, the database tables must already be created. These are not created automatically and are not automatically adjusted if the data structure changes. |
|
If you write to a database, the database data is written first via a DB transaction (this is normally more error-prone) and only then the data is written to OpenSearch. When writing to the database, a DB update is performed first (see keys setting) and if this fails, a DB insert is performed. |
|
As mentioned above, the Log Service data is usually displayed via the monitor.
This uses a process index (= parent data) and a history index (= child data). |
Configuration parameters Log Service module
Mandatory parameters are highlighted in bold.
| Setting (Key) | Group | Description |
|---|---|---|
Security_ClientCertificateAuthMandatory |
security |
Specifies whether the log service endpoints can only be called via Client Certificate Authentication. |
OpenSearch_DataCountLimit |
opensearch |
Maximum number of documents that can be queried. |
OpenSearch_DataViewLimit |
opensearch |
Up to this value, OpenSearch should return exact information about the number of documents (trackTotalHitsUpTo). |
Configuration parameters per Log Service component
Values highlighted in bold are mandatory parameters.
| Setting (Key) | Group | Description | ||
|---|---|---|---|---|
MaintenanceEnabled |
config |
If something needs to be adjusted in the database or the OpenSearch index, the endpoint for this instance can be set to maintenance mode.
In this case, the caller of the endpoint receives an HTTP status 503 (Service Unavailable). |
||
Keys |
config |
The key fields are defined here. When writing to the database, these are used for the DB update. For OpenSearch to define the document IDs. If several fields are affected, they must be separated from each other by commas. Default:
|
||
Fields |
config |
Among other things, fine adjustments can be made to the data types here (mapping for the OpenSearch indexes). Default
Example
By default, all fields are written to OpenSearch and the database. If individual fields are not to be written, they can be specified here.
true = the field is not written to OpenSearch false = the field is written to OpenSearch (default)
true = the field is not written to the database false = the field is written to the database (default) |
||
OpenSearch_LoggingEnabled |
opensearch |
Only if activated, the data is written to OpenSearch. |
||
OpenSearch_Indices |
opensearch |
Definition of which indexes the parent/child data should be written to. The indexes are created automatically. The defined 'Fields' types are taken into account.
Optional:
Example:
|
||
Joins |
config |
Can be used to automatically enrich the logging documents with additional data. For example, if there is only a partner ID in the data to be logged and the name of the partner etc. is still required in the monitor. Is the same configuration as for the 'Lookup Joins' of the Replication service. Example:
|
||
DB_RDMSLoggingEnabled |
rdms |
The data is only written to the database if activated. |
||
DB_RDMS |
rdms |
Definition of which database tables the parent/child data should be written to. The database tables must exist and are not created automatically.
Example:
|
Endpoints
To use the endpoints, a logged-in user is required logged in user or API key with the respective roles/rights is required to use the endpoints.
As additional protection, the "Log Service" module setting Security_ClientCertificateAuthMandatory can be used to activate Client Certificate Authentication for all Log Service endpoints.
| Method | Endpoint | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
|
||||||||||
Description Get a compact overview of the available log service instances. Only the following data is returned for each instance:
|
||||||||||
Returns The log service instances as JSON. HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Get the current configuration of the log service instance. |
||||||||||
Path Parameter
|
||||||||||
Returns The requested config as JSON. HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Gets the data of a log service instance. |
||||||||||
Path Parameter
|
||||||||||
Query Parameter
|
||||||||||
Returns The requested data as JSON. HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Get the data of a log service entry with its child entries. |
||||||||||
Path Parameter
|
||||||||||
Query Parameter
|
||||||||||
Returns The requested data as JSON. HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Get the data of a log service child entry. |
||||||||||
Path Parameter
|
||||||||||
Returns The requested data as JSON. HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Writes the data provided in the body to OpenSearch and/or the database. For calls from Iguasu the following HTTP headers are used and their values written to the following 'externalReference' object fields of all 'parent' related documents:
|
||||||||||
Consumes
|
||||||||||
Path Parameter
|
||||||||||
Query Parameter
|
||||||||||
Returns HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Deletes log service entries and their child entries from OpenSearch and the database. Attention: When you use 'childQuery' and/or 'childFilter' only those children get deleted, not the parent itself. |
||||||||||
Path Parameter
|
||||||||||
Query Parameter
|
||||||||||
Returns HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Deletes log service child entries from OpenSearch and the database. |
||||||||||
Path Parameter
|
||||||||||
Returns HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Deletes log service entries and their child entries from OpenSearch by query. Deletion from database is not supported. |
||||||||||
Consumes
|
||||||||||
Path Parameter
|
||||||||||
Query Parameter
|
||||||||||
Returns HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Deletes only log service child entries from OpenSearch by query. Deletion from database is not supported. |
||||||||||
Consumes
|
||||||||||
Path Parameter
|
||||||||||
Query Parameter
|
||||||||||
Returns HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Deletes/drops the parent and child indices of a log service instance. |
||||||||||
Path Parameter
|
||||||||||
Returns HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights The logged in user or API Key must have either the following role or right.
|
||||||||||
|
||||||||||
Description Creates a BPC deeplink to redirect the caller to the admin page of a log service instance. |
||||||||||
Path Parameter
|
||||||||||
Returns The requested data as JSON. HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights Can be used without a user session. |
||||||||||
|
||||||||||
Description Redirects the user to the monitor using the OpenSearch indices of the log service instance. All provided query params are used to build a monitor filter on fields of the 'externalReference' object. For Iguasu the following query parameters can be used to access the HTTP header values when the entry has been created.
|
||||||||||
Path Parameter
|
||||||||||
Query Parameter
|
||||||||||
Returns The requested data as JSON. HTTP Status Code
Content-Type
|
||||||||||
Required Access Rights Can be used without a user session. |
Response in the event of an error
For every error (status != 200), a JSON with the following exemplary structure is returned:
{
"error": {
"code": 301,
"name": "MODULE_INSTANCE_NOT_FOUND",
"message": "Did not found the instance 'of_test2' of the module 'logservice'.",
"properties": {
"instanceId": "of_test2",
"moduleId": "logservice"
}
}
}
The possible error codes
Error Code |
|
Info |
1 |
|
should not occur |
20 |
|
the 'Log Service' module was not found |
21 |
|
the 'Log Service' instance was not found |
10000 |
|
No data was transferred for "logging" |
10010 |
|
The transferred data is not valid JSON or does not correspond to the structure specified here do not correspond to the structure specified here |
20000 |
|
The access data does not match the stored data (Log Service module setting) |
20001 |
|
The Log Service is currently in maintenance mode or the DB and OpenSearch module is not available. the DB and OpenSearchLogging are deactivated |
20002 |
|
A Service required by the Log Service is not available |
20100 |
|
An error has occurred during DB logging |
20101 |
|
An error occurred while deleting DB entries |
20102 |
|
An invalid table name was specified |
20103 |
|
The database table used could not be found 20102 An error occurred while deleting DB entriesTable could not be found |
20104 |
|
The database column to be used does not existColumn does not exist |
20105 |
|
The database metadata could not be read |
20106 |
|
The requested functionality is not available for relational databases relational databases |
20201 |
|
An error occurred while accessing OpenSearch |
20202 |
|
The data required to form the document key is missing in the databaseKey, the required data is missing in the document to be logged |
20203 |
|
There was a problem writing the data to OpenSearch |
20204 |
|
An error occurred while deleting documents from the OpenSearch index |
20205 |
|
More data was requested than is available |
POST : Structure of the JSON message to be transferred
Create/update entry
{
"entries": [
{
"parent": {
"keyname_parent": keywertparent1,
"feldname1": wert1,
"feldname2": wert2,
...
},
"childs": [
{
"keyname_parent": keywertparent1,
"keyname_child": keywertchild1,
"feldname1": wert1,
"feldname2": wert2,
"datei_binary": base64_3,
...
},
{
"keyname_parent": keywertparent1,
"keyname_child": keywertchild2,
"feldname1": wert3,
"feldname2": wert4,
"datei_binary": base64_5,
...
},
...
]
},
...
]
}
Update individual fields (partial update)
{
"entries": [
{
"partialUpdate": true,
"parent": {
"keyname_parent": keywertparent1,
"feldname1": wert1,
"feldname2": wert2
},
...
},
...
]
}
Information on the partial update
The partial update relates to the parent entry and the child entries. This means that if you want to update a parent entry, you should not create a "new" child log in the same JSON document. If you want to create a new child log when updating a parent entry, it should be implemented in two log service calls. For example, it could look like this (see INFOBLOCK 1 and 2):
In the following example, the fields "INQUIRY_NUMBER" correspond to "keyname_parent" and "CHILDID" to "keyname_child" from before.
Values of the JSON fields
| Type | Value range |
|---|---|
Boolean |
true / false without quotation marks |
Date |
as ISO-8601 with quotation marks, e.g. "2017-05-17T15:28:23.181Z".
In the database, create the field as |
Numbers |
Value without quotation marks |
Binary |
Base64 encoded.
In the database as |
Text |
Value with quotation marks |
Field name convention
The data types per field can be defined via the module instance setting fields (see below).
This is not absolutely necessary for binary types and can also be done using the field name convention FIELDNAME_binary (_binary as postfix).
Background: In OpenSearch we store binary data Base64 encoded (so they must also be in the transferred JSON). However, as with replication, we want to store these under the special type 'attachment'. Since both are strings for OpenSearch, we need to know this and perform a special mapping.
Example: If we have a field with the name 'file' and can freely assign the names, then it can become the name 'file_binary' and OpenSearch stores the data as 'attachment' or creates the mapping for it.
Example call
curl -H "X-ApiKey: 1f697af5-c147-3d94-c529-e06f3f15bb87" \
-H "Content-Type: application/json" \
-XPOST 'localhost:8181/cxf/bpc-logservice/log/of_test' \
-d @logservice.json
Explanation:
| Option | Description |
|---|---|
|
The API key with the appropriate role or right |
|
The content type must be set to |
|
HTTP POST is required |
|
|
|
The JSON to be posted (see below) |
logservice.json (create/update entry)
{
"entries": [
{
"parent": {
"processid": 40,
"name": "hello world",
"city": "Berlin",
"lastupdate": "2017-05-17T15:28:23.181Z"
},
"childs": [
{
"processid": 40,
"childid": 1,
"ersteller": "Oliver",
"kommentar": "Bild und langer Text",
"datei": "iVBORw0KGgoAAAANSUhEUgAAAAIAAAACAQMAAABIeJ9nAAAABlBMVEUAAAD///+l2Z/dAAAADElEQVQIHWNwYGgAAAFEAMGoX3f9AAAAAElFTkSuQmCC",
"langertext": "Dieser Text ist aber ziemlich .................... lang. :-)",
"lastupdate": "2017-05-17T15:28:23.181Z"
}
]
}
]
}
