Components

Value to track

The expression on which to evaluate each FlowFile. The result of the expression will be added to the rolling window value.

Time window

The time window on which to calculate the rolling window.

Sub-window length

When set, values will be batched into sub-windows of the set length. This allows for much larger length total windows to be set but sacrifices some precision. If this is not set (or is 0) then each value is stored in state with the timestamp of when it was received. After the length of time stated in Time window elaspes the value will be removed. If this is set, values will be batched together every X amount of time (where X is the time period set for this property) and removed all at once.

Attribute List

Comma separated list of attributes to be included in the resulting CSV. If this value is left empty then all existing Attributes will be included. This list of attributes is case sensitive and supports attribute names that contain commas. If an attribute specified in the list is not found it will be emitted to the resulting CSV with an empty string or null depending on the 'Null Value' property. If a core attribute is specified in this list and the 'Include Core Attributes' property is false, the core attribute will be included. The attribute list ALWAYS wins.

Attributes Regular Expression

Regular expression that will be evaluated against the flow file attributes to select the matching attributes. This property can be used in combination with the attributes list property. The final output will contain a combination of matches found in the ATTRIBUTE_LIST and ATTRIBUTE_REGEX.

Destination

Control if CSV value is written as a new flowfile attribute 'CSVData' or written in the flowfile content.

Include Core Attributes

Determines if the FlowFile org.apache.nifi.flowfile.attributes.CoreAttributes, which are contained in every FlowFile, should be included in the final CSV value generated. Core attributes will be added to the end of the CSVData and CSVSchema strings. The Attribute List property overrides this setting.

Null Value

If true a non existing or empty attribute will be 'null' in the resulting CSV. If false an empty string will be placed in the CSV

Include Schema

If true the schema (attribute names) will also be converted to a CSV string which will either be applied to a new attribute named 'CSVSchema' or applied at the first row in the content depending on the DESTINATION property setting.

Attributes List

Comma separated list of attributes to be included in the resulting JSON. If this value is left empty then all existing Attributes will be included. This list of attributes is case sensitive. If an attribute specified in the list is not found it will be be emitted to the resulting JSON with an empty string or NULL value.

Attributes Regular Expression

Regular expression that will be evaluated against the flow file attributes to select the matching attributes. This property can be used in combination with the attributes list property.

Destination

Control if JSON value is written as a new flowfile attribute 'JSONAttributes' or written in the flowfile content. Writing to flowfile content will overwrite any existing flowfile content.

Include Core Attributes

Determines if the FlowFile org.apache.nifi.flowfile.attributes.CoreAttributes which are contained in every FlowFile should be included in the final JSON value generated.

Null Value

If true a non existing selected attribute will be NULL in the resulting JSON. If false an empty string will be placed in the JSON

JSON Handling Strategy

Strategy to use for handling attributes which contain nested JSON.

Pretty Print

Apply pretty print formatting to the output.

Records Per Split

Specifies how many records should be covered in each FlowFile

Zero Content Output

Whether to do, or do not copy the content of input FlowFile.

Zero Content Output

Whether to do, or do not copy the content of input FlowFile.

Record Reader

A record reader to use for reading the records.

record-stats-limit

Limit the number of individual stats that are returned for each record path to the top N results.

The name of the category. For example, sport

Specifies a category that should be counted. For example, if the property name is 'sport' and the value is '/sport', the processor will count how many records have a value of 'soccer' for the /sport field, how many have a value of 'baseball' for the /sport, and so on. These counts be added as attributes named recordStats.sport.soccer, recordStats.sport.baseball.

MySQL Nodes

A list of hostname (and optional port) entries corresponding to nodes in a MySQL cluster. The entries should be comma separated using a colon (if the port is to be specified) such as host1:port,host2:port,…​. For example mysql.myhost.com:3306. The port need not be specified, when omitted the default MySQL port value of 3306 will be used. This processor will attempt to connect to the hosts in the list in order. If one node goes down and failover is enabled for the cluster, then the processor will connect to the active node (assuming its node entry is specified in this property).

MySQL Driver Class Name

The class name of the MySQL database driver class

MySQL Driver Location(s)

Comma-separated list of files/folders and/or URLs containing the MySQL driver JAR and its dependencies (if any). For example '/var/tmp/mysql-connector-java-5.1.38-bin.jar'

Username

Username to access the MySQL cluster

Password

Password to access the MySQL cluster

Event Processing Strategy

Specifies the strategy to use when writing events to FlowFile(s), such as 'Max Events Per FlowFile'

Events Per FlowFile

Specifies how many events should be written to a single FlowFile. If the processor is stopped before the specified number of events has been written,the events will still be written as a FlowFile before stopping.

Server ID

The client connecting to the MySQL replication group is actually a simplified replica (server), and the Server ID value must be unique across the whole replication group (i.e. different from any other Server ID being used by any primary or replica). Thus, each instance of CaptureChangeMySQL must have a Server ID unique across the replication group. If the Server ID is not specified, it defaults to 65535.

Database/Schema Name Pattern

A regular expression (regex) for matching databases (or schemas, depending on your RDBMS' terminology) against the list of CDC events. The regex must match the database name as it is stored in the RDBMS. If the property is not set, the database name will not be used to filter the CDC events. NOTE: DDL events, even if they affect different databases, are associated with the database used by the session to execute the DDL. This means if a connection is made to one database, but the DDL is issued against another, then the connected database will be the one matched against the specified pattern.

Table Name Pattern

A regular expression (regex) for matching CDC events affecting matching tables. The regex must match the table name as it is stored in the database. If the property is not set, no events will be filtered based on table name.

Max Wait Time

The maximum amount of time allowed for a connection to be established, zero means there is effectively no limit.

Distributed Map Cache Client - unused

This is a legacy property that is no longer used to store table information, the processor will handle the table information (column names, types, etc.)

Retrieve All Records

Specifies whether to get all available CDC events, regardless of the current binlog filename and/or position. If binlog filename and position values are present in the processor’s State, this property’s value is ignored. This allows for 4 different configurations: 1) If binlog data is available in processor State, that is used to determine the start location and the value of Retrieve All Records is ignored. 2) If no binlog data is in processor State, then Retrieve All Records set to true means start at the beginning of the binlog history. 3) If no binlog data is in processor State and Initial Binlog Filename/Position are not set, then Retrieve All Records set to false means start at the end of the binlog history. 4) If no binlog data is in processor State and Initial Binlog Filename/Position are set, then Retrieve All Records set to false means start at the specified initial binlog file/position. To reset the behavior, clear the processor state (refer to the State Management section of the processor’s documentation).

Include Begin/Commit Events

Specifies whether to emit events corresponding to a BEGIN or COMMIT event in the binary log. Set to true if the BEGIN/COMMIT events are necessary in the downstream flow, otherwise set to false, which suppresses generation of these events and can increase flow performance.

Include DDL Events

Specifies whether to emit events corresponding to Data Definition Language (DDL) events such as ALTER TABLE, TRUNCATE TABLE, e.g. in the binary log. Set to true if the DDL events are desired/necessary in the downstream flow, otherwise set to false, which suppresses generation of these events and can increase flow performance.

Initial Sequence ID

Specifies an initial sequence identifier to use if this processor’s State does not have a current sequence identifier. If a sequence identifier is present in the processor’s State, this property is ignored. Sequence identifiers are monotonically increasing integers that record the order of flow files generated by the processor. They can be used with the EnforceOrder processor to guarantee ordered delivery of CDC events.

Initial Binlog Filename

Specifies an initial binlog filename to use if this processor’s State does not have a current binlog filename. If a filename is present in the processor’s State or "Use GTID" property is set to false, this property is ignored. This can be used along with Initial Binlog Position to "skip ahead" if previous events are not desired. Note that NiFi Expression Language is supported, but this property is evaluated when the processor is configured, so FlowFile attributes may not be used. Expression Language is supported to enable the use of the environment properties.

Initial Binlog Position

Specifies an initial offset into a binlog (specified by Initial Binlog Filename) to use if this processor’s State does not have a current binlog filename. If a filename is present in the processor’s State or "Use GTID" property is false, this property is ignored. This can be used along with Initial Binlog Filename to "skip ahead" if previous events are not desired. Note that NiFi Expression Language is supported, but this property is evaluated when the processor is configured, so FlowFile attributes may not be used. Expression Language is supported to enable the use of the environment properties.

Use Binlog GTID

Specifies whether to use Global Transaction ID (GTID) for binlog tracking. If set to true, processor’s state of binlog file name and position is ignored. The main benefit of using GTID is to have much reliable failover than using binlog filename/position.

Initial Binlog GTID

Specifies an initial GTID to use if this processor’s State does not have a current GTID. If a GTID is present in the processor’s State or "Use GTID" property is set to false, this property is ignored. This can be used to "skip ahead" if previous events are not desired. Note that NiFi Expression Language is supported, but this property is evaluated when the processor is configured, so FlowFile attributes may not be used. Expression Language is supported to enable the use of the environment properties.

SSL Mode

SSL Mode used when SSL Context Service configured supporting certificate verification options

SSL Context Service

SSL Context Service supporting encrypted socket communication

Chunking Strategy

Specifies which splitter should be used to split the text

Separator

Specifies the character sequence to use for splitting apart the text. If using a Chunking Strategy of Recursively Split by Character, it is a comma-separated list of character sequences. Meta-characters \n, \r and \t are automatically un-escaped.

Separator Format

Specifies how to interpret the value of the <Separator> property

Chunk Size

The maximum size of a chunk that should be returned

Chunk Overlap

The number of characters that should be overlapped between each chunk of text

Keep Separator

Whether or not to keep the text separator in each chunk of data

Strip Whitespace

Whether or not to strip the whitespace at the beginning and end of each chunk

Language

The language to use for the Code’s syntax

Mode

Indicates whether the processor should compress content or decompress content. Must be either 'compress' or 'decompress'

Compression Format

The compression format to use. Valid values are: GZIP, Deflate, ZSTD, BZIP2, XZ-LZMA2, LZMA, Brotli, Snappy, Snappy Hadoop, Snappy Framed, and LZ4-Framed

Compression Level

The compression level to use; this is valid only when using gzip, deflate or xz-lzma2 compression. A lower value results in faster processing but less compression; a value of 0 indicates no (that is, simple archiving) for gzip or minimal for xz-lzma2 compression. Higher levels can mean much larger memory usage such as the case with levels 7-9 for xz-lzma/2 so be careful relative to heap size.

Update Filename

If true, will remove the filename extension when decompressing data (only if the extension indicates the appropriate compression format) and add the appropriate extension when compressing data

WebSocket Client ControllerService

A WebSocket CLIENT Controller Service which can connect to a WebSocket server.

WebSocket Client Id

The client ID to identify WebSocket session. It should be unique within the WebSocket Client Controller Service. Otherwise, it throws WebSocketConfigurationException when it gets started.

Queue

The name of the existing AMQP Queue from which messages will be consumed. Usually pre-defined by AMQP administrator.

Auto-Acknowledge Messages

If false (Non-Auto-Acknowledge), the messages will be acknowledged by the processor after transferring the FlowFiles to success and committing the NiFi session. Non-Auto-Acknowledge mode provides 'at-least-once' delivery semantics. If true (Auto-Acknowledge), messages that are delivered to the AMQP Client will be auto-acknowledged by the AMQP Broker just after sending them out. This generally will provide better throughput but will also result in messages being lost upon restart/crash of the AMQP Broker, NiFi or the processor. Auto-Acknowledge mode provides 'at-most-once' delivery semantics and it is recommended only if loosing messages is acceptable.

Batch Size

The maximum number of messages that should be processed in a single session. Once this many messages have been received (or once no more messages are readily available), the messages received will be transferred to the 'success' relationship and the messages will be acknowledged to the AMQP Broker. Setting this value to a larger number could result in better performance, particularly for very small messages, but can also result in more messages being duplicated upon sudden restart of NiFi.

Prefetch Count

The maximum number of unacknowledged messages for the consumer. If consumer has this number of unacknowledged messages, AMQP broker will no longer send new messages until consumer acknowledges some of the messages already delivered to it.Allowed values: from 0 to 65535. 0 means no limit

Header Output Format

Defines how to output headers from the received message

Header Key Prefix

Text to be prefixed to header keys as the are added to the FlowFile attributes. Processor will append '.' to the value of this property

Header Separator

The character that is used to separate key-value for header in String. The value must be only one character.

Remove Curly Braces

If true Remove Curly Braces, Curly Braces in the header will be automatically remove.

Brokers

A comma-separated list of known AMQP Brokers in the format <host>:<port> (e.g., localhost:5672). If this is set, Host Name and Port are ignored. Only include hosts from the same AMQP cluster.

Host Name

Network address of AMQP broker (e.g., localhost). If Brokers is set, then this property is ignored.

Port

Numeric value identifying Port of AMQP broker (e.g., 5671). If Brokers is set, then this property is ignored.

Virtual Host

Virtual Host name which segregates AMQP system for enhanced security.

User Name

User Name used for authentication and authorization.

Password

Password used for authentication and authorization.

AMQP Version

AMQP Version. Currently only supports AMQP v0.9.1.

SSL Context Service

The SSL Context Service used to provide client certificate information for TLS/SSL connections.

Use Client Certificate Authentication

Authenticate using the SSL certificate rather than user name/password.

Event Hub Namespace

The namespace that the Azure Event Hubs is assigned to. This is generally equal to <Event Hub Names>-ns.

Event Hub Name

The name of the event hub to pull messages from.

Service Bus Endpoint

To support namespaces not in the default windows.net domain.

Transport Type

Advanced Message Queuing Protocol Transport Type for communication with Azure Event Hubs

Shared Access Policy Name

The name of the shared access policy. This policy must have Listen claims.

Shared Access Policy Key

The key of the shared access policy. Either the primary or the secondary key can be used.

Use Azure Managed Identity

Choose whether or not to use the managed identity of Azure VM/VMSS

Consumer Group

The name of the consumer group to use.

Record Reader

The Record Reader to use for reading received messages. The event hub name can be referred by Expression Language '${eventhub.name}' to access a schema.

Record Writer

The Record Writer to use for serializing Records to an output FlowFile. The event hub name can be referred by Expression Language '${eventhub.name}' to access a schema. If not specified, each message will create a FlowFile.

Initial Offset

Specify where to start receiving messages if offset is not yet stored in the checkpoint store.

Prefetch Count

Batch Size

The number of messages to process within a NiFi session. This parameter affects throughput and consistency. NiFi commits its session and Event Hubs checkpoints after processing this number of messages. If NiFi session is committed, but fails to create an Event Hubs checkpoint, then it is possible that the same messages will be received again. The higher number, the higher throughput, but possibly less consistent.

Message Receive Timeout

The amount of time this consumer should wait to receive the Batch Size before returning.

Checkpoint Strategy

Specifies which strategy to use for storing and retrieving partition ownership and checkpoint information for each partition.

Storage Account Name

Name of the Azure Storage account to store event hub consumer group state.

Storage Account Key

The Azure Storage account key to store event hub consumer group state.

Storage SAS Token

The Azure Storage SAS token to store Event Hub consumer group state. Always starts with a ? character.

Storage Container Name

Name of the Azure Storage container to store the event hub consumer group state. If not specified, event hub name is used.

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Range Query Field

Field to be tracked as part of an Elasticsearch Range query using a "gt" bound match. This field must exist within the Elasticsearch document for it to be retrieved.

Sort Order

The order in which to sort the "Range Query Field". A "sort" clause for the "Range Query Field" field will be prepended to any provided "Sort" clauses. If a "sort" clause already exists for the "Range Query Field" field, it will not be updated.

Initial Value

The initial value to use for the query if the processor has not run previously. If the processor has run previously and stored a value in its state, this property will be ignored. If no value is provided, and the processor has not previously run, no Range query bounds will be used, i.e. all documents will be retrieved in the specified "Sort Order".

Initial Value Date Format

If the "Range Query Field" is a Date field, convert the "Initial Value" to a date with this format. If not specified, Elasticsearch will use the date format provided by the "Range Query Field"'s mapping. For valid syntax, see https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-date-format.html

Initial Value Date Time Zone

If the "Range Query Field" is a Date field, convert the "Initial Value" to UTC with this time zone. Valid values are ISO 8601 UTC offsets, such as "+01:00" or "-08:00", and IANA time zone IDs, such as "Europe/London".

Additional Filters

One or more query filters in JSON syntax, not Lucene syntax. Ex: [{"match":{"somefield":"somevalue"}}, {"match":{"anotherfield":"anothervalue"}}]. These filters wil be used as part of a Bool query’s filter.

Size

The maximum number of documents to retrieve in the query. If the query is paginated, this "size" applies to each page of the query, not the "size" of the entire result set.

Sort

Sort results by one or more fields, in JSON syntax. Ex: [{"price" : {"order" : "asc", "mode" : "avg"}}, {"post_date" : {"format": "strict_date_optional_time_nanos"}}]

Aggregations

One or more query aggregations (or "aggs"), in JSON syntax. Ex: {"items": {"terms": {"field": "product", "size": 10}}}

Fields

Fields of indexed documents to be retrieved, in JSON syntax. Ex: ["user.id", "http.response.*", {"field": "@timestamp", "format": "epoch_millis"}]

Script Fields

Fields to created using script evaluation at query runtime, in JSON syntax. Ex: {"test1": {"script": {"lang": "painless", "source": "doc['price'].value * 2"}}, "test2": {"script": {"lang": "painless", "source": "doc['price'].value * params.factor", "params": {"factor": 2.0}}}}

Query Attribute

If set, the executed query will be set on each result flowfile in the specified attribute.

Index

The name of the index to use.

Type

The type of this document (used by Elasticsearch for indexing and searching).

Max JSON Field String Length

The maximum allowed length of a string value when parsing a JSON document or attribute.

Client Service

An Elasticsearch client service to use for running queries.

Search Results Split

Output a flowfile containing all hits or one flowfile for each individual hit or one flowfile containing all hits from all paged responses.

Search Results Format

Format of Hits output.

Aggregation Results Split

Output a flowfile containing all aggregations or one flowfile for each individual aggregation.

Aggregation Results Format

Format of Aggregation output.

Output No Hits

Output a "hits" flowfile even if no hits found for query. If true, an empty "hits" flowfile will be output even if "aggregations" are output.

Pagination Type

Pagination method to use. Not all types are available for all Elasticsearch versions, check the Elasticsearch docs to confirm which are applicable and recommended for your service.

Pagination Keep Alive

Pagination "keep_alive" period. Period Elasticsearch will keep the scroll/pit cursor alive in between requests (this is not the time expected for all pages to be returned, but the maximum allowed time for requests between page retrievals).

The name of a URL query parameter to add

Adds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request body. For SCROLL type queries, these parameters are only used in the initial (first page) query as the Elasticsearch Scroll API does not support the same query parameters for subsequent pages of data.

GCP Credentials Provider Service

The Controller Service used to obtain Google Cloud Platform credentials.

Project ID

Google Cloud Project ID

Subscription

Name of the Google Cloud Pub/Sub Subscription

Batch Size Threshold

Indicates the number of messages the cloud service should bundle together in a batch. If not set and left empty, only one message will be used in a batch

API Endpoint

Override the gRPC endpoint in the form of [host:port]

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Host Name

Network address of Email server (e.g., pop.gmail.com, imap.gmail.com . . .)

Port

Numeric value identifying Port of Email server (e.g., 993)

Authorization Mode

How to authorize sending email on the user’s behalf.

OAuth2 Access Token Provider

OAuth2 service that can provide access tokens.

User Name

User Name used for authentication and authorization with Email server.

Password

Password used for authentication and authorization with Email server.

Folder

Email folder to retrieve messages from (e.g., INBOX)

Fetch Size

Specify the maximum number of Messages to fetch per call to Email Server.

Delete Messages

Specify whether mail messages should be deleted after retrieval.

Connection timeout

The amount of time to wait to connect to Email server

Mark Messages as Read

Specify if messages should be marked as read after retrieval.

Use SSL

Specifies if IMAP connection must be obtained via SSL encrypted connection (i.e., IMAPS)

Connection Factory Service

The Controller Service that is used to obtain Connection Factory. Alternatively, the 'JNDI *' or the 'JMS *' properties can also be be used to configure the Connection Factory.

Destination Name

The name of the JMS Destination. Usually provided by the administrator (e.g., 'topic://myTopic' or 'myTopic').

Destination Type

The type of the JMS Destination. Could be one of 'QUEUE' or 'TOPIC'. Usually provided by the administrator. Defaults to 'QUEUE'

Message Selector

The JMS Message Selector to filter the messages that the processor will receive

User Name

User Name used for authentication and authorization.

Password

Password used for authentication and authorization.

Connection Client ID

The client id to be set on the connection, if set. For durable non shared consumer this is mandatory, for all others it is optional, typically with shared consumers it is undesirable to be set. Please see JMS spec for further details

Character Set

The name of the character set to use to construct or interpret TextMessages

Acknowledgement Mode

The JMS Acknowledgement Mode. Using Auto Acknowledge can cause messages to be lost on restart of NiFi but may provide better performance than Client Acknowledge.

Durable Subscription

If destination is Topic if present then make it the consumer durable. @see https://jakarta.ee/specifications/platform/9/apidocs/jakarta/jms/session#createDurableConsumer-jakarta.jms.Topic-java.lang.String-

Shared Subscription

If destination is Topic if present then make it the consumer shared. @see https://jakarta.ee/specifications/platform/9/apidocs/jakarta/jms/session#createSharedConsumer-jakarta.jms.Topic-java.lang.String-

Subscription Name

The name of the subscription to use if destination is Topic and is shared or durable.

Timeout

How long to wait to consume a message from the remote broker before giving up.

Maximum Batch Size

The maximum number of messages to publish or consume in each invocation of the processor.

Error Queue Name

The name of a JMS Queue where - if set - unprocessed messages will be routed. Usually provided by the administrator (e.g., 'queue://myErrorQueue' or 'myErrorQueue').Only applicable if 'Destination Type' is set to 'QUEUE'

Record Reader

The Record Reader to use for parsing received JMS Messages into Records.

Record Writer

The Record Writer to use for serializing Records before writing them to a FlowFile.

Output Strategy

The format used to output the JMS message into a FlowFile record.

JNDI Initial Context Factory Class

The fully qualified class name of the JNDI Initial Context Factory Class (java.naming.factory.initial).

JNDI Provider URL

The URL of the JNDI Provider to use as the value for java.naming.provider.url. See additional details documentation for allowed URL schemes.

JNDI Name of the Connection Factory

The name of the JNDI Object to lookup for the Connection Factory.

JNDI / JMS Client Libraries

Specifies jar files and/or directories to add to the ClassPath in order to load the JNDI / JMS client libraries. This should be a comma-separated list of files, directories, and/or URLs. If a directory is given, any files in that directory will be included, but subdirectories will not be included (i.e., it is not recursive).

JNDI Principal

The Principal to use when authenticating with JNDI (java.naming.security.principal).

JNDI Credentials

The Credentials to use when authenticating with JNDI (java.naming.security.credentials).

JMS Connection Factory Implementation Class

The fully qualified name of the JMS ConnectionFactory implementation class (eg. org.apache.activemq.ActiveMQConnectionFactory).

JMS Client Libraries

Path to the directory with additional resources (eg. JARs, configuration files etc.) to be added to the classpath (defined as a comma separated list of values). Such resources typically represent target JMS client libraries for the ConnectionFactory implementation.

JMS Broker URI

URI pointing to the network location of the JMS Message broker. Example for ActiveMQ: 'tcp://myhost:61616'. Examples for IBM MQ: 'myhost(1414)' and 'myhost01(1414),myhost02(1414)'.

JMS SSL Context Service

The SSL Context Service used to provide client certificate information for TLS/SSL connections.

The name of a Connection Factory configuration property.

Additional configuration property for the Connection Factory. It can be used when the Connection Factory is being configured via the 'JNDI *' or the 'JMS *'properties of the processor. For more information, see the Additional Details page.

Kafka Connection Service

Provides connections to Kafka Broker for publishing Kafka Records

Group ID

Kafka Consumer Group Identifier corresponding to Kafka group.id property

Topic Format

Specifies whether the Topics provided are a comma separated list of names or a single regular expression

Topics

The name or pattern of the Kafka Topics from which the Processor consumes Kafka Records. More than one can be supplied if comma separated.

Auto Offset Reset

Automatic offset configuration applied when no previous consumer offset found corresponding to Kafka auto.offset.reset property

Commit Offsets

Specifies whether this Processor should commit the offsets to Kafka after receiving messages. Typically, this value should be set to true so that messages that are received are not duplicated. However, in certain scenarios, we may want to avoid committing the offsets, that the data can be processed and later acknowledged by PublishKafka in order to provide Exactly Once semantics.

Max Uncommitted Time

Specifies the maximum amount of time allowed to pass before offsets must be committed. This value impacts how often offsets will be committed. Committing offsets less often increases throughput but also increases the window of potential data duplication in the event of a rebalance or JVM restart between commits. This value is also related to maximum poll records and the use of a message demarcator. When using a message demarcator we can have far more uncommitted messages than when we’re not as there is much less for us to keep track of in memory.

Header Name Pattern

Regular Expression Pattern applied to Kafka Record Header Names for selecting Header Values to be written as FlowFile attributes

Header Encoding

Character encoding applied when reading Kafka Record Header values and writing FlowFile attributes

Processing Strategy

Strategy for processing Kafka Records and writing serialized output to FlowFiles

Record Reader

The Record Reader to use for incoming Kafka messages

Record Writer

The Record Writer to use in order to serialize the outgoing FlowFiles

Output Strategy

The format used to output the Kafka Record into a FlowFile Record.

Key Attribute Encoding

Encoding for value of configured FlowFile attribute containing Kafka Record Key.

Key Format

Specifies how to represent the Kafka Record Key in the output FlowFile

Key Record Reader

The Record Reader to use for parsing the Kafka Record Key into a Record

Message Demarcator

Since KafkaConsumer receives messages in batches, this Processor has an option to output FlowFiles which contains all Kafka messages in a single batch for a given topic and partition and this property allows you to provide a string (interpreted as UTF-8) to use for demarcating apart multiple Kafka messages. This is an optional property and if not provided each Kafka message received will result in a single FlowFile which time it is triggered. To enter special character such as 'new line' use CTRL+Enter or Shift+Enter depending on the OS

Separate By Key

When this property is enabled, two messages will only be added to the same FlowFile if both of the Kafka Messages have identical keys.

Amazon Kinesis Stream Name

The name of Kinesis Stream

Application Name

The Kinesis stream reader application name.

Record Reader

The Record Reader to use for reading received messages. The Kinesis Stream name can be referred to by Expression Language '${kinesis.name}' to access a schema. If Record Reader/Writer are not specified, each Kinesis Record will create a FlowFile.

Record Writer

The Record Writer to use for serializing Records to an output FlowFile. The Kinesis Stream name can be referred to by Expression Language '${kinesis.name}' to access a schema. If Record Reader/Writer are not specified, each Kinesis Record will create a FlowFile.

Region

Endpoint Override URL

Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.

DynamoDB Override

DynamoDB override to use non-AWS deployments

Initial Stream Position

Initial position to read Kinesis streams.

Stream Position Timestamp

Timestamp position in stream from which to start reading Kinesis Records. Required if Initial position to read Kinesis streams. is AT_TIMESTAMP. Uses the Timestamp Format to parse value into a Date.

Timestamp Format

Format to use for parsing the Stream Position Timestamp into a Date and converting the Kinesis Record’s Approximate Arrival Timestamp into a FlowFile attribute.

Failover Timeout

Kinesis Client Library failover timeout

Graceful Shutdown Timeout

Kinesis Client Library graceful shutdown timeout

Checkpoint Interval

Interval between Kinesis checkpoints

Retry Count

Number of times to retry a Kinesis operation (process record, checkpoint, shutdown)

Retry Wait

Interval between Kinesis operation retries (process record, checkpoint, shutdown)

Report Metrics to CloudWatch

Whether to report Kinesis usage metrics to CloudWatch.

Communications Timeout

AWS Credentials Provider Service

The Controller Service that is used to obtain AWS credentials provider

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Kinesis Client Library (KCL) Configuration property name

Override default KCL Configuration ConfigsBuilder properties with required values. Supports setting of values directly on the ConfigsBuilder, such as 'namespace', as well as properties on nested builders. For example, to set configsBuilder.retrievalConfig().maxListShardsRetryAttempts(value), name the property as 'retrievalConfig.maxListShardsRetryAttempts'. Only supports setting of simple property values, e.g. String, int, long and boolean. Does not allow override of KCL Configuration settings handled by non-dynamic processor properties.

Broker URI

The URI(s) to use to connect to the MQTT broker (e.g., tcp://localhost:1883). The 'tcp', 'ssl', 'ws' and 'wss' schemes are supported. In order to use 'ssl', the SSL Context Service property must be set. When a comma-separated URI list is set (e.g., tcp://localhost:1883,tcp://localhost:1884), the processor will use a round-robin algorithm to connect to the brokers on connection failure.

MQTT Specification Version

The MQTT specification version when connecting with the broker. See the allowable value descriptions for more details.

Username

Username to use when connecting to the broker

Password

Password to use when connecting to the broker

SSL Context Service

The SSL Context Service used to provide client certificate information for TLS/SSL connections.

Session state

Whether to start a fresh or resume previous flows. See the allowable value descriptions for more details.

Session Expiry Interval

After this interval the broker will expire the client and clear the session state.

Client ID

MQTT client ID to use. If not set, a UUID will be generated.

Group ID

MQTT consumer group ID to use. If group ID not set, client will connect as individual consumer.

Topic Filter

The MQTT topic filter to designate the topics to subscribe to.

Quality of Service (QoS)

The Quality of Service (QoS) to receive the message with. Accepts values '0', '1' or '2'; '0' for 'at most once', '1' for 'at least once', '2' for 'exactly once'.

Record Reader

The Record Reader to use for parsing received MQTT Messages into Records.

Record Writer

The Record Writer to use for serializing Records before writing them to a FlowFile.

Add attributes as fields

If setting this property to true, default fields are going to be added in each record: _topic, _qos, _isDuplicate, _isRetained.

Message Demarcator

With this property, you have an option to output FlowFiles which contains multiple messages. This property allows you to provide a string (interpreted as UTF-8) to use for demarcating apart multiple messages. This is an optional property ; if not provided, and if not defining a Record Reader/Writer, each message received will result in a single FlowFile. To enter special character such as 'new line' use CTRL+Enter or Shift+Enter depending on the OS.

Connection Timeout (seconds)

Maximum time interval the client will wait for the network connection to the MQTT server to be established. The default timeout is 30 seconds. A value of 0 disables timeout processing meaning the client will wait until the network connection is made successfully or fails.

Keep Alive Interval (seconds)

Defines the maximum time interval between messages sent or received. It enables the client to detect if the server is no longer available, without having to wait for the TCP/IP timeout. The client will ensure that at least one message travels across the network within each keep alive period. In the absence of a data-related message during the time period, the client sends a very small "ping" message, which the server will acknowledge. A value of 0 disables keepalive processing in the client.

Last Will Message

The message to send as the client’s Last Will.

Last Will Topic

The topic to send the client’s Last Will to.

Last Will Retain

Whether to retain the client’s Last Will.

Last Will QoS Level

QoS level to be used when publishing the Last Will Message.

Max Queue Size

The MQTT messages are always being sent to subscribers on a topic regardless of how frequently the processor is scheduled to run. If the 'Run Schedule' is significantly behind the rate at which the messages are arriving to this processor, then a back up can occur in the internal queue of this processor. This property specifies the maximum number of messages this processor will hold in memory at one time in the internal queue. This data would be lost in case of a NiFi restart.

Host Name

Network address of Email server (e.g., pop.gmail.com, imap.gmail.com . . .)

Port

Numeric value identifying Port of Email server (e.g., 993)

Authorization Mode

How to authorize sending email on the user’s behalf.

OAuth2 Access Token Provider

OAuth2 service that can provide access tokens.

User Name

User Name used for authentication and authorization with Email server.

Password

Password used for authentication and authorization with Email server.

Folder

Email folder to retrieve messages from (e.g., INBOX)

Fetch Size

Specify the maximum number of Messages to fetch per call to Email Server.

Delete Messages

Specify whether mail messages should be deleted after retrieval.

Connection timeout

The amount of time to wait to connect to Email server

Channels

A comma-separated list of Slack Channels to Retrieve Messages From. Each element in the list may be either a Channel ID, such as C0L9VCD47, or (for public channels only) the name of a channel, prefixed with a # sign, such as #general. If any channel name is provided instead,instead of an ID, the Access Token provided must be granted the channels:read scope in order to resolve the Channel ID. See the Processor’s Additional Details for information on how to find a Channel ID.

Access Token

OAuth Access Token used for authenticating/authorizing the Slack request sent by NiFi. This may be either a User Token or a Bot Token. It must be granted the channels:history, groups:history, im:history, or mpim:history scope, depending on the type of conversation being used.

Reply Monitor Window

After consuming all messages in a given channel, this Processor will periodically poll all "threaded messages", aka Replies, whose timestamp is between now and this amount of time in the past in order to check for any new replies. Setting this value to a larger value may result in additional resource use and may result in Rate Limiting. However, if a user replies to an old thread that was started outside of this window, the reply may not be captured.

Reply Monitor Frequency

After consuming all messages in a given channel, this Processor will periodically poll all "threaded messages", aka Replies, whose timestamp falls between now and the amount of time specified by the <Reply Monitor Window> property. This property determines how frequently those messages are polled. Setting the value to a shorter duration may result in replies to messages being captured more quickly, providing a lower latency. However, it will also result in additional resource use and could trigger Rate Limiting to occur.

Batch Size

The maximum number of messages to retrieve in a single request to Slack. The entire response will be parsed into memory, so it is important that this be kept in mind when setting this value.

Resolve Usernames

Specifies whether or not User IDs should be resolved to usernames. By default, Slack Messages provide the ID of the user that sends a message, such as U0123456789, but not the username, such as NiFiUser. The username may be resolved, but it may require additional calls to the Slack API and requires that the Token used be granted the users:read scope. If set to true, usernames will be resolved with a best-effort policy: if a username cannot be obtained, it will be skipped over. Also, note that when a username is obtained, the Message’s <username> field is populated, and the <text> field is updated such that any mention will be output such as "Hi @user" instead of "Hi <@U1234567>".

Include Message Blocks

Specifies whether or not the output JSON should include the value of the 'blocks' field for each Slack Message. This field includes information such as individual parts of a message that are formatted using rich text. This may be useful, for instance, for parsing. However, it often accounts for a significant portion of the data and as such may be set to null when it is not useful to you.

Include Null Fields

Specifies whether or not fields that have null values should be included in the output JSON. If true, any field in a Slack Message that has a null value will be included in the JSON with a value of null. If false, the key omitted from the output JSON entirely. Omitting null values results in smaller messages that are generally more efficient to process, but including the values may provide a better understanding of the format, especially for schema inference.

Stream Endpoint

The source from which the processor will consume Tweets.

Base Path

The base path that the processor will use for making HTTP requests. The default value should be sufficient for most use cases.

Bearer Token

The Bearer Token provided by Twitter.

Queue Size

Maximum size of internal queue for streamed messages

Batch Size

The maximum size of the number of Tweets to be written to a single FlowFile. Will write fewer Tweets based on the number available in the queue at the time of processor invocation.

Backoff Attempts

The number of reconnection tries the processor will attempt in the event of a disconnection of the stream for any reason, before throwing an exception. To start a stream after this exception occur and the connection is fixed, please stop and restart the processor. If the valueof this property is 0, then backoff will never occur and the processor will always need to be restartedif the stream fails.

Backoff Time

The duration to backoff before requesting a new stream ifthe current one fails for any reason. Will increase by factor of 2 every time a restart fails

Maximum Backoff Time

The maximum duration to backoff to start attempting a new stream.It is recommended that this number be much higher than the 'Backoff Time' property

Connect Timeout

The maximum time in which client should establish a connection with the Twitter API before a time out. Setting the value to 0 disables connection timeouts.

Read Timeout

The maximum time of inactivity between receiving tweets from Twitter through the API before a timeout. Setting the value to 0 disables read timeouts.

Backfill Minutes

The number of minutes (up to 5 minutes) of streaming data to be requested after a disconnect. Only available for project with academic research access. See https://developer.twitter.com/en/docs/twitter-api/tweets/filtered-stream/integrate/recovery-and-redundancy-features

Tweet Fields

A comma-separated list of tweet fields to be returned as part of the tweet. Refer to https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/tweet for proper usage. Possible field values include: attachments, author_id, context_annotations, conversation_id, created_at, entities, geo, id, in_reply_to_user_id, lang, non_public_metrics, organic_metrics, possibly_sensitive, promoted_metrics, public_metrics, referenced_tweets, reply_settings, source, text, withheld

User Fields

A comma-separated list of user fields to be returned as part of the tweet. Refer to https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/user for proper usage. Possible field values include: created_at, description, entities, id, location, name, pinned_tweet_id, profile_image_url, protected, public_metrics, url, username, verified, withheld

Media Fields

A comma-separated list of media fields to be returned as part of the tweet. Refer to https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/media for proper usage. Possible field values include: alt_text, duration_ms, height, media_key, non_public_metrics, organic_metrics, preview_image_url, promoted_metrics, public_metrics, type, url, width

Poll Fields

A comma-separated list of poll fields to be returned as part of the tweet. Refer to https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/poll for proper usage. Possible field values include: duration_minutes, end_datetime, id, options, voting_status

Place Fields

A comma-separated list of place fields to be returned as part of the tweet. Refer to https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/place for proper usage. Possible field values include: contained_within, country, country_code, full_name, geo, id, name, place_type

Expansions

A comma-separated list of expansions for objects in the returned tweet. See https://developer.twitter.com/en/docs/twitter-api/expansions for proper usage. Possible field values include: author_id, referenced_tweets.id, referenced_tweets.id.author_id, entities.mentions.username, attachments.poll_ids, attachments.media_keys ,in_reply_to_user_id, geo.place_id

Channel

The Windows Event Log Channel to listen to.

XPath Query

XPath Query to filter events. (See https://msdn.microsoft.com/en-us/library/windows/desktop/dd996910(v=vs.85).aspx for examples.)

Maximum Buffer Size

The individual Event Log XMLs are rendered to a buffer. This specifies the maximum size in bytes that the buffer will be allowed to grow to. (Limiting the maximum size of an individual Event XML.)

Maximum queue size

Events are received asynchronously and must be output as FlowFiles when the processor is triggered. This specifies the maximum number of events to queue for transformation into FlowFiles.

Inactive duration to reconnect

If no new event logs are processed for the specified time period, this processor will try reconnecting to recover from a state where any further messages cannot be consumed. Such situation can happen if Windows Event Log service is restarted, or ERROR_EVT_QUERY_RESULT_STALE (15011) is returned. Setting no duration, e.g. '0 ms' disables auto-reconnection.

Rate Control Criteria

Indicates the criteria that is used to control the throughput rate. Changing this value resets the rate counters.

Time Duration

The amount of time to which the Maximum Rate pertains. Changing this value resets the rate counters.

Maximum Rate

The maximum rate at which data should pass through this processor. The format of this property is expected to be a positive integer, or a Data Size (such as '1 MB') if Rate Control Criteria is set to 'data rate'.

Maximum Data Rate

The maximum rate at which data should pass through this processor. The format of this property is expected to be a Data Size (such as '1 MB') representing bytes per Time Duration.

Maximum FlowFile Rate

The maximum rate at which FlowFiles should pass through this processor. The format of this property is expected to be a positive integer representing FlowFiles count per Time Duration

Rate Exceeded Strategy

Specifies how to handle an incoming FlowFile when the maximum data rate has been exceeded.

Rate Controlled Attribute

The name of an attribute whose values build toward the rate limit if Rate Control Criteria is set to 'attribute value'. The value of the attribute referenced by this property must be a positive long, or the FlowFile will be routed to failure. This value is ignored if Rate Control Criteria is not set to 'attribute value'. Changing this value resets the rate counters.

Grouping Attribute

By default, a single "throttle" is used for all FlowFiles. If this value is specified, a separate throttle is used for each value specified by the attribute with this name. Changing this value resets the rate counters.

Compression Type

The type of compression for the file being written.

Row Group Size

The row group size used by the Parquet writer. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.

Page Size

The page size used by the Parquet writer. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.

Dictionary Page Size

The dictionary page size used by the Parquet writer. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.

Max Padding Size

The maximum amount of padding that will be used to align row groups with blocks in the underlying filesystem. If the underlying filesystem is not a block filesystem like HDFS, this has no effect. The value is specified in the format of <Data Size> <Data Unit> where Data Unit is one of B, KB, MB, GB, TB.

Enable Dictionary Encoding

Specifies whether dictionary encoding should be enabled for the Parquet writer

Enable Validation

Specifies whether validation should be enabled for the Parquet writer

Writer Version

Specifies the version used by Parquet writer

Input Character Set

The name of the CharacterSet to expect for Input

Output Character Set

The name of the CharacterSet to convert to

XML Output Format

Use either the StAEDI format or Virtimo’s own XML format. With the Virtimo format, errors can also be skipped or only marked in the XML.

Verbose Output

Put more information in output - min occur, max occur, title and description (if available).

Throw Error

If an error occurs, do not write it to the XML, but throw an error directly.

Use Control Schema

EDI Schema that allows for user-specified validation rules.

Control Schema Body

Own EDI Schema in Property.

Control Schema File

Own EDI Schema from external file.

Validate Control Code Values

When set to false, enumerated code values of control structure elements will be ignore. Element size and type validation will still occur. Default value: true

Validate Control Structure

When set to false, control structures, segments, elements, and codes will not be validated unless a user-provided control schema has been set. When set to true AND no user-provided control schema has been set, the reader will attempt to find a known control schema for the detected EDI dialect and version to be used for control structure validation. Default value: true.

Ignore Extraneous Characters

When set to true, non-graphical, control characters will be ignored in the EDI input stream. This includes characters ranging from 0x00 through 0x1F and 0x7F. Default value: false.

Nest Hierarchical Loops

When set to true, hierarchical loops will be nested in the EDI input stream. The nesting structure is determined by the linkage specified by the EDI data itself using pointers given in the EDI schema for a loop. For example, the hierarchical information given by the X12 HL segment. Default value: true.

Trim Discriminator Values

When set to true, discriminator values from the EDI input will be trimmed (leading and trailing whitespace removed) prior to testing whether the value matches the enumerated values for a loop or segment defined in an implementation schema. Default value: false.

Pretty Print

Apply pretty-print formatting to the XML output.

Record Reader

Specifies the Controller Service to use for reading incoming data

Record Writer

Specifies the Controller Service to use for writing out the records

Include Zero Record FlowFiles

When converting an incoming FlowFile, if the conversion results in no data, this property specifies whether or not a FlowFile will be sent to the corresponding relationship

Source Storage Credentials

Credentials Service used to obtain Azure Blob Storage Credentials to read Source Blob information

Source Container Name

Name of the Azure storage container that will be copied

Source Blob Name

The full name of the source blob

Destination Storage Credentials

Controller Service used to obtain Azure Blob Storage Credentials.

Destination Container Name

Name of the Azure storage container destination defaults to the Source Container Name when not specified

Destination Blob Name

The full name of the destination blob defaults to the Source Blob Name when not specified

Conflict Resolution Strategy

Specifies whether an existing blob will have its contents replaced upon conflict.

Create Container

Specifies whether to check if the container exists and to automatically create it if it does not. Permission to list containers is required. If false, this check is not made, but the Put operation will fail if the container does not exist.

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.

Source Bucket

The bucket that contains the file to be copied.

Source Key

The source key in the source bucket

Destination Bucket

The bucket that will receive the copy.

Destination Key

The target key in the target bucket

AWS Credentials Provider Service

The Controller Service that is used to obtain AWS credentials provider

Region

The AWS Region to connect to.

Communications Timeout

The amount of time to wait in order to establish a connection to AWS or receive data from AWS before timing out.

FullControl User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have Full Control for an object

Read Permission User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have Read Access for an object

Write Permission User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have Write Access for an object

Read ACL User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have permissions to read the Access Control List for an object

Write ACL User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have permissions to change the Access Control List for an object

Canned ACL

Amazon Canned ACL for an object, one of: BucketOwnerFullControl, BucketOwnerRead, LogDeliveryWrite, AuthenticatedRead, PublicReadWrite, PublicRead, Private; will be ignored if any other ACL/permission/owner property is specified

Owner

The Amazon ID to use for the object’s owner

SSL Context Service

Specifies an optional SSL Context Service that, if provided, will be used to create connections

Endpoint Override URL

Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.

Signer Override

The AWS S3 library uses Signature Version 4 by default but this property allows you to specify the Version 2 signer to support older S3-compatible services or even to plug in your own custom signer implementation.

Custom Signer Class Name

Fully qualified class name of the custom signer class. The signer must implement com.amazonaws.auth.Signer interface.

Custom Signer Module Location

Comma-separated list of paths to files and/or directories which contain the custom signer’s JAR file and its dependencies (if any).

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Count Lines

If enabled, will count the number of lines present in the incoming text.

Count Non-Empty Lines

If enabled, will count the number of lines that contain a non-whitespace character present in the incoming text.

Count Words

If enabled, will count the number of words (alphanumeric character groups bounded by whitespace) present in the incoming text. Common logical delimiters [_-.] do not bound a word unless 'Split Words on Symbols' is true.

Count Characters

If enabled, will count the number of characters (including whitespace and symbols, but not including newlines and carriage returns) present in the incoming text.

Split Words on Symbols

If enabled, the word count will identify strings separated by common logical delimiters [ _ - . ] as independent words (ex. split-words-on-symbols = 4 words).

Character Encoding

Specifies a character encoding to use.

Call Immediate Adjustment

If true, the counter will be updated immediately, without regard to whether the ProcessSession is commit or rolled back;otherwise, the counter will be incremented only if and when the ProcessSession is committed.

Hadoop Configuration Resources

A file or comma separated list of files which contains the Hadoop file system configuration. Without this, Hadoop will search the classpath for a 'core-site.xml' and 'hdfs-site.xml' file or will revert to a default configuration. To use swebhdfs, see 'Additional Details' section of PutHDFS’s documentation.

Kerberos User Service

Specifies the Kerberos User Controller Service that should be used for authenticating with Kerberos

Additional Classpath Resources

A comma-separated list of paths to files and/or directories that will be added to the classpath and used for loading native libraries. When specifying a directory, all files with in the directory will be added to the classpath, but further sub-directories will not be included.

Compression type

Type of compression to use when creating Sequence File

Compression codec

Fail if the content is empty

Route to failure if the content is empty. While hashing an empty value is valid, some flows may want to detect empty input.

Hash Algorithm

The hash algorithm to use. Note that not all of the algorithms available are recommended for use (some are provided for legacy compatibility). There are many things to consider when picking an algorithm; it is recommended to use the most secure algorithm possible.

FlowFile Success Iterations

Number of FlowFiles to forward to success relationship.

FlowFile Failure Iterations

Number of FlowFiles to forward to failure relationship.

FlowFile Rollback Iterations

Number of FlowFiles to roll back (without penalty).

FlowFile Rollback Yield Iterations

Number of FlowFiles to roll back and yield.

FlowFile Rollback Penalty Iterations

Number of FlowFiles to roll back with penalty.

FlowFile Exception Iterations

Number of FlowFiles to throw exception.

FlowFile Exception Class

Exception class to be thrown (must extend java.lang.RuntimeException).

No FlowFile Skip Iterations

Number of times to skip onTrigger if no FlowFile.

No FlowFile Exception Iterations

Number of times to throw NPE exception if no FlowFile.

No FlowFile Yield Iterations

Number of times to yield if no FlowFile.

No FlowFile Exception Class

Exception class to be thrown if no FlowFile (must extend java.lang.RuntimeException).

Write Iterations

Number of times to write to the FlowFile

Content Size

The number of bytes to write each time that the FlowFile is written to

@OnScheduled Pause Time

Specifies how long the processor should sleep in the @OnScheduled method, so that the processor can be forced to take a long time to start up

Fail When @OnScheduled called

Specifies whether or not the Processor should throw an Exception when the methods annotated with @OnScheduled are called

@OnUnscheduled Pause Time

Specifies how long the processor should sleep in the @OnUnscheduled method, so that the processor can be forced to take a long time to respond when user clicks stop

Fail When @OnUnscheduled called

Specifies whether or not the Processor should throw an Exception when the methods annotated with @OnUnscheduled are called

@OnStopped Pause Time

Specifies how long the processor should sleep in the @OnStopped method, so that the processor can be forced to take a long time to shutdown

Fail When @OnStopped called

Specifies whether or not the Processor should throw an Exception when the methods annotated with @OnStopped are called

OnTrigger Pause Time

Specifies how long the processor should sleep in the onTrigger() method, so that the processor can be forced to take a long time to perform its task

CustomValidate Pause Time

Specifies how long the processor should sleep in the customValidate() method

Ignore Interrupts When Paused

If the Processor’s thread(s) are sleeping (due to one of the "Pause Time" properties above), and the thread is interrupted, this indicates whether the Processor should ignore the interrupt and continue sleeping or if it should allow itself to be interrupted.

Private Key Source

Source of information determines the loading strategy for X25519 Private Key Identities

Private Key Identities

One or more X25519 Private Key Identities, separated with newlines, encoded according to the age specification, starting with AGE-SECRET-KEY-1

Private Key Identity Resources

One or more files or URLs containing X25519 Private Key Identities, separated with newlines, encoded according to the age specification, starting with AGE-SECRET-KEY-1

Decryption Strategy

Strategy for writing files to success after decryption

Passphrase

Passphrase used for decrypting data encrypted with Password-Based Encryption

Private Key Service

PGP Private Key Service for decrypting data encrypted with Public Key Encryption

Deduplication Strategy

The strategy to use for detecting and routing duplicate records. The option for detecting duplicates across a single FlowFile operates in-memory, whereas detection spanning multiple FlowFiles utilises a distributed map cache.

Distributed Map Cache client

This property is required when the deduplication strategy is set to 'multiple files.' The map cache will for each record, atomically check whether the cache key exists and if not, set it.

Cache Identifier

An optional expression language field that overrides the record’s computed cache key. This field has an additional attribute available: ${record.hash.value}, which contains the cache key derived from dynamic properties (if set) or record fields.

Cache the Entry Identifier

For each record, check whether the cache identifier exists in the distributed map cache. If it doesn’t exist and this property is true, put the identifier to the cache.

Record Reader

Specifies the Controller Service to use for reading incoming data

Record Writer

Specifies the Controller Service to use for writing out the records

Include Zero Record FlowFiles

If a FlowFile sent to either the duplicate or non-duplicate relationships contains no records, a value of false in this property causes the FlowFile to be dropped. Otherwise, the empty FlowFile is emitted.

Record Hashing Algorithm

The algorithm used to hash the cache key.

Filter Type

The filter used to determine whether a record has been seen before based on the matching RecordPath criteria. If hash set is selected, a Java HashSet object will be used to deduplicate all encountered records. If the bloom filter option is selected, a bloom filter will be used. The bloom filter option is less memory intensive, but has a chance of having false positives.

Filter Capacity Hint

An estimation of the total number of unique records to be processed. The more accurate this number is will lead to fewer false negatives on a BloomFilter.

Bloom Filter Certainty

The desired false positive probability when using the BloomFilter type. Using a value of .05 for example, guarantees a five-percent probability that the result is a false positive. The closer to 1 this value is set, the more precise the result at the expense of more storage space utilization.

Name of the property.

A record’s cache key is generated by combining the name of each dynamic property with its evaluated record value (as specified by the corresponding RecordPath).

Storage Credentials

Controller Service used to obtain Azure Blob Storage Credentials.

Container Name

Name of the Azure storage container. In case of PutAzureBlobStorage processor, container can be created if it does not exist.

Blob Name

The full name of the blob

Delete Snapshots Option

Specifies the snapshot deletion options to be used when deleting a blob.

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.

ADLS Credentials

Controller Service used to obtain Azure Credentials.

Filesystem Name

Name of the Azure Storage File System (also called Container). It is assumed to be already existing.

Filesystem Object Type

They type of the file system object to be deleted. It can be either folder or file.

Directory Name

Name of the Azure Storage Directory. The Directory Name cannot contain a leading '/'. The root directory can be designated by the empty string value. In case of the PutAzureDataLakeStorage processor, the directory will be created if not already existing.

File Name

The filename

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP, SOCKS In case of SOCKS, it is not guaranteed that the selected SOCKS Version will be used by the processor.

BPC Controller

Controller used to define the connection to the BPC. The API-Key used by the controller requires 'LOG_SERVICE_DELETE_DATA'-Rights.

BPC Logger

Select the logger from available ones.

BPC Logger ID

The ID of the logger (i.e. the Component ID of the Log Service in BPC).

Delete all entries

WARNING: All entries from the chosen logger will be deleted.

Parent ID

The ID of the log entry to delete (in most cases the 'PROCESSID'-value). Multiple IDs can be set separated by comma.

Child ID

The ID of the child log entry to delete (in most cases the 'CHILDID'-value). If not set, deletes all child entries along with the parent log entry. Multiple IDs can be set separated by comma. Must not be set if multiple Parent IDs or set.

Query Definition Style

How the JSON Query will be defined for use by the processor.

Query

A query in JSON syntax, not Lucene syntax. Ex: {"query":{"match":{"somefield":"somevalue"}}}. If this parameter is not set, the query will be read from the flowfile content. If the query (property and flowfile content) is empty, a default empty JSON Object will be used, which will result in a "match_all" query in Elasticsearch.

Query Clause

A "query" clause in JSON syntax, not Lucene syntax. Ex: {"match":{"somefield":"somevalue"}}. If the query is empty, a default JSON Object will be used, which will result in a "match_all" query in Elasticsearch.

Query Attribute

If set, the executed query will be set on each result flowfile in the specified attribute.

Index

The name of the index to use.

Type

The type of this document (used by Elasticsearch for indexing and searching).

Max JSON Field String Length

The maximum allowed length of a string value when parsing a JSON document or attribute.

Client Service

An Elasticsearch client service to use for running queries.

The name of a URL query parameter to add

Adds the specified property name/value as a query parameter in the Elasticsearch URL used for processing. These parameters will override any matching parameters in the query request body

Table Name

The DynamoDB table name

Region

AWS Credentials Provider Service

The Controller Service that is used to obtain AWS credentials provider

Hash Key Name

The hash key name of the item

Range Key Name

The range key name of the item

Hash Key Value

The hash key value of the item

Range Key Value

Hash Key Value Type

The hash key value type of the item

Range Key Value Type

The range key value type of the item

Batch items for each request (between 1 and 50)

The items to be retrieved in one batch

Communications Timeout

Endpoint Override URL

Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.

SSL Context Service

Specifies an optional SSL Context Service that, if provided, will be used to create connections

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Directory Path

The path to the directory the file to delete is located in.

Filename

The name of the file to delete.

GCP Credentials Provider Service

The Controller Service used to obtain Google Cloud Platform credentials.

Project ID

Google Cloud Project ID

Bucket

Bucket of the object.

Key

Name of the object.

Generation

The generation of the object to be deleted. If null, will use latest version of the object.

Number of retries

How many retry attempts should be made before routing to the failure relationship.

Storage API URL

Overrides the default storage URL. Configuring an alternative Storage API URL also overrides the HTTP Host header on requests as described in the Google documentation for Private Service Connections.

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Client Service

The MongoDB client service to use for database connections.

Mongo Database Name

The name of the database to use

Bucket Name

The GridFS bucket where the files will be stored. If left blank, it will use the default value 'fs' that the MongoDB client driver uses.

File Name

The name of the file in the bucket that is the target of this processor. GridFS file names do not include path information because GridFS does not sort files into folders within a bucket.

Query

A valid MongoDB query to use to find and delete one or more files from GridFS.

Query Output Attribute

If set, the query will be written to a specified attribute on the output flowfiles.

Hadoop Configuration Resources

A file or comma separated list of files which contains the Hadoop file system configuration. Without this, Hadoop will search the classpath for a 'core-site.xml' and 'hdfs-site.xml' file or will revert to a default configuration. To use swebhdfs, see 'Additional Details' section of PutHDFS’s documentation.

Kerberos User Service

Specifies the Kerberos User Controller Service that should be used for authenticating with Kerberos

Additional Classpath Resources

A comma-separated list of paths to files and/or directories that will be added to the classpath and used for loading native libraries. When specifying a directory, all files with in the directory will be added to the classpath, but further sub-directories will not be included.

Path

The HDFS file or directory to delete. A wildcard expression may be used to only delete certain files

Recursive

Remove contents of a non-empty directory recursively

Client Service

If configured, this property will use the assigned client service for connection pooling.

Mongo Database Name

The name of the database to use

Mongo Collection Name

The name of the collection to use

Delete Mode

Choose between deleting one document by query or many documents by query.

Fail When Nothing Is Deleted

Determines whether to send the flowfile to the success or failure relationship if nothing is successfully deleted.

Bucket

The S3 Bucket to interact with

Object Key

The S3 Object Key to use. This is analogous to a filename for traditional file systems.

AWS Credentials Provider Service

The Controller Service that is used to obtain AWS credentials provider

Region

The AWS Region to connect to.

Communications Timeout

The amount of time to wait in order to establish a connection to AWS or receive data from AWS before timing out.

Version

The Version of the Object to delete

FullControl User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have Full Control for an object

Read Permission User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have Read Access for an object

Write Permission User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have Write Access for an object

Read ACL User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have permissions to read the Access Control List for an object

Write ACL User List

A comma-separated list of Amazon User ID’s or E-mail addresses that specifies who should have permissions to change the Access Control List for an object

Owner

The Amazon ID to use for the object’s owner

SSL Context Service

Specifies an optional SSL Context Service that, if provided, will be used to create connections

Endpoint Override URL

Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.

Signer Override

The AWS S3 library uses Signature Version 4 by default but this property allows you to specify the Version 2 signer to support older S3-compatible services or even to plug in your own custom signer implementation.

Custom Signer Class Name

Fully qualified class name of the custom signer class. The signer must implement com.amazonaws.auth.Signer interface.

Custom Signer Module Location

Comma-separated list of paths to files and/or directories which contain the custom signer’s JAR file and its dependencies (if any).

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Directory Path

The path to the directory the file to delete is located in.

Filename

The name of the file to delete.

Hostname

The fully qualified hostname or IP address of the remote system

Port

The port that the remote system is listening on for file transfers

Username

Username

Password

Password for the user account

Private Key Path

The fully qualified path to the Private Key file

Private Key Passphrase

Password for the private key

Strict Host Key Checking

Indicates whether or not strict enforcement of hosts keys should be applied

Host Key File

If supplied, the given file will be used as the Host Key; otherwise, if 'Strict Host Key Checking' property is applied (set to true) then uses the 'known_hosts' and 'known_hosts2' files from ~/.ssh directory else no host key file will be used

Batch Size

The maximum number of FlowFiles to send in a single connection

Connection Timeout

Amount of time to wait before timing out while creating a connection

Data Timeout

When transferring a file between the local and remote system, this value specifies how long is allowed to elapse without any data being transferred between systems

Send Keep Alive On Timeout

Send a Keep Alive message every 5 seconds up to 5 times for an overall timeout of 25 seconds.

Use Compression

Indicates whether or not ZLIB compression should be used when transferring files

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN, SOCKS + AuthN

Ciphers Allowed

A comma-separated list of Ciphers allowed for SFTP connections. Leave unset to allow all. Available options are: 3des-cbc, 3des-ctr, aes128-cbc, aes128-ctr, aes128-gcm@openssh.com, aes192-cbc, aes192-ctr, aes256-cbc, aes256-ctr, aes256-gcm@openssh.com, arcfour, arcfour128, arcfour256, blowfish-cbc, blowfish-ctr, cast128-cbc, cast128-ctr, chacha20-poly1305@openssh.com, idea-cbc, idea-ctr, serpent128-cbc, serpent128-ctr, serpent192-cbc, serpent192-ctr, serpent256-cbc, serpent256-ctr, twofish-cbc, twofish128-cbc, twofish128-ctr, twofish192-cbc, twofish192-ctr, twofish256-cbc, twofish256-ctr

Key Algorithms Allowed

A comma-separated list of Key Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: ecdsa-sha2-nistp256, ecdsa-sha2-nistp256-cert-v01@openssh.com, ecdsa-sha2-nistp384, ecdsa-sha2-nistp384-cert-v01@openssh.com, ecdsa-sha2-nistp521, ecdsa-sha2-nistp521-cert-v01@openssh.com, rsa-sha2-256, rsa-sha2-512, ssh-dss, ssh-dss-cert-v01@openssh.com, ssh-ed25519, ssh-ed25519-cert-v01@openssh.com, ssh-rsa, ssh-rsa-cert-v01@openssh.com

Key Exchange Algorithms Allowed

A comma-separated list of Key Exchange Algorithms allowed for SFTP connections. Leave unset to allow all. Available options are: curve25519-sha256, curve25519-sha256@libssh.org, diffie-hellman-group-exchange-sha1, diffie-hellman-group-exchange-sha256, diffie-hellman-group1-sha1, diffie-hellman-group14-sha1, diffie-hellman-group14-sha256, diffie-hellman-group14-sha256@ssh.com, diffie-hellman-group15-sha256, diffie-hellman-group15-sha256@ssh.com, diffie-hellman-group15-sha384@ssh.com, diffie-hellman-group15-sha512, diffie-hellman-group16-sha256, diffie-hellman-group16-sha384@ssh.com, diffie-hellman-group16-sha512, diffie-hellman-group16-sha512@ssh.com, diffie-hellman-group17-sha512, diffie-hellman-group18-sha512, diffie-hellman-group18-sha512@ssh.com, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, ext-info-c

Message Authentication Codes Allowed

A comma-separated list of Message Authentication Codes allowed for SFTP connections. Leave unset to allow all. Available options are: hmac-md5, hmac-md5-96, hmac-md5-96-etm@openssh.com, hmac-md5-etm@openssh.com, hmac-ripemd160, hmac-ripemd160-96, hmac-ripemd160-etm@openssh.com, hmac-ripemd160@openssh.com, hmac-sha1, hmac-sha1-96, hmac-sha1-96@openssh.com, hmac-sha1-etm@openssh.com, hmac-sha2-256, hmac-sha2-256-etm@openssh.com, hmac-sha2-512, hmac-sha2-512-etm@openssh.com

Queue URL

The URL of the queue delete from

Region

AWS Credentials Provider Service

The Controller Service that is used to obtain AWS credentials provider

SSL Context Service

Specifies an optional SSL Context Service that, if provided, will be used to create connections

Receipt Handle

The identifier that specifies the receipt of the message

Communications Timeout

Endpoint Override URL

Endpoint URL to use instead of the AWS default including scheme, host, port, and path. The AWS libraries select an endpoint URL based on the AWS region, but this property overrides the selected endpoint URL, allowing use with other S3-compatible endpoints.

Proxy Configuration Service

Specifies the Proxy Configuration Controller Service to proxy network requests. Supported proxies: HTTP + AuthN

Cache Entry Identifier

A FlowFile attribute, or the results of an Attribute Expression Language statement, which will be evaluated against a FlowFile in order to determine the value used to identify duplicates; it is this value that is cached

FlowFile Description

When a FlowFile is added to the cache, this value is stored along with it so that if a duplicate is found, this description of the original FlowFile will be added to the duplicate’s "original.flowfile.description" attribute

Age Off Duration

Time interval to age off cached FlowFiles

Distributed Cache Service

The Controller Service that is used to cache unique identifiers, used to determine duplicates

Cache The Entry Identifier

When true this cause the processor to check for duplicates and cache the Entry Identifier. When false, the processor would only check for duplicates and not cache the Entry Identifier, requiring another processor to add identifiers to the distributed cache.

Number of Relationships

Determines the number of Relationships to which the load should be distributed

Distribution Strategy

Determines how the load will be distributed. Relationship weight is in numeric order where '1' has the greatest weight.

The relationship name (positive number)

Adding a property with the name '5' and value '10' means that the relationship with name '5' will receive 10 FlowFiles in each iteration instead of 1.

Number of Copies

Specifies how many copies of each incoming FlowFile will be made

Mode

Specifies whether the content should be encoded or decoded.

Encoding

Specifies the type of encoding used.

Line Output Mode

Controls the line formatting for encoded content based on selected property values.

Encoded Line Length

Each line of encoded data will contain up to the configured number of characters, rounded down to the nearest multiple of 4.

File Encoding

Output encoding for encrypted files. Binary encoding provides optimal processing performance.

Public Key Source

Source of information determines the loading strategy for X25519 Public Key Recipients

Public Key Recipients

One or more X25519 Public Key Recipients, separated with newlines, encoded according to the age specification, starting with age1

Public Key Recipient Resources

One or more files or URLs containing X25519 Public Key Recipients, separated with newlines, encoded according to the age specification, starting with age1

Symmetric-Key Algorithm

Symmetric-Key Algorithm for encryption

File Encoding

File Encoding for encryption

Passphrase

Passphrase used for encrypting data with Password-Based Encryption

Public Key Service

PGP Public Key Service for encrypting data with Public Key Encryption

Public Key Search

PGP Public Key Search will be used to match against the User ID or Key ID when formatted as uppercase hexadecimal string of 16 characters

Group Identifier

EnforceOrder is capable of multiple ordering groups. 'Group Identifier' is used to determine which group a FlowFile belongs to. This property will be evaluated with each incoming FlowFile. If evaluated result is empty, the FlowFile will be routed to failure.

Order Attribute

A name of FlowFile attribute whose value will be used to enforce order of FlowFiles within a group. If a FlowFile does not have this attribute, or its value is not an integer, the FlowFile will be routed to failure.

Initial Order

When the first FlowFile of a group arrives, initial target order will be computed and stored in the managed state. After that, target order will start being tracked by EnforceOrder and stored in the state management store. If Expression Language is used but evaluated result was not an integer, then the FlowFile will be routed to failure, and initial order will be left unknown until consecutive FlowFiles provide a valid initial order.

Maximum Order

If specified, any FlowFiles that have larger order will be routed to failure. This property is computed only once for a given group. After a maximum order is computed, it will be persisted in the state management store and used for other FlowFiles belonging to the same group. If Expression Language is used but evaluated result was not an integer, then the FlowFile will be routed to failure, and maximum order will be left unknown until consecutive FlowFiles provide a valid maximum order.

Batch Count

The maximum number of FlowFiles that EnforceOrder can process at an execution.

Wait Timeout

Indicates the duration after which waiting FlowFiles will be routed to the 'overtook' relationship.

Inactive Timeout

Indicates the duration after which state for an inactive group will be cleared from managed state. Group is determined as inactive if any new incoming FlowFile has not seen for a group for specified duration. Inactive Timeout must be longer than Wait Timeout. If a FlowFile arrives late after its group is already cleared, it will be treated as a brand new group, but will never match the order since expected preceding FlowFiles are already gone. The FlowFile will eventually timeout for waiting and routed to 'overtook'. To avoid this, group states should be kept long enough, however, shorter duration would be helpful for reusing the same group identifier again.

Destination

Indicates whether the results of the JsonPath evaluation are written to the FlowFile content or a FlowFile attribute; if using attribute, must specify the Attribute Name property. If set to flowfile-content, only one JsonPath may be specified, and the property name is ignored.

Return Type

Indicates the desired return type of the JSON Path expressions. Selecting 'auto-detect' will set the return type to 'json' for a Destination of 'flowfile-content', and 'scalar' for a Destination of 'flowfile-attribute'.

Path Not Found Behavior

Indicates how to handle missing JSON path expressions when destination is set to 'flowfile-attribute'. Selecting 'warn' will generate a warning when a JSON path expression is not found. Selecting 'skip' will omit attributes for any unmatched JSON path expressions.

Null Value Representation

Indicates the desired representation of JSON Path expressions resulting in a null value.

Max String Length

The maximum allowed length of a string value when parsing the JSON document

A FlowFile attribute(if <Destination> is set to 'flowfile-attribute')

If <Destination>='flowfile-attribute' then that FlowFile attribute will be set to any JSON objects that match the JsonPath. If <Destination>='flowfile-content' then the FlowFile content will be updated to any JSON objects that match the JsonPath.

Destination

Indicates whether the results of the XPath evaluation are written to the FlowFile content or a FlowFile attribute; if using attribute, must specify the Attribute Name property. If set to flowfile-content, only one XPath may be specified, and the property name is ignored.

Return Type

Indicates the desired return type of the Xpath expressions. Selecting 'auto-detect' will set the return type to 'nodeset' for a Destination of 'flowfile-content', and 'string' for a Destination of 'flowfile-attribute'.

Allow DTD

Allow embedded Document Type Declaration in XML. This feature should be disabled to avoid XML entity expansion vulnerabilities.

A FlowFile attribute(if <Destination> is set to 'flowfile-attribute'

If <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XPath Expression. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XPath Expression.

Destination

Indicates whether the results of the XQuery evaluation are written to the FlowFile content or a FlowFile attribute. If set to <flowfile-content>, only one XQuery may be specified and the property name is ignored. If set to <flowfile-attribute> and the XQuery returns more than one result, multiple attributes will be added to theFlowFile, each named with a '.n' one-up number appended to the specified attribute name

Output: Method

Identifies the overall method that should be used for outputting a result tree.

Output: Omit XML Declaration

Specifies whether the processor should output an XML declaration when transforming a result tree.

Output: Indent

Specifies whether the processor may add additional whitespace when outputting a result tree.

Allow DTD

Allow embedded Document Type Declaration in XML. This feature should be disabled to avoid XML entity expansion vulnerabilities.

A FlowFile attribute(if <Destination> is set to 'flowfile-attribute'

If <Destination>='flowfile-attribute' then the FlowFile attribute is set to the result of the XQuery. If <Destination>='flowfile-content' then the FlowFile content is set to the result of the XQuery.

Client Service

The graph client service for connecting to the graph database.

Graph Query

Specifies the graph query. If it is left blank, the processor will attempt to get the query from body.

Client Service

The graph client service for connecting to a graph database.

Record Reader

The record reader to use with this processor.

Failed Record Writer

The record writer to use for writing failed records.

Graph Record Script

Script to perform the business logic on graph, using flow file attributes and custom properties as variable-value pairs in its logic.

A dynamic property to be used as a parameter in the graph script

Uses a record path to set a variable as a parameter in the graph script

Script File

Path to script file to execute. Only one of Script File or Script Body may be used

Script Body

Body of script to execute. Only one of Script File or Script Body may be used

Failure strategy

What to do with unhandled exceptions. If you want to manage exception by code then keep the default value rollback. If transfer to failure selected and unhandled exception occurred then all flowFiles received from incoming queues in this session will be transferred to failure relationship with additional attributes set: ERROR_MESSAGE and ERROR_STACKTRACE. If rollback selected and unhandled exception occurred then all flowFiles received from incoming queues will be penalized and returned. If the processor has no incoming connections then this parameter has no effect.

Additional classpath

Classpath list separated by semicolon or comma. You can use masks like , .jar in file name.

A script engine property to update

Updates a script engine property specified by the Dynamic Property’s key with the value specified by the Dynamic Property’s value. Use CTL. to access any controller services, SQL. to access any DBCPServices, RecordReader. to access RecordReaderFactory instances, or RecordWriter. to access any RecordSetWriterFactory instances.

Command

Specifies the command to be executed; if just the name of an executable is provided, it must be in the user’s environment PATH.

Command Arguments

The arguments to supply to the executable delimited by white space. White space can be escaped by enclosing it in double-quotes.

Batch Duration

If the process is expected to be long-running and produce textual output, a batch duration can be specified so that the output will be captured for this amount of time and a FlowFile will then be sent out with the results and a new FlowFile will be started, rather than waiting for the process to finish before sending out the results

Redirect Error Stream

If true will redirect any error stream output of the process to the output stream. This is particularly helpful for processes which write extensively to the error stream or for troubleshooting.

Working Directory

The directory to use as the current working directory when executing the command

Argument Delimiter

Delimiter to use to separate arguments for a command [default: space]. Must be a single character.

Output MIME Type

Specifies the value to set for the "mime.type" attribute. This property is ignored if 'Batch Duration' is set.

An environment variable name

These environment variables are passed to the process spawned by this Processor

Script Engine

No Script Engines found

Script File

Path to script file to execute. Only one of Script File or Script Body may be used

Script Body

Body of script to execute. Only one of Script File or Script Body may be used

Module Directory

Comma-separated list of paths to files and/or directories which contain modules required by the script.

Script Engine Binding property

Updates a script engine property specified by the Dynamic Property’s key with the value specified by the Dynamic Property’s value