RouteOnAttribute
Routes FlowFiles based on their Attributes using the Attribute Expression Language
Use Cases
Route data to one or more relationships based on its attributes using the NiFi Expression Language.
Keywords: attributes, routing, expression language
Input Requirement: This component allows an incoming relationship.
-
Set the "Routing Strategy" property to "Route to Property name".
-
For each route that a FlowFile might be routed to, add a new property. The name of the property should describe the route.
-
The value of the property is an Attribute Expression Language expression that returns a boolean value indicating whether or not a given FlowFile will be routed to the associated relationship. .
-
For example, we might route data based on its file extension using the following properties:
-
- "Routing Strategy" = "Route to Property Name"
-
- "jpg" = "${filename:endsWith('.jpg')}"
-
- "png" = "${filename:endsWith('.png')}"
-
- "pdf" = "${filename:endsWith('.pdf')}" .
-
The Processor will now have 3 relationships:
jpg,png, andpdf. Each of these should be connected to the appropriate downstream processor. .
Keep data only if its attributes meet some criteria, such as its filename ends with .txt.
Keywords: keep, filter, remove, delete, expression language
Input Requirement: This component allows an incoming relationship.
-
Add a new property for each condition that must be satisfied in order to keep the data.
-
If the data should be kept in the case that any of the provided conditions is met, set the "Routing Strategy" property to "Route to 'matched' if any matches".
-
If all conditions must be met in order to keep the data, set the "Routing Strategy" property to "Route to 'matched' if all match". .
-
For example, to keep files whose filename ends with .txt and have a file size of at least 1000 bytes, we will use the following properties:
-
- "ends_with_txt" = "${filename:endsWith('.txt')}"
-
- "large_enough" = "${fileSize:ge(1000)}
-
- "Routing Strategy" = "Route to 'matched' if all match" .
-
Auto-terminate the 'unmatched' relationship.
-
Connect the 'matched' relationship to the next processor in the flow. .
Discard or drop a file based on attributes, such as filename.
Keywords: discard, drop, filter, remove, delete, expression language
Input Requirement: This component allows an incoming relationship.
-
Add a new property for each condition that must be satisfied in order to drop the data.
-
If the data should be dropped in the case that any of the provided conditions is met, set the "Routing Strategy" property to "Route to 'matched' if any matches".
-
If all conditions must be met in order to drop the data, set the "Routing Strategy" property to "Route to 'matched' if all match". .
-
Here are a couple of examples for configuring the properties:
-
Example 1 Use Case: Data should be dropped if its "uuid" attribute has an 'a' in it or ends with '0'.
-
Here, we will use the following properties:
-
- "has_a" = "${uuid:contains('a')}"
-
- "ends_with_0" = "${uuid:endsWith('0')}
-
- "Routing Strategy" = "Route to 'matched' if any matches"
-
Example 2 Use Case: Data should be dropped if its 'uuid' attribute has an 'a' AND it ends with a '1'.
-
Here, we will use the following properties:
-
- "has_a" = "${uuid:contains('a')}"
-
- "ends_with_1" = "${uuid:endsWith('1')}
-
- "Routing Strategy" = "Route to 'matched' if all match" .
-
Auto-terminate the 'matched' relationship.
-
Connect the 'unmatched' relationship to the next processor in the flow. .
Route record-oriented data based on whether or not the record’s values meet some criteria
Keywords: record, route, content, data
PartitionRecord:
-
Choose a RecordReader that is appropriate based on the format of the incoming data.
-
Choose a RecordWriter that writes the data in the desired output format. .
-
Add a single additional property. The name of the property should describe the criteria to route on. The property’s value should be a RecordPath that returns
trueif the Record meets the criteria orfalseotherwise. This adds a new attribute to the FlowFile whose name is equal to the property name. . -
Connect the 'success' Relationship to RouteOnAttribute. .
RouteOnAttribute:
-
Set "Routing Strategy" to "Route to Property name" .
-
Add two additional properties. For the first one, the name of the property should describe data that matches the criteria. The value is an Expression Language expression that checks if the attribute added by the PartitionRecord processor has a value of
true. For example,${criteria:equals('true')}. -
The second property should have a name that describes data that does not match the criteria. The value is an Expression Language that evaluates to the opposite of the first property value. For example,
${criteria:equals('true'):not()}. . -
Connect each of the newly created Relationships to the appropriate downstream processors. .
Tags: attributes, routing, Attribute Expression Language, regexp, regex, Regular Expression, Expression Language, find, text, string, search, filter, detect
Properties
Routing Strategy |
Specifies how to determine which relationship to use when evaluating the Expression Language |
Dynamic Properties
Relationship Name |
Routes FlowFiles whose attributes match the Expression Language specified in the Dynamic Property Value to the Relationship specified in the Dynamic Property Key |
Dynamic Relationships
-
Name from Dynamic Property: FlowFiles that match the Dynamic Property’s Attribute Expression Language
Additional Details
Usage Example
This processor routes FlowFiles based on their attributes using the NiFi Expression Language. Users add properties with valid NiFi Expression Language Expressions as the values. Each Expression must return a value of type Boolean (true or false).
Example: The goal is to route all files with filenames that start with ABC down a certain path. Add a property with the following name and value:
-
property name: ABC
-
property value: ${filename:startsWith(‘ABC’)}
In this example, all files with filenames that start with ABC will follow the ABC relationship.