Package org.apache.nifi.processors.standard

Class PartitionRecord

java.lang.Object
org.apache.nifi.components.AbstractConfigurableComponent
org.apache.nifi.processor.AbstractSessionFactoryProcessor
org.apache.nifi.processor.AbstractProcessor
org.apache.nifi.processors.standard.PartitionRecord
All Implemented Interfaces:
org.apache.nifi.components.ConfigurableComponent, org.apache.nifi.processor.Processor
@SupportsBatching @InputRequirement(INPUT_REQUIRED) @CapabilityDescription("Splits, or partitions, record-oriented data based on the configured fields in the data. One or more properties must be added. The name of the property is the name of an attribute to add. The value of the property is a RecordPath to evaluate against each Record. Two records will go to the same outbound FlowFile only if they have the same value for each of the given RecordPaths. Because we know that all records in a given output FlowFile have the same value for the fields that are specified by the RecordPath, an attribute is added for each field. See Additional Details on the Usage page for more information and examples.") @DynamicProperty(name="The name given to the dynamic property is the name of the attribute that will be used to denote the value of the associated RecordPath.", value="A RecordPath that points to a field in the Record.", description="Each dynamic property represents a RecordPath that will be evaluated against each record in an incoming FlowFile. When the value of the RecordPath is determined for a Record, an attribute is added to the outgoing FlowFile. The name of the attribute is the same as the name of this property. The value of the attribute is the same as the value of the field in the Record that the RecordPath points to. Note that no attribute will be added if the value returned for the RecordPath is null or is not a scalar value (i.e., the value is an Array, Map, or Record).", expressionLanguageScope=FLOWFILE_ATTRIBUTES) @WritesAttribute(attribute="record.count",description="The number of records in an outgoing FlowFile") @WritesAttribute(attribute="mime.type",description="The MIME Type that the configured Record Writer indicates is appropriate") @WritesAttribute(attribute="fragment.identifier",description="All partitioned FlowFiles produced from the same parent FlowFile will have the same randomly generated UUID added for this attribute") @WritesAttribute(attribute="fragment.index",description="A one-up number that indicates the ordering of the partitioned FlowFiles that were created from a single parent FlowFile") @WritesAttribute(attribute="fragment.count",description="The number of partitioned FlowFiles generated from the parent FlowFile") @WritesAttribute(attribute="segment.original.filename ",description="The filename of the parent FlowFile") @WritesAttribute(attribute="<dynamic property name>",description="For each dynamic property that is added, an attribute may be added to the FlowFile. See the description for Dynamic Properties for more information.") @Tags({"record","partition","recordpath","rpath","segment","split","group","bin","organize"}) @SeeAlso({ConvertRecord.class,SplitRecord.class,UpdateRecord.class,QueryRecord.class}) @UseCase(description="Separate records into separate FlowFiles so that all of the records in a FlowFile have the same value for a given field or set of fields.",keywords={"separate","split","partition","break apart","colocate","segregate","record","field","recordpath"},configuration="Choose a RecordReader that is appropriate based on the format of the incoming data.\nChoose a RecordWriter that writes the data in the desired output format.\n\nAdd a single additional property. The name of the property should describe the type of data that is being used to partition the data. The property\'s value should be a RecordPath that specifies which output FlowFile the Record belongs to.\n\nFor example, if we want to separate records based on their `transactionType` field, we could add a new property named `transactionType`. The value of the property might be `/transaction/type`. An input FlowFile will then be separated into as few FlowFiles as possible such that each output FlowFile has the same value for the `transactionType` field.\n") @UseCase(description="Separate records based on whether or not they adhere to a specific criteria",keywords={"separate","split","partition","break apart","segregate","record","field","recordpath","criteria"},configuration="Choose a RecordReader that is appropriate based on the format of the incoming data.\nChoose a RecordWriter that writes the data in the desired output format.\n\nAdd a single additional property. The name of the property should describe the criteria. The property\'s value should be a RecordPath that returns `true` if the Record meets the criteria or `false` otherwise.\n\nFor example, if we want to separate records based on whether or not they have a transaction total of more than $1,000 we could add a new property named `largeTransaction` with a value of `/transaction/total > 1000`. This will create two FlowFiles. In the first, all records will have a total over `1000`. In the second, all records will have a transaction less than or equal to 1000. Each FlowFile will have an attribute named `largeTransaction` with a value of `true` or `false`.\n") public class PartitionRecord extends org.apache.nifi.processor.AbstractProcessor

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    private static class 
    PartitionRecord.RecordValueMap
     
    (package private) static class 
    PartitionRecord.ValueWrapper
    We have this ValueWrapper class here because we want to use it as part of the key to a Map and we want two values that may or may not be arrays.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private static final List<org.apache.nifi.components.PropertyDescriptor>
    PROPERTY_DESCRIPTORS
     
    (package private) static final org.apache.nifi.components.PropertyDescriptor
    RECORD_READER
     
    (package private) static final org.apache.nifi.components.PropertyDescriptor
    RECORD_WRITER
     
    private final RecordPathCache
    recordPathCache
     
    (package private) static final org.apache.nifi.processor.Relationship
    REL_FAILURE
     
    (package private) static final org.apache.nifi.processor.Relationship
    REL_ORIGINAL
     
    (package private) static final org.apache.nifi.processor.Relationship
    REL_SUCCESS
     
    private static final Set<org.apache.nifi.processor.Relationship>
    RELATIONSHIPS
     

  • Constructor Summary

    Constructors
    Constructor
    Description
    PartitionRecord()
     

  • Method Summary

    Modifier and Type
    Method
    Description
    protected Collection<org.apache.nifi.components.ValidationResult>
    customValidate(org.apache.nifi.components.ValidationContext validationContext)
     
    private RecordPath
    getRecordPath(org.apache.nifi.processor.ProcessContext context, org.apache.nifi.components.PropertyDescriptor prop, org.apache.nifi.flowfile.FlowFile flowFile)
     
    Set<org.apache.nifi.processor.Relationship>
    getRelationships()
     
    protected org.apache.nifi.components.PropertyDescriptor
    getSupportedDynamicPropertyDescriptor(String propertyDescriptorName)
     
    protected List<org.apache.nifi.components.PropertyDescriptor>
    getSupportedPropertyDescriptors()
     
    void
    onTrigger(org.apache.nifi.processor.ProcessContext context, org.apache.nifi.processor.ProcessSession session)
     

    Methods inherited from class org.apache.nifi.processor.AbstractProcessor

    onTrigger

    Methods inherited from class org.apache.nifi.processor.AbstractSessionFactoryProcessor

    getControllerServiceLookup, getIdentifier, getLogger, getNodeTypeProvider, init, initialize, isConfigurationRestored, isScheduled, toString, updateConfiguredRestoredTrue, updateScheduledFalse, updateScheduledTrue

    Methods inherited from class org.apache.nifi.components.AbstractConfigurableComponent

    equals, getPropertyDescriptor, getPropertyDescriptors, hashCode, onPropertyModified, validate

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.apache.nifi.components.ConfigurableComponent

    getPropertyDescriptor, getPropertyDescriptors, onPropertyModified, validate

    Methods inherited from interface org.apache.nifi.processor.Processor

    isStateful, migrateProperties, migrateRelationships

  • Field Details

    • recordPathCache

      private final RecordPathCache recordPathCache

    • RECORD_READER

      static final org.apache.nifi.components.PropertyDescriptor RECORD_READER

    • RECORD_WRITER

      static final org.apache.nifi.components.PropertyDescriptor RECORD_WRITER

    • PROPERTY_DESCRIPTORS

      private static final List<org.apache.nifi.components.PropertyDescriptor> PROPERTY_DESCRIPTORS

    • REL_SUCCESS

      static final org.apache.nifi.processor.Relationship REL_SUCCESS

    • REL_ORIGINAL

      static final org.apache.nifi.processor.Relationship REL_ORIGINAL

    • REL_FAILURE

      static final org.apache.nifi.processor.Relationship REL_FAILURE

    • RELATIONSHIPS

      private static final Set<org.apache.nifi.processor.Relationship> RELATIONSHIPS

  • Constructor Details

    • PartitionRecord

      public PartitionRecord()

  • Method Details

    • getSupportedPropertyDescriptors

      protected List<org.apache.nifi.components.PropertyDescriptor> getSupportedPropertyDescriptors()
      Overrides:
      getSupportedPropertyDescriptors in class org.apache.nifi.components.AbstractConfigurableComponent

    • getRelationships

      public Set<org.apache.nifi.processor.Relationship> getRelationships()
      Specified by:
      getRelationships in interface org.apache.nifi.processor.Processor
      Overrides:
      getRelationships in class org.apache.nifi.processor.AbstractSessionFactoryProcessor

    • customValidate

      protected Collection<org.apache.nifi.components.ValidationResult> customValidate(org.apache.nifi.components.ValidationContext validationContext)
      Overrides:
      customValidate in class org.apache.nifi.components.AbstractConfigurableComponent

    • getSupportedDynamicPropertyDescriptor

      protected org.apache.nifi.components.PropertyDescriptor getSupportedDynamicPropertyDescriptor(String propertyDescriptorName)
      Overrides:
      getSupportedDynamicPropertyDescriptor in class org.apache.nifi.components.AbstractConfigurableComponent

    • onTrigger

      public void onTrigger(org.apache.nifi.processor.ProcessContext context, org.apache.nifi.processor.ProcessSession session) throws org.apache.nifi.processor.exception.ProcessException
      Specified by:
      onTrigger in class org.apache.nifi.processor.AbstractProcessor
      Throws:
      org.apache.nifi.processor.exception.ProcessException

    • getRecordPath

      private RecordPath getRecordPath(org.apache.nifi.processor.ProcessContext context, org.apache.nifi.components.PropertyDescriptor prop, org.apache.nifi.flowfile.FlowFile flowFile)