Package org.apache.nifi.processors.standard.merge

Class RecordBinManager

java.lang.Object
org.apache.nifi.processors.standard.merge.RecordBinManager
public class RecordBinManager extends Object

  • Field Details

    • context

      private final org.apache.nifi.processor.ProcessContext context

    • sessionFactory

      private final org.apache.nifi.processor.ProcessSessionFactory sessionFactory

    • logger

      private final org.apache.nifi.logging.ComponentLog logger

    • maxBinCount

      private final int maxBinCount

    • maxBinAgeNanos

      private final AtomicLong maxBinAgeNanos

    • groupBinMap

      private final Map<String,List<RecordBin>> groupBinMap

    • lock

      private final Lock lock

    • binCount

      private final AtomicInteger binCount

  • Constructor Details

    • RecordBinManager

      public RecordBinManager(org.apache.nifi.processor.ProcessContext context, org.apache.nifi.processor.ProcessSessionFactory sessionFactory, org.apache.nifi.logging.ComponentLog logger)

  • Method Details

    • purge

      public void purge()
      Must be called only when there are no active threads modifying the bins.

    • setMaxBinAge

      public void setMaxBinAge(Long timePeriod, TimeUnit timeUnit)

    • getBinCount

      public int getBinCount()

    • add

      public void add(String groupIdentifier, org.apache.nifi.flowfile.FlowFile flowFile, RecordReader reader, org.apache.nifi.processor.ProcessSession session, boolean block) throws IOException
      Adds the given flowFiles to the first available bin in which it fits for the given group or creates a new bin in the specified group if necessary.

      Parameters:
      groupIdentifier - the group to which the flow file belongs; can be null
      flowFile - flowFile to bin
      reader - RecordReader to use for reading FlowFile
      session - the ProcessSession to which the FlowFiles belong
      block - if another thread is already writing to the desired bin, passing true for this parameter will block until the other thread(s) have finished so that the records can still be added to the desired bin. Passing false will result in moving on to another bin.
      Throws:
      IOException - if there is an IO problem reading from the stream or writing to the stream

    • createThresholds

      private RecordBinThresholds createThresholds(org.apache.nifi.flowfile.FlowFile flowfile)

    • completeOldestBin

      public void completeOldestBin() throws IOException
      Throws:
      IOException

    • completeExpiredBins

      public int completeExpiredBins() throws IOException
      Throws:
      IOException

    • completeFullEnoughBins

      public int completeFullEnoughBins() throws IOException
      Throws:
      IOException

    • completeFullBins

      public int completeFullBins() throws IOException
      Throws:
      IOException

    • handleCompletedBins

      private int handleCompletedBins(Predicate<RecordBin> completionTest, String completionReason) throws IOException
      Throws:
      IOException

    • removeBins

      private void removeBins(String key, List<RecordBin> bins)