won.utils.batch

Class BatchingConsumer<K,I>

java.lang.Object

won.utils.batch.BatchingConsumer<K,I>

public class BatchingConsumer<K,I>
extends java.lang.Object

Accepts items together with a key for grouping them, a Consumer<Collection<Object>> for consuming them. The consumer's purpose is to balance the following goals

• the consumer should be called as soon as possible
• frequent invocations of the consumer should be avoided

The BatchingConsumer passed items via its accept() method. It consumes them in 'chunks' of one or multiple items at a time. The following properties can be set for each key to govern this behaviour (higher up on this list is more imporant):

• consumeFirst (boolean): if set, the filter consumes the accepted item immediately if there is not yet a batch for that key. However, the batch is created and subsequent items with the same key are collected in the batch.
• maxBatchAge (Duration): the whole batch cannot become older than this value. When it is reached, the whole batch is consumed.
• minChunkInterval (Duration): wait at least for this duration between chunks
• maxItemInterval (Duration): consume the current chunk when the instant the last item was accepted is longer ago than this value.
• maxBatchSize (integer): consume whole batch if its size has reached this value

Author:

fkleedorfer

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BatchingConsumer.Config
static class	BatchingConsumer.ConfigBuilder

Constructor Summary

Constructors

Constructor and Description
BatchingConsumer()
BatchingConsumer(BatchingConsumer.Config config)

Method Summary

Modifier and Type	Method and Description
void	accept(K key, I item, java.util.function.Consumer<java.util.Collection<I>> consumer)
void	accept(K key, I item, java.util.function.Consumer<java.util.Collection<I>> consumer, BatchingConsumer.Config config)
void	accept(K key, I item, java.util.Optional<java.lang.String> deduplicationKey, java.util.function.Consumer<java.util.Collection<I>> consumer, java.util.Optional<BatchingConsumer.Config> config) Offer an item to the BatchingConsumer, providing an optional deduplicationKey, a consumer and an optional config.
void	accept(K key, I item, java.lang.String deduplicationKey, java.util.function.Consumer<java.util.Collection<I>> consumer)
void	accept(K key, I item, java.lang.String deduplicationKey, java.util.function.Consumer<java.util.Collection<I>> consumer, BatchingConsumer.Config config)
void	cancelBatch(K key) Removes the batch, the consumer is not invoked.
void	consumeAllBatches() Invokes the consumers for all batches immediately and removes all batches.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchingConsumer

public BatchingConsumer()

BatchingConsumer

public BatchingConsumer(BatchingConsumer.Config config)

Method Detail

accept

public void accept(K key,
                   I item,
                   java.util.function.Consumer<java.util.Collection<I>> consumer)

accept

public void accept(K key,
                   I item,
                   java.util.function.Consumer<java.util.Collection<I>> consumer,
                   BatchingConsumer.Config config)

accept

public void accept(K key,
                   I item,
                   java.lang.String deduplicationKey,
                   java.util.function.Consumer<java.util.Collection<I>> consumer)

accept

public void accept(K key,
                   I item,
                   java.lang.String deduplicationKey,
                   java.util.function.Consumer<java.util.Collection<I>> consumer,
                   BatchingConsumer.Config config)

accept

public void accept(K key,
                   I item,
                   java.util.Optional<java.lang.String> deduplicationKey,
                   java.util.function.Consumer<java.util.Collection<I>> consumer,
                   java.util.Optional<BatchingConsumer.Config> config)

Offer an item to the BatchingConsumer, providing an optional deduplicationKey, a consumer and an optional config. The consumer must be specified just in case a new Batch is created in this invocation. If that happens, the consumer is used, otherwise it will not be. The config is optional because the BatchingConsumer has a defaultConfig that is used if no config is specified. Again, the config presented when a new Batch is instantiated will be used. The deduplicationKey is used to avoid adding the same item multiple times: if accept() is called multiple times with the same deduplicationKey, only the first one is actually accepted, the subsequent ones are dropped silently. Note: There is a small chance that an item with a deduplicationKey will be dropped despite being the first one with that key.

Parameters:

key -

item -

consumer -

config -

consumeAllBatches

public void consumeAllBatches()

Invokes the consumers for all batches immediately and removes all batches.

cancelBatch

public void cancelBatch(K key)

Removes the batch, the consumer is not invoked.

Parameters:

key -