Package org.apache.flink.streaming.connectors.elasticsearch

Interface ElasticsearchApiCallBridge<C extends AutoCloseable>

Type Parameters:

C - The Elasticsearch client, that implements AutoCloseable.

All Superinterfaces:

Serializable

@Internal
public interface ElasticsearchApiCallBridge<C extends AutoCloseable>
extends Serializable

An ElasticsearchApiCallBridge is used to bridge incompatible Elasticsearch Java API calls across different versions. This includes calls to create Elasticsearch clients, handle failed item responses, etc. Any incompatible Elasticsearch Java APIs should be bridged using this interface.

Implementations are allowed to be stateful. For example, for Elasticsearch 1.x, since connecting via an embedded node is allowed, the call bridge will hold reference to the created embedded node. Each instance of the sink will hold exactly one instance of the call bridge, and state cleanup is performed when the sink is closed.

Method Summary

Modifier and Type	Method	Description
default void	cleanup()	Perform any necessary state cleanup.
void	configureBulkProcessorBackoff(org.elasticsearch.action.bulk.BulkProcessor.Builder builder, ElasticsearchSinkBase.BulkFlushBackoffPolicy flushBackoffPolicy)	Set backoff-related configurations on the provided BulkProcessor.Builder.
void	configureBulkProcessorFlushInterval(org.elasticsearch.action.bulk.BulkProcessor.Builder builder, long flushIntervalMillis)	Sets the bulk flush interval, in milliseconds on the provided BulkProcessor.Builder.
org.elasticsearch.action.bulk.BulkProcessor.Builder	createBulkProcessorBuilder(C client, org.elasticsearch.action.bulk.BulkProcessor.Listener listener)	Creates a BulkProcessor.Builder for creating the bulk processor.
RequestIndexer	createBulkProcessorIndexer(org.elasticsearch.action.bulk.BulkProcessor bulkProcessor, boolean flushOnCheckpoint, AtomicLong numPendingRequestsRef)	Creates a RequestIndexer that is able to work with BulkProcessor binary compatible.
C	createClient()	Creates an Elasticsearch client implementing AutoCloseable.
Throwable	extractFailureCauseFromBulkItemResponse(org.elasticsearch.action.bulk.BulkItemResponse bulkItemResponse)	Extracts the cause of failure of a bulk item action.
void	verifyClientConnection(C client)	Verify the client connection by making a test request/ping to the Elasticsearch cluster.

Method Detail

createClient

C createClient()

Creates an Elasticsearch client implementing AutoCloseable.

Returns:

The created client.

createBulkProcessorBuilder

org.elasticsearch.action.bulk.BulkProcessor.Builder createBulkProcessorBuilder(C client,
                                                                               org.elasticsearch.action.bulk.BulkProcessor.Listener listener)

Creates a BulkProcessor.Builder for creating the bulk processor.

Parameters:

client - the Elasticsearch client.

listener - the bulk processor listener.

Returns:

the bulk processor builder.

extractFailureCauseFromBulkItemResponse

@Nullable
Throwable extractFailureCauseFromBulkItemResponse(org.elasticsearch.action.bulk.BulkItemResponse bulkItemResponse)

Extracts the cause of failure of a bulk item action.

Parameters:

bulkItemResponse - the bulk item response to extract cause of failure

Returns:

the extracted Throwable from the response (null is the response is successful).

configureBulkProcessorFlushInterval

void configureBulkProcessorFlushInterval(org.elasticsearch.action.bulk.BulkProcessor.Builder builder,
                                         long flushIntervalMillis)

Sets the bulk flush interval, in milliseconds on the provided BulkProcessor.Builder. The builder will be later on used to instantiate the actual BulkProcessor.

Parameters:

builder - the BulkProcessor.Builder to configure.

flushIntervalMillis - the flush interval in milliseconds.

configureBulkProcessorBackoff

void configureBulkProcessorBackoff(org.elasticsearch.action.bulk.BulkProcessor.Builder builder,
                                   @Nullable
                                   ElasticsearchSinkBase.BulkFlushBackoffPolicy flushBackoffPolicy)

Set backoff-related configurations on the provided BulkProcessor.Builder. The builder will be later on used to instantiate the actual BulkProcessor.

Parameters:

builder - the BulkProcessor.Builder to configure.

flushBackoffPolicy - user-provided backoff retry settings (null if the user disabled backoff retries).

verifyClientConnection

void verifyClientConnection(C client)
                     throws IOException

Verify the client connection by making a test request/ping to the Elasticsearch cluster.

Called by ElasticsearchSinkBase.open(org.apache.flink.configuration.Configuration) after creating the client. This makes sure the underlying client is closed if the connection is not successful and preventing thread leak.

Parameters:

client - the Elasticsearch client.

Throws:

IOException

createBulkProcessorIndexer

RequestIndexer createBulkProcessorIndexer(org.elasticsearch.action.bulk.BulkProcessor bulkProcessor,
                                          boolean flushOnCheckpoint,
                                          AtomicLong numPendingRequestsRef)

Creates a RequestIndexer that is able to work with BulkProcessor binary compatible.

cleanup

default void cleanup()

Perform any necessary state cleanup.