com.amazonaws.services.kinesis.connectors.dynamodb

Class DynamoDBEmitter

java.lang.Object

com.amazonaws.services.kinesis.connectors.dynamodb.DynamoDBEmitter

All Implemented Interfaces:

IEmitter<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>>

public class DynamoDBEmitter
extends Object
implements IEmitter<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>>

This class is used to store records from a stream in an Amazon DynamoDB table. It requires the use of a DynamoDBTransformer, which is able to transform records into a format that can be sent to Amazon DynamoDB. An Amazon DynamoDB client is used to perform batch requests on the contents of a buffer when emitting. This class requires the configuration of an Amazon DynamoDB endpoint and table name.

Field Summary

Fields

Modifier and Type	Field and Description
protected com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient	dynamoDBClient
protected String	dynamoDBEndpoint
protected String	dynamoDBTableName

Constructor Summary

Constructors

Constructor and Description
DynamoDBEmitter(KinesisConnectorConfiguration configuration)

Method Summary

Modifier and Type	Method and Description
List<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>>	emit(UnmodifiableBuffer<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> buffer) Invoked when the buffer is full.
void	fail(List<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> records) This method defines how to handle a set of records that cannot successfully be emitted.
void	shutdown() This method is called when the KinesisConnectorRecordProcessor is shutdown.
Set<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>>	uniqueItems(List<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> items) This helper method is used to dedupe a list of items.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

dynamoDBEndpoint

protected final String dynamoDBEndpoint

dynamoDBTableName

protected final String dynamoDBTableName

dynamoDBClient

protected final com.amazonaws.services.dynamodbv2.AmazonDynamoDBClient dynamoDBClient

Constructor Detail

DynamoDBEmitter

public DynamoDBEmitter(KinesisConnectorConfiguration configuration)

Method Detail

emit

public List<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> emit(UnmodifiableBuffer<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> buffer)
                                                                              throws IOException

Description copied from interface: IEmitter

Invoked when the buffer is full. This method emits the set of filtered records. It should return a list of records that were not emitted successfully. Returning Collections.emptyList() is considered a success.

Specified by:

emit in interface IEmitter<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>>

Parameters:

buffer - The full buffer of records

Returns:

A list of records that failed to emit to be retried

Throws:

IOException - A failure was reached that is not recoverable, no retry will occur and the fail method will be called

fail

public void fail(List<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> records)

Description copied from interface: IEmitter

This method defines how to handle a set of records that cannot successfully be emitted.

Specified by:

fail in interface IEmitter<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>>

Parameters:

records - a list of records that were not successfully emitted

uniqueItems

public Set<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> uniqueItems(List<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>> items)

This helper method is used to dedupe a list of items. Use this method to dedupe the contents of a buffer before performing an Amazon DynamoDB batch write request.

Parameters:

items - a list of Map items

Returns:

the subset of unique items

shutdown

public void shutdown()

Description copied from interface: IEmitter

This method is called when the KinesisConnectorRecordProcessor is shutdown. It should close any existing client connections.

Specified by:

shutdown in interface IEmitter<Map<String,com.amazonaws.services.dynamodbv2.model.AttributeValue>>