com.amazonaws.services.kinesis.connectors.redshift

Class RedshiftBasicEmitter

java.lang.Object

com.amazonaws.services.kinesis.connectors.s3.S3Emitter

com.amazonaws.services.kinesis.connectors.redshift.RedshiftBasicEmitter

All Implemented Interfaces:

IEmitter<byte[]>

public class RedshiftBasicEmitter
extends S3Emitter

This class is an implementation of IEmitter that emits records into Amazon Redshift one by one. It utilizes the Amazon Redshift copy command on each file by first inserting records into Amazon S3 and then performing the Amazon Redshift copy command. Amazon S3 insertion is done by extending the Amazon S3 emitter.

* This class requires the configuration of an Amazon S3 bucket and endpoint, as well as the following Amazon Redshift items:

• Redshift URL
• username and password
• data table and key column (data table stores items from the manifest copy)
• file table and key column (file table is used to store file names to prevent duplicate entries)
• the delimiter used for string parsing when inserting entries into Redshift

NOTE: The Amazon S3 bucket and the Amazon Redshift cluster need to be in the same region.

Field Summary

Fields inherited from class com.amazonaws.services.kinesis.connectors.s3.S3Emitter

s3Bucket, s3client, s3Endpoint

Constructor Summary

Constructors

Constructor and Description
RedshiftBasicEmitter(KinesisConnectorConfiguration configuration)

Method Summary

Modifier and Type	Method and Description
List<byte[]>	emit(UnmodifiableBuffer<byte[]> buffer) Invoked when the buffer is full.
void	fail(List<byte[]> records) This method defines how to handle a set of records that cannot successfully be emitted.
protected String	generateCopyStatement(String s3File)
void	shutdown() This method is called when the KinesisConnectorRecordProcessor is shutdown.

Methods inherited from class com.amazonaws.services.kinesis.connectors.s3.S3Emitter

getS3FileName, getS3URI

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

RedshiftBasicEmitter

public RedshiftBasicEmitter(KinesisConnectorConfiguration configuration)

Method Detail

emit

public List<byte[]> emit(UnmodifiableBuffer<byte[]> buffer)
                  throws IOException

Description copied from interface: IEmitter

Invoked when the buffer is full. This method emits the set of filtered records. It should return a list of records that were not emitted successfully. Returning Collections.emptyList() is considered a success.

Specified by:

emit in interface IEmitter<byte[]>

Overrides:

emit in class S3Emitter

Parameters:

buffer - The full buffer of records

Returns:

A list of records that failed to emit to be retried

Throws:

IOException - A failure was reached that is not recoverable, no retry will occur and the fail method will be called

fail

public void fail(List<byte[]> records)

Description copied from interface: IEmitter

This method defines how to handle a set of records that cannot successfully be emitted.

Specified by:

fail in interface IEmitter<byte[]>

Overrides:

fail in class S3Emitter

Parameters:

records - a list of records that were not successfully emitted

shutdown

public void shutdown()

Description copied from interface: IEmitter

This method is called when the KinesisConnectorRecordProcessor is shutdown. It should close any existing client connections.

Specified by:

shutdown in interface IEmitter<byte[]>

Overrides:

shutdown in class S3Emitter

generateCopyStatement

protected String generateCopyStatement(String s3File)