org.apache.nifi.controller.repository

Interface FlowFileSwapManager

public interface FlowFileSwapManager

Defines a mechanism by which FlowFiles can be move into external storage or memory so that they can be removed from the Java heap and vice-versa

Method Summary

Modifier and Type	Method and Description
String	changePartitionName(String swapLocation, String newPartitionName) Updates the name of the partition that owns a given swap file
String	getQueueIdentifier(String swapLocation) Returns the ID of the queue that the given swap file belongs to
Set<String>	getSwappedPartitionNames(FlowFileQueue queue) Determines the names of each of the Partitions for which there are swap files for the given queue
SwapSummary	getSwapSummary(String swapLocation) Parses the contents of the swap file at the given location and provides a SwapSummary that provides pertinent information about the information stored within the swap file
void	initialize(SwapManagerInitializationContext initializationContext) Initializes the Swap Manager, providing a SwapManagerInitializationContext so that the Swap Manager has access to all of the components necessary to perform its functions
SwapContents	peek(String swapLocation, FlowFileQueue flowFileQueue) Recovers the FlowFiles from the swap file that lives at the given location.
void	purge() Purge all known Swap Files without updating FlowFileRepository or Provenance Repository
List<String>	recoverSwapLocations(FlowFileQueue flowFileQueue, String partitionName) Determines swap files that exist for the given FlowFileQueue
SwapContents	swapIn(String swapLocation, FlowFileQueue flowFileQueue) Recovers the FlowFiles from the swap file that lives at the given location and belongs to the FlowFile Queue with the given identifier.
String	swapOut(List<FlowFileRecord> flowFiles, FlowFileQueue flowFileQueue, String partitionName) Swaps out the given FlowFiles that belong to the queue with the given identifier.

Method Detail

initialize

void initialize(SwapManagerInitializationContext initializationContext)

Initializes the Swap Manager, providing a SwapManagerInitializationContext so that the Swap Manager has access to all of the components necessary to perform its functions

Parameters:

initializationContext - the context the provides the swap manager with access to the resources that it needs to perform its functions

swapOut

String swapOut(List<FlowFileRecord> flowFiles,
               FlowFileQueue flowFileQueue,
               String partitionName)
        throws IOException

Swaps out the given FlowFiles that belong to the queue with the given identifier.

Parameters:

flowFiles - the FlowFiles to swap out to external storage

flowFileQueue - the queue that the FlowFiles belong to

partitionName - the name of the partition within the queue, or null if the queue is not partitioned

Returns:

the location of the externally stored swap file

Throws:

IOException - if unable to swap the FlowFiles out

peek

SwapContents peek(String swapLocation,
                  FlowFileQueue flowFileQueue)
           throws IncompleteSwapFileException,
                  IOException

Recovers the FlowFiles from the swap file that lives at the given location. This action provides a view of the FlowFiles but does not actively swap them in, meaning that the swap file at the given location remains in that location and the FlowFile Repository is not updated.

Parameters:

swapLocation - the location of the swap file

flowFileQueue - the queue that the FlowFiles belong to

Returns:

a SwapContents that includes the FlowFiles that live at the given swap location

Throws:

IOException - if unable to recover the FlowFiles from the given location

IncompleteSwapFileException

swapIn

SwapContents swapIn(String swapLocation,
                    FlowFileQueue flowFileQueue)
             throws IncompleteSwapFileException,
                    IOException

Recovers the FlowFiles from the swap file that lives at the given location and belongs to the FlowFile Queue with the given identifier. The FlowFile Repository is then updated and the swap file is permanently removed from the external storage

Parameters:

swapLocation - the location of the swap file

flowFileQueue - the queue to which the FlowFiles belong

Returns:

a SwapContents that includes FlowFiles that are stored in the given location

Throws:

IOException - if unable to recover the FlowFiles from the given location or update the FlowFileRepository

IncompleteSwapFileException

recoverSwapLocations

List<String> recoverSwapLocations(FlowFileQueue flowFileQueue,
                                  String partitionName)
                           throws IOException

Determines swap files that exist for the given FlowFileQueue

Parameters:

flowFileQueue - the queue for which the FlowFiles should be recovered

partitionName - the partition within the FlowFileQueue to recover, or null if the queue is not partitioned

Returns:

all swap locations that have been identified for the given queue, in the order that they should be swapped back in

Throws:

IOException

getSwappedPartitionNames

Set<String> getSwappedPartitionNames(FlowFileQueue queue)
                              throws IOException

Determines the names of each of the Partitions for which there are swap files for the given queue

Parameters:

queue - the queue to which the FlowFiles belong

Returns:

the Set of names of all Partitions for which there are swap files

Throws:

IOException - if unable to read the information from the underlying storage

changePartitionName

String changePartitionName(String swapLocation,
                           String newPartitionName)
                    throws IOException

Updates the name of the partition that owns a given swap file

Parameters:

swapLocation - the location of the swap file

newPartitionName - the new name of the new partition that owns the swap file

Returns:

the new swap location

Throws:

IOException - if unable to rename the swap file

getSwapSummary

SwapSummary getSwapSummary(String swapLocation)
                    throws IOException

Parses the contents of the swap file at the given location and provides a SwapSummary that provides pertinent information about the information stored within the swap file

Parameters:

swapLocation - the location of the swap file

Returns:

a SwapSummary that provides information about what is contained within the swap file

Throws:

IOException - if unable to read or parse the swap file

purge

void purge()

Purge all known Swap Files without updating FlowFileRepository or Provenance Repository

getQueueIdentifier

String getQueueIdentifier(String swapLocation)

Returns the ID of the queue that the given swap file belongs to

Parameters:

swapLocation - the swap location

Returns:

the ID of the queue, or null if unknown