com.addthis.muxy

Class MuxStreamDirectory

java.lang.Object

com.addthis.muxy.ReadMuxStreamDirectory

com.addthis.muxy.MuxStreamDirectory

public class MuxStreamDirectory
extends ReadMuxStreamDirectory

stream multiplexer. allows for a large number of append-only streams to exist inside of a much smaller number of on-disk files. files consist of a series of fast-skip blocks. each block contains a linked list of bytes for 1 or more streams.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected class	MuxStreamDirectory.StreamOut
protected class	MuxStreamDirectory.StreamOutWriter

Field Summary

Fields

Modifier and Type	Field and Description
static FileAttribute<?>	DATA_FILE_ATTRIBUTES
protected Map<Integer,MuxStreamDirectory.StreamOut>	openStreamWrites
protected AtomicLong	openWriteBytes
protected FileChannel	openWriteFile
protected ReentrantLock	openWritesLock
protected Map<Integer,MuxStreamDirectory.StreamOut>	pendingStreamCloses

Fields inherited from class com.addthis.muxy.ReadMuxStreamDirectory

closeTime, DEFAULT_MAP_SIZE, deleteFreed, dirDataFile, dirMetaFile, eventListener, fileFormat, MAX_RECORDS_READ, releaseComplete, startFile, streamDirectory, streamDirectoryConfig, streamDirectoryMap, writeMutexFile, writeMutexLock

Constructor Summary

Constructors

Constructor and Description
MuxStreamDirectory(Path dir, MuxyEventListener listener)

Method Summary

Modifier and Type	Method and Description
protected void	acquireWritable()
OutputStream	appendStream(MuxStream meta)
protected int	bumpCurrentFile()
protected void	compactMetaLog() write out a new meta log from in-memory map.
MuxStream	createStream()
protected MuxStream	deleteStream(int streamID)
MuxStream	findStream(int streamID)
Collection<Path>	getActiveFiles()
boolean	isWritingComplete()
Collection<MuxStream>	listStreams()
protected boolean	maybeTrimOutputBuffers() Trims memory overhead if the openWritesLock is immediately available.
protected int	reserveStreamID()
void	setDeleteFreed(boolean deleteFreed) automatically delete files no longer referenced by any streams.
void	setMaxBlockSize(int size)
void	setMaxFileSize(int size)
int	size()
void	waitForWriteClosure() this method MUST be called when an application that performs writes is done with this class.
protected long	writeStreamsToBlock()

Methods inherited from class com.addthis.muxy.ReadMuxStreamDirectory

formatFileName, getFileByID, getFileByID, publishEvent, readMetaLog

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DATA_FILE_ATTRIBUTES

public static final FileAttribute<?> DATA_FILE_ATTRIBUTES

openWritesLock

protected final ReentrantLock openWritesLock

openStreamWrites

protected final Map<Integer,MuxStreamDirectory.StreamOut> openStreamWrites

pendingStreamCloses

protected final Map<Integer,MuxStreamDirectory.StreamOut> pendingStreamCloses

openWriteBytes

protected final AtomicLong openWriteBytes

openWriteFile

protected FileChannel openWriteFile

Constructor Detail

MuxStreamDirectory

public MuxStreamDirectory(Path dir,
                          MuxyEventListener listener)
                   throws Exception

Throws:

Exception

Method Detail

setDeleteFreed

public void setDeleteFreed(boolean deleteFreed)

automatically delete files no longer referenced by any streams.

isWritingComplete

public boolean isWritingComplete()

reserveStreamID

protected int reserveStreamID()
                       throws IOException

Throws:

IOException

bumpCurrentFile

protected int bumpCurrentFile()
                       throws IOException

Throws:

IOException

setMaxBlockSize

public void setMaxBlockSize(int size)
                     throws IOException

Throws:

IOException

setMaxFileSize

public void setMaxFileSize(int size)
                    throws IOException

Throws:

IOException

waitForWriteClosure

public void waitForWriteClosure()

this method MUST be called when an application that performs writes is done with this class. it ensures that the file meta-data has been properly compacted and written back out to disk.

acquireWritable

protected void acquireWritable()
                        throws IOException

Throws:

IOException

compactMetaLog

protected void compactMetaLog()
                       throws IOException

write out a new meta log from in-memory map. must be called with write lock held and in a sync block on openStreamWrites

Throws:

IOException

createStream

public MuxStream createStream()
                       throws IOException

Throws:

IOException

listStreams

public Collection<MuxStream> listStreams()
                                  throws IOException

Overrides:

listStreams in class ReadMuxStreamDirectory

Throws:

IOException

size

public int size()

Overrides:

size in class ReadMuxStreamDirectory

findStream

public MuxStream findStream(int streamID)
                     throws IOException

Overrides:

findStream in class ReadMuxStreamDirectory

Throws:

IOException

getActiveFiles

public Collection<Path> getActiveFiles()
                                throws IOException

Overrides:

getActiveFiles in class ReadMuxStreamDirectory

Throws:

IOException

deleteStream

protected MuxStream deleteStream(int streamID)
                          throws IOException

Throws:

IOException

appendStream

public OutputStream appendStream(MuxStream meta)
                          throws IOException

Throws:

IOException

maybeTrimOutputBuffers

protected boolean maybeTrimOutputBuffers()

Trims memory overhead if the openWritesLock is immediately available.

writeStreamsToBlock

protected long writeStreamsToBlock()
                            throws IOException

Throws:

IOException