org.datavec.api.split

Class NumberedFileInputSplit

java.lang.Object

org.datavec.api.split.NumberedFileInputSplit

All Implemented Interfaces:

Serializable, InputSplit, Writable

public class NumberedFileInputSplit
extends Object
implements InputSplit

InputSplit for sequences of numbered files. Example usages:
Suppose files are sequenced according to "myFile_100.txt", "myFile_101.txt", ..., "myFile_200.txt" then use new NumberedFileInputSplit("myFile_%d.txt",100,200) NumberedFileInputSplit utilizes String.format(), hence the requirement for "%d" to represent the integer index.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
NumberedFileInputSplit(String baseString, int minIdxInclusive, int maxIdxInclusive)

Method Summary

Modifier and Type	Method and Description
WritableType	getType()
long	length() Length of the split
URI[]	locations() Locations of the splits
Iterator<URI>	locationsIterator()
Iterator<String>	locationsPathIterator()
void	readFields(DataInput in) Deserialize the fields of this object from in.
void	reset() Reset the InputSplit without reinitializing it from scratch.
double	toDouble() Convert Writable to double.
float	toFloat() Convert Writable to float.
int	toInt() Convert Writable to int.
long	toLong() Convert Writable to long.
void	write(DataOutput out) Serialize the fields of this object to out.
void	writeType(DataOutput out) Write the type (a single short value) to the DataOutput.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NumberedFileInputSplit

public NumberedFileInputSplit(String baseString,
                              int minIdxInclusive,
                              int maxIdxInclusive)

Parameters:

baseString - String that defines file format. Must contain "%d", which will be replaced with the index of the file.

minIdxInclusive - Minimum index/number (starting number in sequence of files, inclusive)

maxIdxInclusive - Maximum index/number (last number in sequence of files, inclusive)

Method Detail

length

public long length()

Description copied from interface: InputSplit

Length of the split

Specified by:

length in interface InputSplit

Returns:

locations

public URI[] locations()

Description copied from interface: InputSplit

Locations of the splits

Specified by:

locations in interface InputSplit

Returns:

locationsIterator

public Iterator<URI> locationsIterator()

Specified by:

locationsIterator in interface InputSplit

locationsPathIterator

public Iterator<String> locationsPathIterator()

Specified by:

locationsPathIterator in interface InputSplit

reset

public void reset()

Description copied from interface: InputSplit

Reset the InputSplit without reinitializing it from scratch. In many cases, this is a no-op. For InputSplits that have randomization: reset should shuffle the order.

Specified by:

reset in interface InputSplit

write

public void write(DataOutput out)
           throws IOException

Description copied from interface: Writable

Serialize the fields of this object to out.

Specified by:

write in interface Writable

Parameters:

out - DataOuput to serialize this object into.

Throws:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

Description copied from interface: Writable

Deserialize the fields of this object from in.

For efficiency, implementations should attempt to re-use storage in the existing object where possible.

Specified by:

readFields in interface Writable

Parameters:

in - DataInput to deseriablize this object from.

Throws:

IOException

toDouble

public double toDouble()

Description copied from interface: Writable

Convert Writable to double. Whether this is supported depends on the specific writable.

Specified by:

toDouble in interface Writable

toFloat

public float toFloat()

Description copied from interface: Writable

Convert Writable to float. Whether this is supported depends on the specific writable.

Specified by:

toFloat in interface Writable

toInt

public int toInt()

Description copied from interface: Writable

Convert Writable to int. Whether this is supported depends on the specific writable.

Specified by:

toInt in interface Writable

toLong

public long toLong()

Description copied from interface: Writable

Convert Writable to long. Whether this is supported depends on the specific writable.

Specified by:

toLong in interface Writable

getType

public WritableType getType()

Specified by:

getType in interface Writable

writeType

public void writeType(DataOutput out)
               throws IOException

Description copied from interface: Writable

Write the type (a single short value) to the DataOutput. See WritableFactory for details.

Specified by:

writeType in interface Writable

Parameters:

out - DataOutput to write to

Throws:

IOException - For errors during writing