org.datavec.api.split

Class BaseInputSplit

java.lang.Object

org.datavec.api.split.BaseInputSplit

All Implemented Interfaces:

Serializable, InputSplit, Writable

Direct Known Subclasses:

CollectionInputSplit, FileSplit, TransformSplit

public abstract class BaseInputSplit
extends Object
implements InputSplit

Base input split

Author:

Adam Gibson

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected int[]	iterationOrder
protected long	length
protected List<String>	uriStrings

Constructor Summary

Constructors

Constructor and Description
BaseInputSplit()

Method Summary

Modifier and Type	Method and Description
WritableType	getType()
long	length() Length of the split
URI[]	locations() Locations of the splits
Iterator<URI>	locationsIterator()
Iterator<String>	locationsPathIterator()
InputSplit[]	sample(PathFilter pathFilter, double... weights) Samples the locations based on the PathFilter and splits the result into an array of InputSplit objects, with sizes proportional to the weights.
double	toDouble() Convert Writable to double.
float	toFloat() Convert Writable to float.
int	toInt() Convert Writable to int.
long	toLong() Convert Writable to long.
void	writeType(DataOutput out) Write the type (a single short value) to the DataOutput.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.datavec.api.split.InputSplit

reset

Methods inherited from interface org.datavec.api.writable.Writable

readFields, write

Field Detail

uriStrings

protected List<String> uriStrings

iterationOrder

protected int[] iterationOrder

length

protected long length

Constructor Detail

BaseInputSplit

public BaseInputSplit()

Method Detail

locations

public URI[] locations()

Description copied from interface: InputSplit

Locations of the splits

Specified by:

locations in interface InputSplit

Returns:

locationsIterator

public Iterator<URI> locationsIterator()

Specified by:

locationsIterator in interface InputSplit

locationsPathIterator

public Iterator<String> locationsPathIterator()

Specified by:

locationsPathIterator in interface InputSplit

length

public long length()

Description copied from interface: InputSplit

Length of the split

Specified by:

length in interface InputSplit

Returns:

toDouble

public double toDouble()

Description copied from interface: Writable

Convert Writable to double. Whether this is supported depends on the specific writable.

Specified by:

toDouble in interface Writable

toFloat

public float toFloat()

Description copied from interface: Writable

Convert Writable to float. Whether this is supported depends on the specific writable.

Specified by:

toFloat in interface Writable

toInt

public int toInt()

Description copied from interface: Writable

Convert Writable to int. Whether this is supported depends on the specific writable.

Specified by:

toInt in interface Writable

toLong

public long toLong()

Description copied from interface: Writable

Convert Writable to long. Whether this is supported depends on the specific writable.

Specified by:

toLong in interface Writable

writeType

public void writeType(DataOutput out)
               throws IOException

Description copied from interface: Writable

Write the type (a single short value) to the DataOutput. See WritableFactory for details.

Specified by:

writeType in interface Writable

Parameters:

out - DataOutput to write to

Throws:

IOException - For errors during writing

sample

public InputSplit[] sample(PathFilter pathFilter,
                           double... weights)

Samples the locations based on the PathFilter and splits the result into an array of InputSplit objects, with sizes proportional to the weights.

Parameters:

pathFilter - to modify the locations in some way (null == as is)

weights - to split the locations into multiple InputSplit

Returns:

the sampled locations

getType

public WritableType getType()

Specified by:

getType in interface Writable