org.datavec.api.records.reader.impl.csv

Class CSVLineSequenceRecordReader

java.lang.Object

org.datavec.api.records.reader.BaseRecordReader

org.datavec.api.records.reader.impl.LineRecordReader

org.datavec.api.records.reader.impl.csv.CSVRecordReader

org.datavec.api.records.reader.impl.csv.CSVLineSequenceRecordReader

All Implemented Interfaces:

Closeable, Serializable, AutoCloseable, Configurable, RecordReader, SequenceRecordReader

public class CSVLineSequenceRecordReader
extends CSVRecordReader
implements SequenceRecordReader

CSVLineSequenceRecordReader: Used for loading univariance (single valued) sequences from a CSV, where each line in a CSV represents an independent sequence, and each sequence has exactly 1 value per time step.
For example, a CSV file with content:

 a,b,c
 1,2,3,4
 

will produce two sequences, both with one value per time step; one of length 3 (values a, b, then c for the 3 time steps respectively) and one of length 4 (values 1, 2, 3, then 4 for each of the 4 time steps respectively)

Author:

Alex Black

See Also:

Serialized Form

Field Summary

Fields inherited from class org.datavec.api.records.reader.impl.csv.CSVRecordReader

DEFAULT_DELIMITER, DEFAULT_QUOTE, DELIMITER, QUOTE, SKIP_NUM_LINES, skipNumLines

Fields inherited from class org.datavec.api.records.reader.impl.LineRecordReader

charset, conf, initialized, lineIndex, locations, splitIndex

Fields inherited from class org.datavec.api.records.reader.BaseRecordReader

inputSplit, listeners, streamCreatorFn

Fields inherited from interface org.datavec.api.records.reader.RecordReader

APPEND_LABEL, LABELS, NAME_SPACE

Constructor Summary

Constructors

Constructor and Description
CSVLineSequenceRecordReader() Default settings: skip 0 lines, use ',' as the delimiter, and '"' for quotes
CSVLineSequenceRecordReader(int skipNumLines, char delimiter) Skip lines and use delimiter
CSVLineSequenceRecordReader(int skipNumLines, char delimiter, char quote) Skip lines, use delimiter, and strip quotes

Method Summary

Modifier and Type	Method and Description
protected SequenceRecord	convert(Record r)
List<SequenceRecord>	loadSequenceFromMetaData(List<RecordMetaData> recordMetaDatas) Load multiple sequence records from the given a list of RecordMetaData instances
SequenceRecord	loadSequenceFromMetaData(RecordMetaData recordMetaData) Load a single sequence record from the given RecordMetaData instance Note: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using SequenceRecordReader.loadSequenceFromMetaData(List)
SequenceRecord	nextSequence() Similar to SequenceRecordReader.sequenceRecord(), but returns a Record object, that may include metadata such as the source of the data
List<List<Writable>>	sequenceRecord() Returns a sequence record.
List<List<Writable>>	sequenceRecord(URI uri, DataInputStream dataInputStream) Load a sequence record from the given DataInputStream Unlike RecordReader.next() the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStream

Methods inherited from class org.datavec.api.records.reader.impl.csv.CSVRecordReader

batchesSupported, hasNext, initialize, loadFromMetaData, loadFromMetaData, next, next, nextRecord, onLocationOpen, parseLine, readStringLine, record, reset

Methods inherited from class org.datavec.api.records.reader.impl.LineRecordReader

close, closeIfRequired, getConf, getIterator, getLabels, initialize, resetSupported, setConf

Methods inherited from class org.datavec.api.records.reader.BaseRecordReader

getListeners, invokeListeners, setListeners, setListeners

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.datavec.api.records.reader.RecordReader

batchesSupported, getLabels, getListeners, hasNext, initialize, initialize, loadFromMetaData, loadFromMetaData, next, next, nextRecord, record, reset, resetSupported, setListeners, setListeners

Methods inherited from interface java.io.Closeable

close

Methods inherited from interface org.datavec.api.conf.Configurable

getConf, setConf

Constructor Detail

CSVLineSequenceRecordReader

public CSVLineSequenceRecordReader()

Default settings: skip 0 lines, use ',' as the delimiter, and '"' for quotes

CSVLineSequenceRecordReader

public CSVLineSequenceRecordReader(int skipNumLines,
                                   char delimiter)

Skip lines and use delimiter

Parameters:

skipNumLines - the number of lines to skip

delimiter - the delimiter

CSVLineSequenceRecordReader

public CSVLineSequenceRecordReader(int skipNumLines,
                                   char delimiter,
                                   char quote)

Skip lines, use delimiter, and strip quotes

Parameters:

skipNumLines - the number of lines to skip

delimiter - the delimiter

quote - the quote to strip

Method Detail

sequenceRecord

public List<List<Writable>> sequenceRecord()

Description copied from interface: SequenceRecordReader

Returns a sequence record.

Specified by:

sequenceRecord in interface SequenceRecordReader

Returns:

a sequence of records

sequenceRecord

public List<List<Writable>> sequenceRecord(URI uri,
                                           DataInputStream dataInputStream)
                                    throws IOException

Description copied from interface: SequenceRecordReader

Load a sequence record from the given DataInputStream Unlike RecordReader.next() the internal state of the RecordReader is not modified Implementations of this method should not close the DataInputStream

Specified by:

sequenceRecord in interface SequenceRecordReader

Throws:

IOException - if error occurs during reading from the input stream

nextSequence

public SequenceRecord nextSequence()

Description copied from interface: SequenceRecordReader

Similar to SequenceRecordReader.sequenceRecord(), but returns a Record object, that may include metadata such as the source of the data

Specified by:

nextSequence in interface SequenceRecordReader

Returns:

next sequence record

loadSequenceFromMetaData

public SequenceRecord loadSequenceFromMetaData(RecordMetaData recordMetaData)
                                        throws IOException

Description copied from interface: SequenceRecordReader

Load a single sequence record from the given RecordMetaData instance
Note: that for data that isn't splittable (i.e., text data that needs to be scanned/split), it is more efficient to load multiple records at once using SequenceRecordReader.loadSequenceFromMetaData(List)

Specified by:

loadSequenceFromMetaData in interface SequenceRecordReader

Parameters:

recordMetaData - Metadata for the sequence record that we want to load from

Returns:

Single sequence record for the given RecordMetaData instance

Throws:

IOException - If I/O error occurs during loading

loadSequenceFromMetaData

public List<SequenceRecord> loadSequenceFromMetaData(List<RecordMetaData> recordMetaDatas)
                                              throws IOException

Description copied from interface: SequenceRecordReader

Load multiple sequence records from the given a list of RecordMetaData instances

Specified by:

loadSequenceFromMetaData in interface SequenceRecordReader

Parameters:

recordMetaDatas - Metadata for the records that we want to load from

Returns:

Multiple sequence record for the given RecordMetaData instances

Throws:

IOException - If I/O error occurs during loading

convert

protected SequenceRecord convert(Record r)