org.datavec.api.records.writer.impl.misc

Class SVMLightRecordWriter

java.lang.Object

org.datavec.api.records.writer.impl.FileRecordWriter

org.datavec.api.records.writer.impl.misc.SVMLightRecordWriter

All Implemented Interfaces:

Closeable, AutoCloseable, Configurable, RecordWriter

Direct Known Subclasses:

LibSvmRecordWriter

public class SVMLightRecordWriter
extends FileRecordWriter

Record writer for SVMLight format, which can generally be described as LABEL INDEX:VALUE INDEX:VALUE ... SVMLight format is well-suited to sparse data (e.g., bag-of-words) because it omits all features with value zero. We support an "extended" version that allows for multiple targets (or labels) separated by a comma, as follows: LABEL1,LABEL2,... INDEX:VALUE INDEX:VALUE ... This can be used to represent either multitask problems or multilabel problems with sparse binary labels (controlled via the "MULTILABEL" configuration option). Like scikit-learn, we support both zero-based and one-based indexing. Further details on the format can be found at
- http://svmlight.joachims.org/
- http://www.csie.ntu.edu.tw/~cjlin/libsvmtools/datasets/multilabel.html
- http://scikit-learn.org/stable/modules/generated/sklearn.datasets.load_svmlight_file.html

Author:

Adam Gibson (original), Josh Patterson, dave@skymind.io

Field Summary

Fields

Modifier and Type	Field and Description
static String	FEATURE_FIRST_COLUMN
static String	FEATURE_LAST_COLUMN
protected int	featureFirstColumn
protected int	featureLastColumn
static String	HAS_LABELS
protected boolean	hasLabel
static String	LABEL_FIRST_COLUMN
static String	LABEL_LAST_COLUMN
protected int	labelFirstColumn
protected int	labelLastColumn
protected boolean	multilabel
static String	MULTILABEL
static String	NAME_SPACE
static String	UNLABELED
static String	ZERO_BASED_INDEXING
static String	ZERO_BASED_LABEL_INDEXING
protected boolean	zeroBasedIndexing
protected boolean	zeroBasedLabelIndexing

Fields inherited from class org.datavec.api.records.writer.impl.FileRecordWriter

conf, DEFAULT_CHARSET, encoding, NEW_LINE, out, partitioner

Fields inherited from interface org.datavec.api.records.writer.RecordWriter

APPEND

Constructor Summary

Constructors

Constructor and Description
SVMLightRecordWriter()

Method Summary

Modifier and Type	Method and Description
void	setConf(Configuration conf) Set DataVec configuration
boolean	supportsBatch() Returns true if this record writer supports efficient batch writing using RecordWriter.writeBatch(List)
PartitionMetaData	write(List<Writable> record) Write next record.
PartitionMetaData	writeBatch(List<List<Writable>> batch) Write a batch of records

Methods inherited from class org.datavec.api.records.writer.impl.FileRecordWriter

close, getConf, initialize, initialize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NAME_SPACE

public static final String NAME_SPACE

FEATURE_FIRST_COLUMN

public static final String FEATURE_FIRST_COLUMN

FEATURE_LAST_COLUMN

public static final String FEATURE_LAST_COLUMN

ZERO_BASED_INDEXING

public static final String ZERO_BASED_INDEXING

ZERO_BASED_LABEL_INDEXING

public static final String ZERO_BASED_LABEL_INDEXING

HAS_LABELS

public static final String HAS_LABELS

MULTILABEL

public static final String MULTILABEL

LABEL_FIRST_COLUMN

public static final String LABEL_FIRST_COLUMN

LABEL_LAST_COLUMN

public static final String LABEL_LAST_COLUMN

UNLABELED

public static final String UNLABELED

See Also:

Constant Field Values

featureFirstColumn

protected int featureFirstColumn

featureLastColumn

protected int featureLastColumn

zeroBasedIndexing

protected boolean zeroBasedIndexing

zeroBasedLabelIndexing

protected boolean zeroBasedLabelIndexing

hasLabel

protected boolean hasLabel

multilabel

protected boolean multilabel

labelFirstColumn

protected int labelFirstColumn

labelLastColumn

protected int labelLastColumn

Constructor Detail

SVMLightRecordWriter

public SVMLightRecordWriter()

Method Detail

setConf

public void setConf(Configuration conf)

Set DataVec configuration

Specified by:

setConf in interface Configurable

Overrides:

setConf in class FileRecordWriter

Parameters:

conf -

supportsBatch

public boolean supportsBatch()

Description copied from interface: RecordWriter

Returns true if this record writer supports efficient batch writing using RecordWriter.writeBatch(List)

Returns:

write

public PartitionMetaData write(List<Writable> record)
                        throws IOException

Write next record.

Parameters:

record -

Throws:

IOException

writeBatch

public PartitionMetaData writeBatch(List<List<Writable>> batch)
                             throws IOException

Description copied from interface: RecordWriter

Write a batch of records

Parameters:

batch - the batch to write

Throws:

IOException