org.datavec.api.transform.transform.nlp

Class TextToCharacterIndexTransform

java.lang.Object

org.datavec.api.transform.sequence.expansion.BaseSequenceExpansionTransform

org.datavec.api.transform.transform.nlp.TextToCharacterIndexTransform

All Implemented Interfaces:

Serializable, ColumnOp, Transform

public class TextToCharacterIndexTransform
extends BaseSequenceExpansionTransform

Convert each text value in a sequence to a longer sequence of integer indices. For example, "abc" would be converted to [1, 2, 3]. Values in other columns will be duplicated.

Author:

Alex Black

See Also:

Serialized Form

Field Summary

Fields inherited from class org.datavec.api.transform.sequence.expansion.BaseSequenceExpansionTransform

expandedColumnNames, inputSchema, requiredColumns

Constructor Summary

Constructors

Constructor and Description
TextToCharacterIndexTransform(String columnName, String newColumnName, Map<Character,Integer> characterIndexMap, boolean exceptionOnUnknown)

Method Summary

Modifier and Type	Method and Description
protected List<ColumnMetaData>	expandedColumnMetaDatas(List<ColumnMetaData> origColumnMeta, List<String> expandedColumnNames)
protected List<List<Writable>>	expandTimeStep(List<Writable> currentStepValues)

Methods inherited from class org.datavec.api.transform.sequence.expansion.BaseSequenceExpansionTransform

columnName, columnNames, map, map, mapSequence, mapSequence, outputColumnName, outputColumnNames, transform

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.datavec.api.transform.ColumnOp

getInputSchema, setInputSchema

Constructor Detail

TextToCharacterIndexTransform

public TextToCharacterIndexTransform(String columnName,
                                     String newColumnName,
                                     Map<Character,Integer> characterIndexMap,
                                     boolean exceptionOnUnknown)

Parameters:

columnName - Name of the text column

newColumnName - Name of the column after expansion

characterIndexMap - Character to integer index map

exceptionOnUnknown - If true: throw an exception on unknown characters. False: skip unknown characters.

Method Detail

expandedColumnMetaDatas

protected List<ColumnMetaData> expandedColumnMetaDatas(List<ColumnMetaData> origColumnMeta,
                                                       List<String> expandedColumnNames)

Specified by:

expandedColumnMetaDatas in class BaseSequenceExpansionTransform

expandTimeStep

protected List<List<Writable>> expandTimeStep(List<Writable> currentStepValues)

Specified by:

expandTimeStep in class BaseSequenceExpansionTransform