ai.djl.modality.nlp

Class SimpleVocabulary.VocabularyBuilder

java.lang.Object

ai.djl.modality.nlp.SimpleVocabulary.VocabularyBuilder

Enclosing class:

SimpleVocabulary

public static class SimpleVocabulary.VocabularyBuilder
extends java.lang.Object

Builder class that is used to build the SimpleVocabulary.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	minFrequency
protected java.util.Set<java.lang.String>	reservedTokens
protected java.util.List<java.util.List<java.lang.String>>	sentences
protected java.lang.String	unknownToken

Constructor Summary

Constructors

Constructor and Description
VocabularyBuilder()

Method Summary

Modifier and Type	Method and Description
SimpleVocabulary.VocabularyBuilder	add(java.util.List<java.lang.String> sentence) Adds the given sentence to the SimpleVocabulary.
SimpleVocabulary.VocabularyBuilder	addAll(java.util.List<java.util.List<java.lang.String>> sentences) Adds the given list of sentences to the SimpleVocabulary.
SimpleVocabulary	build() Builds the SimpleVocabulary object with the set arguments.
SimpleVocabulary.VocabularyBuilder	optMinFrequency(int minFrequency) Sets the optional parameter that specifies the minimum frequency to consider a token to be part of the SimpleVocabulary.
SimpleVocabulary.VocabularyBuilder	optReservedTokens(java.util.Collection<java.lang.String> reservedTokens) Sets the optional parameter that sets the list of reserved tokens.
SimpleVocabulary.VocabularyBuilder	optUnknownToken(java.lang.String unknownToken) Sets the optional parameter that specifies the unknown token's string value.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sentences

protected java.util.List<java.util.List<java.lang.String>> sentences

reservedTokens

protected java.util.Set<java.lang.String> reservedTokens

minFrequency

protected int minFrequency

unknownToken

protected java.lang.String unknownToken

Constructor Detail

VocabularyBuilder

public VocabularyBuilder()

Method Detail

optMinFrequency

public SimpleVocabulary.VocabularyBuilder optMinFrequency(int minFrequency)

Sets the optional parameter that specifies the minimum frequency to consider a token to be part of the SimpleVocabulary. Defaults to 10.

Parameters:

minFrequency - the minimum frequency to consider a token to be part of the SimpleVocabulary

Returns:

this VocabularyBuilder

optUnknownToken

public SimpleVocabulary.VocabularyBuilder optUnknownToken(java.lang.String unknownToken)

Sets the optional parameter that specifies the unknown token's string value.

Parameters:

unknownToken - the string value of the unknown token

Returns:

this VocabularyBuilder

optReservedTokens

public SimpleVocabulary.VocabularyBuilder optReservedTokens(java.util.Collection<java.lang.String> reservedTokens)

Sets the optional parameter that sets the list of reserved tokens.

Parameters:

reservedTokens - the list of reserved tokens

Returns:

this VocabularyBuilder

add

public SimpleVocabulary.VocabularyBuilder add(java.util.List<java.lang.String> sentence)

Adds the given sentence to the SimpleVocabulary.

Parameters:

sentence - the sentence to be added

Returns:

this VocabularyBuilder

addAll

public SimpleVocabulary.VocabularyBuilder addAll(java.util.List<java.util.List<java.lang.String>> sentences)

Adds the given list of sentences to the SimpleVocabulary.

Parameters:

sentences - the list of sentences to be added

Returns:

this VocabularyBuilder

build

public SimpleVocabulary build()

Builds the SimpleVocabulary object with the set arguments.

Returns:

the SimpleVocabulary object built