public final class CFStringTokenizerTokenType
extends java.lang.Object
| Modifier and Type | Field and Description |
|---|---|
static long |
HasDerivedSubTokensMask
Compound token which may contain derived subtokens.
|
static long |
HasHasNumbersMask
Compound token which may contain derived subtokens.
|
static long |
HasNonLettersMask
Compound token which may contain derived subtokens.
|
static long |
HasSubTokensMask
Compound token which may contain subtokens but with no derived subtokens.
|
static long |
IsCJWordMask
Compound token which may contain derived subtokens.
|
static long |
None
Have no token.
|
static long |
Normal
Normal token
|
public static final long None
public static final long Normal
public static final long HasSubTokensMask
public static final long HasDerivedSubTokensMask
public static final long HasHasNumbersMask
public static final long HasNonLettersMask
public static final long IsCJWordMask