With the default configuration:
For example:
| word | orthography | |
|---|---|---|
| “Why | → | why |
| hasn’t | → | hasn't |
| Inés | → | inés |
| d~ | → | d~ |
| got — | → | got |
| her | → | her |
| X-ray | → | x-ray |
| yet?” | → | yet |
| 😉 | → | 😉 |
The Token Layer setting determines which layer raw word tokens come from.
Whether to convert token labels to lowercase before processing replacements.
This is a list of replacements to be made. The left hand column contains regular expressions to be replaced, and the right hand column contains what matches should be replaced with; if the right-hand box is blank, then matches are deleted.
A new replacement can be added to the list by using the ➕ button.
An existing replacement can be removed from the list by clicking the regular expression text box or the replacement text box and then using the ➖ button.
The replacements are processed in the order they are listed. A replacement can be moved up or down the list by clicking the regular expression text box or the replacement text box and then using the ↑ or ↓ button.
| Construct | Matches |
|---|---|
| Characters | |
| x | The character x |
| \\ | The backslash character |
| \0n | The character with octal value 0n (0 <= n <= 7) |
| \0nn | The character with octal value 0nn (0 <= n <= 7) |
| \0mnn | The character with octal value 0mnn (0 <= m <= 3, 0 <= n <= 7) |
| \xhh | The character with hexadecimal value 0xhh |
| \uhhhh | The character with hexadecimal value 0xhhhh |
| \t | The tab character ('\u0009') |
| Character classes | |
| [abc] | a, b, or c (simple class) |
| [^abc] | Any character except a, b, or c (negation) |
| [a-zA-Z] | a through z or A through Z, inclusive (range) |
| [a-d[m-p]] | a through d, or m through p: [a-dm-p] (union) |
| [a-z&&[def]] | d, e, or f (intersection) |
| [a-z&&[^bc]] | a through z, except for b and c: [ad-z] (subtraction) |
| [a-z&&[^m-p]] | a through z, and not m through p: [a-lq-z](subtraction) |
| Predefined character classes | |
| . | Any character (may or may not match line terminators) |
| \d | A digit: [0-9] |
| \D | A non-digit: [^0-9] |
| \s | A whitespace character: [ \t\n\x0B\f\r] |
| \S | A non-whitespace character: [^\s] |
| \w | A word character: [a-zA-Z_0-9] |
| \W | A non-word character: [^\w] |
| Greedy quantifiers | |
| X? | X, once or not at all |
| X* | X, zero or more times |
| X+ | X, one or more times |
| X{n} | X, exactly n times |
| X{n,} | X, at least n times |
| X{n,m} | X, at least n but not more than m times |
| Logical operators | |
| XY | X followed by Y |
| X|Y | Either X or Y |
| (X) | X |
| Quotation | |
| \ | Nothing, but quotes the following character |
The backslash character ('\') serves to introduce escaped constructs, as defined in the table above, as well as to quote characters that otherwise would be interpreted as unescaped constructs. Thus the expression \\ matches a single backslash and \{ matches a left brace.
It is an error to use a backslash prior to any alphabetic character that does not denote an escaped construct; these are reserved for future extensions to the regular-expression language. A backslash may be used prior to a non-alphabetic character regardless of whether that character is part of an unescaped construct.
Character classes may appear within other character classes, and may be composed by the union operator (implicit) and the intersection operator (&&). The union operator denotes a class that contains every character that is in at least one of its operand classes. The intersection operator denotes a class that contains every character that is in both of its operand classes.
The precedence of character-class operators is as follows, from highest to lowest:
1 Literal escape \x 2 Grouping [...] 3 Range a-z 4 Union [a-e][i-u] 5 Intersection [a-z&&[aeiou]]
Note that a different set of metacharacters are in effect inside a character class than outside a character class. For instance, the regular expression . loses its special meaning inside a character class, while the expression - becomes a range forming metacharacter.
This is the layer that new annotations will be added to.