|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
A Token
is the compile-time abstraction between
different syntax analysis stages in a front end of a compiler,
typically the Lexer
Parser
interface. As
a lexical analyzer runs, it generates Symbol
objects.
These are run-time tokens, as they are actual sequence instances
matched against some pattern. The Token
interface is
the abstraction over the pattern (thus the compile-time
abstraction). Thus, if a specific token instance belongs to some
language implied by a regular expression, the Token
holds that expression.
On a more pragmatic level, the Token
interface
coordinates that lexers and parsers use the same token type numbers
(id's).
Field Summary | |
static int |
EPSILON
This is the universal token type id that denotes the empty string. |
static int |
ERROR
This is the token type used for the ERROR symbol. |
static int |
STOP
This is the universal token type for the STOP
symbol. |
static int |
UNDEF
This is the token type used to indicate that the value of a token is unknown, that it has not been set. |
Method Summary | |
Object |
clone()
Clones the Token . |
int |
getID()
Returns the id given to this Token . |
String |
getName()
Returns the name given to this Token . |
Field Detail |
public static final int STOP
STOP
symbol. The STOP
token type is emitted by the
Lexer
when the end of input has been reached.public static final int UNDEF
DFA.output()
method.public static final int ERROR
ERROR
symbol.
This constant is only used internally by the
Lexer
.public static final int EPSILON
Method Detail |
public int getID()
Token
.public String getName()
Token
.public Object clone() throws CloneNotSupportedException
Token
.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |