org.inxar.syntacs.grammar
Interface Token

All Superinterfaces:
Cloneable
All Known Subinterfaces:
RegularToken

public interface Token
extends Cloneable

A Token is the compile-time abstraction between different syntax analysis stages in a front end of a compiler, typically the Lexer Parser interface. As a lexical analyzer runs, it generates Symbol objects. These are run-time tokens, as they are actual sequence instances matched against some pattern. The Token interface is the abstraction over the pattern (thus the compile-time abstraction). Thus, if a specific token instance belongs to some language implied by a regular expression, the Token holds that expression.

On a more pragmatic level, the Token interface coordinates that lexers and parsers use the same token type numbers (id's).


Field Summary
static int EPSILON
          This is the universal token type id that denotes the empty string.
static int ERROR
          This is the token type used for the ERROR symbol.
static int STOP
          This is the universal token type for the STOP symbol.
static int UNDEF
          This is the token type used to indicate that the value of a token is unknown, that it has not been set.
 
Method Summary
 Object clone()
          Clones the Token.
 int getID()
          Returns the id given to this Token.
 String getName()
          Returns the name given to this Token.
 

Field Detail

STOP

public static final int STOP
This is the universal token type for the STOP symbol. The STOP token type is emitted by the Lexer when the end of input has been reached.

UNDEF

public static final int UNDEF
This is the token type used to indicate that the value of a token is unknown, that it has not been set. This is used as a return value from the DFA.output() method.

ERROR

public static final int ERROR
This is the token type used for the ERROR symbol. This constant is only used internally by the Lexer.

EPSILON

public static final int EPSILON
This is the universal token type id that denotes the empty string.
Method Detail

getID

public int getID()
Returns the id given to this Token.

getName

public String getName()
Returns the name given to this Token.

clone

public Object clone()
             throws CloneNotSupportedException
Clones the Token.