|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
The Lexer
is responsible for reading an
Input
stream and breaking it up into a stream of
tokens. The standard interface for a Lexer
would have
a single method like "Token lex()
" that the parser
would call to fetch the next token in the input. This interface
does not have any such method though it does have methods
start()
and stop()
. Why?
The reason is that this interface assumes the Lexer
is
responsible only for discovering token sequences in the input, NOT
discovering tokens AND wrapping those tokens in Symbol
objects. The difference is subtle, but it allows a separate object
to handle these "lexical events". This is good because it gives
the user complete control of Symbol
instantiation.
Thus, token sequences are communicated to the
LexerInterpreter
via the match(int token_type,
int offset, int length)
method. The start()
method begins (or restarts) the lexing process and can be stopped
or stopped via the stop()
. The Lexer
will check to see if it is stopped before each token search. Note
this does NOT imply that the Lexer
represents a
Thread
.
Another way to describe this setup would be "lexer-directed-translation" versus "parser-driven-translation".
Said again, the Lexer
component is responsible ONLY
for identification of the locations of tokens in the
Input
, not the actual instantiation of
Symbol
(token) objects. Rather, this functionality is
delegated to the LexerInterpreter
.
Method Summary | |
LexerInterpreter |
getLexerInterpreter()
Gets the Listener to be notified of token
Symbol events. |
void |
resume()
Continues the lexing process from the current Input position. |
void |
setLexerInterpreter(LexerInterpreter interpreter)
Sets the Listener to be notified of token
Symbol events. |
void |
start()
Resets the internal state of the Lexer and
triggers the search for the next lexeme from the input. |
void |
stop()
Pauses the lexing process. |
Methods inherited from interface org.inxar.syntacs.translator.lr.LRTranslationComponent |
getAuditor, getInput, getLRTranslatorGrammar, getProperties, initialize, reset, setAuditor, setInput, setLRTranslatorGrammar, setProperties |
Method Detail |
public void start() throws TranslationException
Lexer
and
triggers the search for the next lexeme from the input.
Notification will take place through the
LexerInterpreter
. This method will run until the
input buffer has been exhausted or stop()
is
called.public void stop()
resume()
. The lexer should check before the start
of each token search to see whether stop has been called.
Therefore, the granularity of stop is limited to the moment
between token searches.public void resume() throws TranslationException
Input
position.public void setLexerInterpreter(LexerInterpreter interpreter)
Listener
to be notified of token
Symbol
events.public LexerInterpreter getLexerInterpreter()
Listener
to be notified of token
Symbol
events.
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |