org.inxar.syntacs.analyzer.lexical
Interface Lexer

All Superinterfaces:
LRTranslationComponent
All Known Implementing Classes:
StandardLexer

public interface Lexer
extends LRTranslationComponent

The Lexer is responsible for reading an Input stream and breaking it up into a stream of tokens. The standard interface for a Lexer would have a single method like "Token lex()" that the parser would call to fetch the next token in the input. This interface does not have any such method though it does have methods start() and stop(). Why?

The reason is that this interface assumes the Lexer is responsible only for discovering token sequences in the input, NOT discovering tokens AND wrapping those tokens in Symbol objects. The difference is subtle, but it allows a separate object to handle these "lexical events". This is good because it gives the user complete control of Symbol instantiation.

Thus, token sequences are communicated to the LexerInterpreter via the match(int token_type, int offset, int length) method. The start() method begins (or restarts) the lexing process and can be stopped or stopped via the stop(). The Lexer will check to see if it is stopped before each token search. Note this does NOT imply that the Lexer represents a Thread.

Another way to describe this setup would be "lexer-directed-translation" versus "parser-driven-translation".

Said again, the Lexer component is responsible ONLY for identification of the locations of tokens in the Input, not the actual instantiation of Symbol (token) objects. Rather, this functionality is delegated to the LexerInterpreter.


Method Summary
 LexerInterpreter getLexerInterpreter()
          Gets the Listener to be notified of token Symbol events.
 void resume()
          Continues the lexing process from the current Input position.
 void setLexerInterpreter(LexerInterpreter interpreter)
          Sets the Listener to be notified of token Symbol events.
 void start()
          Resets the internal state of the Lexer and triggers the search for the next lexeme from the input.
 void stop()
          Pauses the lexing process.
 
Methods inherited from interface org.inxar.syntacs.translator.lr.LRTranslationComponent
getAuditor, getInput, getLRTranslatorGrammar, getProperties, initialize, reset, setAuditor, setInput, setLRTranslatorGrammar, setProperties
 

Method Detail

start

public void start()
           throws TranslationException
Resets the internal state of the Lexer and triggers the search for the next lexeme from the input. Notification will take place through the LexerInterpreter. This method will run until the input buffer has been exhausted or stop() is called.

stop

public void stop()
Pauses the lexing process. Lexing can be restarted via calling resume(). The lexer should check before the start of each token search to see whether stop has been called. Therefore, the granularity of stop is limited to the moment between token searches.

resume

public void resume()
            throws TranslationException
Continues the lexing process from the current Input position.

setLexerInterpreter

public void setLexerInterpreter(LexerInterpreter interpreter)
Sets the Listener to be notified of token Symbol events.

getLexerInterpreter

public LexerInterpreter getLexerInterpreter()
Gets the Listener to be notified of token Symbol events.