Doctrine\Common\Lexer
Base class for writing simple lexers, i.e. for creating small DSLs.
- Author: Guilherme Blanco <guilhermeblanco@hotmail.com>
Synopsis
class Lexer
{
- // members
- private array $tokens;
- private integer $position;
- private integer $peek;
- public array $lookahead;
- public array $token;
- // methods
- public void setInput()
- public void reset()
- public void resetPeek()
- public void resetPosition()
- public boolean isNextToken()
- public boolean isNextTokenAny()
- public array|null moveNext()
- public void skipUntil()
- public boolean isA()
- public array peek()
- public array|null glimpse()
- protected void scan()
- public string getLiteral()
- protected abstract array getCatchablePatterns()
- protected abstract array getNonCatchablePatterns()
- protected abstract integer getType()
Hierarchy
Tasks
Line | Task |
---|---|
31+ | Rename: AbstractLexer |
Members
private
- $peek — integer
- $position — integer
- $tokens — array
public
- $lookahead — array
- $token — array
Methods
protected
- getCatchablePatterns() — Lexical catchable patterns.
- getNonCatchablePatterns() — Lexical non-catchable patterns.
- getType() — Retrieve token type. Also processes the token value if necessary.
- scan() — Scans the input string for tokens.
public
- getLiteral() — Gets the literal for a given token.
- glimpse() — Peeks at the next token, returns it and immediately resets the peek.
- isA() — Checks if given value is identical to the given token
- isNextToken() — Checks whether a given token matches the current lookahead.
- isNextTokenAny() — Checks whether any of the given tokens matches the current lookahead
- moveNext() — Moves to the next token in the input string.
- peek() — Moves the lookahead token forward.
- reset() — Resets the lexer.
- resetPeek() — Resets the peek pointer to 0.
- resetPosition() — Resets the lexer position on the input to the given position.
- setInput() — Sets the input data to be tokenized.
- skipUntil() — Tells the lexer to skip input tokens until it sees a token with the given value.