Struct BasicTokenSource 

pub struct BasicTokenSource<'a> { /* private fields */ }

A TokenSource over the shared BasicLexer — the analogue of the Lark test classes’ BasicLexer(copy(lexer_conf)) wrapper (CustomLexerNew), and the building block the conformance custom-lexer port drives. Owns its cursor (LexerState); ignores the parser-state key like Lark’s BasicLexer.next_token.

Implementations

impl<'a> BasicTokenSource<'a>

pub fn new(lexer: &'a BasicLexer, text: &'a str) -> Self

Trait Implementations

impl TokenSource for BasicTokenSource<'_>

fn next_token( &mut self, _parser_state: Option<usize>, _expects: Option<&[u32]>, ) -> Result<Option<CustomToken>, CustomLexError>