Enum EarleyLexerSource
pub enum EarleyLexerSource<'a> {
Basic(&'a BasicLexer),
Dynamic {
matchers: &'a DynamicMatchers,
ignore: &'a [String],
},
DynamicComplete {
matchers: &'a DynamicMatchers,
ignore: &'a [String],
},
Custom(&'a Mutex<dyn TokenSource + Send + 'a>),
}Expand description
Which scan strategy drives an Earley parse (Lark’s resolved lexer; the
frontends facade’s ResolvedLexer). Earley defines its own source enum rather
than importing LALR’s LexerSource (engine boundary): the basic path takes a
pre-lexing BasicLexer token stream; the dynamic paths take the
construction-built matchers.
Variants§
Basic(&'a BasicLexer)
Basic-lexer Earley. Under a postlex the token stream is wrapped by the engine-agnostic connector.
Dynamic
Dynamic-lexer Earley (xearley): per-terminal forward regex; forbids
postlex + lexer_callbacks (enforced at the frontend). The per-terminal
DynamicMatchers are built once at construction (build_dynamic_matchers,
where a zero-width/bad regexp already raised the GrammarError) and borrowed
here — the parse never compiles a matcher.
DynamicComplete
dynamic_complete: dynamic + shorter-prefix enumeration.
Custom(&'a Mutex<dyn TokenSource + Send + 'a>)
A caller-implemented crate::custom::TokenSource feeding the basic
scan (Lark: a lexer class bypasses the parser×lexer matrix and
Earley drives it through _match_earley_basic; the
[custom_old0,earley]/[custom_old1,earley] Lark
cells). Pulled lazily, one token per recognizer step, with the live
pre-closure expects snapshot handed to each pull ([CustomScan]).
Postlex composes on top, as for Basic.
An unknown type name stalls the recognizer at that token — Lark’s
name-mismatch — and the stall reports the raw name
(UnexpectedToken('WAT')).