Feature matrix
What each target supports, tracked against the main branch (this page follows
ongoing development, not a pinned release). Hyperlark is one Rust engine behind
four surfaces; the engine is feature-complete for LALR + Earley, and the
bindings expose progressively more of it. This page is the honest map of the
gaps — see each binding’s own README, and the rest of this docs site, for its
exact API.
Legend: ✓ supported · ◐ supported with caveats (see notes) · ✗ not yet · — not applicable to this target.
| Feature | Rust | Python | TS / WASM | C |
|---|---|---|---|---|
| LALR(1) parser | ✓ | ✓ | ✓ | ✓ |
Earley parser (SPPF, resolve / explicit) | ✓ | ✓ | ✓ | ✗ |
Lexers: basic / contextual | ✓ | ✓ | ✓ | ✓ |
Lexers: dynamic / dynamic_complete | ✓ | ✓ | ✓ | ✗ |
postlex / Indenter | ✓ | ◐ ⁷ ⁸ | ✓ | ✗ |
| Interactive / incremental parser | ✓ ¹ | ✓ ¹ ⁹ | ✓ ¹ | ✓ ¹ |
| Reduce-time (embedded) transformer | ✓ | ✓ | ✓ | ◐ ² |
Post-parse Transformer/Visitor/Interpreter | ✓ ³ | ✓ | ✓ | ◐ ³ |
v_args, merge_transformers, Discard | — | ✓ | ✓ | — |
| Custom lexer | ✓ | ✓ | ✓ | ✗ |
Tree cursors / find_data / find_token / pretty | ✓ | ✓ | ✓ | ✓ |
propagate_positions | ✓ | ✓ | ✓ | ✓ |
Unicode property escapes (\p{...}) in terminals | ✓ | ✓ | ✓ | ✓ |
| Save (serialize grammar) | ✓ | ✓ | ✗ | ✗ |
| Load (deserialize grammar) | ✓ | ✓ | ✓ ⁴ | ✓ ⁴ |
Constructor cache (cache=) | ✗ | ✓ | ✗ | ✗ |
%import (embedded stdlib · filesystem) | ✓ | ✓ | ✓ ⁵ | ◐ ⁵ |
%import from an installed package (open_from_package / FromPackageLoader) | — | ✓ | — | — |
Grammar tooling (find_grammar_errors) | ✓ | ✓ | ✗ | ✗ |
| Browser / bundler / Deno build | — | — | ✓ ⁶ | — |
Notes
-
Interactive parsing is LALR + built-in lexer only (Earley or a custom lexer raise). The LALR-only part mirrors Lark’s own behavior; the built-in-lexer restriction is hyperlark’s own, not inherited from Lark.
-
C’s analogue is a fold-callback API keyed by rule and token rather than a
Transformerclass:lark_parse_foldfolds a value during the parse, with no tree built. -
Rust and C ship primitives rather than the class toolkit. Rust has
Cursor/Fold/EvalNode, covering all three patterns. C has the post-parse bottom-up fold (lark_fold_*+lark_fold_apply, theTransformeranalogue) and a cursor walk for side-effect visiting, but no top-downInterpreterequivalent — hence ◐ rather than ✓. -
WASM/C can load a serialized grammar (
Lark.fromJSON/lark_from_json); emitting the serialized form is a Rust/Python capability. -
WASM and C both resolve
%importagainst the embedded stdlib; WASM also resolves filesystem imports through a loader you supply, while C is stdlib-only. -
Via the
hyperlark/webentry (asyncinit()), in addition to the Node build. -
A Python
postlexcomposes with the contextual lexer: itsprocessis resumed between parser steps, so each raw token is lexed against the live parser state andlexer="contextual"is honoured. A postlexer that reads AHEAD (pulling the next token before yielding the current one) outruns the parser, so terminals that only disambiguate contextually can fail for it — which is true of lark too, and is a property of lookahead rather than of the binding. -
A postlex session is re-driven, not rebuilt. Lark starts a fresh
process()on every driving call —parse,resume_parse,exhaust_lexer,iter_parse— which for anIndenteralso resets its paren and indent state. hyperlark rebuilds only after an error was delivered intoprocess(), so a postlexer that carries state across tokens (a held-token lookahead, an indent stack) sees that state survive where lark’s is reset. Observable when a parse is driven more than once: underon_error, or by callingexhaust_lexeragain after a rejection.Relatedly, an exception raised INSIDE
process()is permanent here: every later driving call re-raises it, where lark starts a freshprocess()and carries on. Deliberate — a retry that silently resumed a stream which had already failed would turn a failed parse into a truncated successful one. -
With an embedded (reduce-time) transformer, the interactive parser runs its callbacks at the FINISHING feed rather than per reduction. Lark transforms inline, so a side effect — or an exception — from an early rule appears while you are still feeding; here it appears when
feed_eofcompletes the parse.parse()is unaffected: its flushes land before it returns, and a failing parse still runs the pre-error prefix.A REJECTED finishing feed shows none of them, because the pending batch is deliberately left intact — a rejected
feed_eofis how a caller probes for completeness, so the session may continue. Nothing is lost if it does: the next successful finish delivers the same exception and the same side effects, in order. Only a caller that abandons the parse there sees the transformer’s exception replaced by the parse error.batch_size=1gives lark’s exact timing. May be revisited.
Deferred features (all targets)
Section titled “Deferred features (all targets)”Not yet implemented in any binding. These are peripheral to core parsing; the engine and the everyday surface do not depend on them.
| Feature | Status |
|---|---|
Reconstructor (Tree → source text) | Deferred — planned for a later release. |
Tree templates (lark.tree_templates) | Deferred. |
ambiguity='forest' / forest transformer | Deferred (Earley resolve/explicit are supported). |
cache_grammar= (unanalyzed grammar in the cache blob) | Deferred — its consumer, the reconstructor, is itself deferred. cache= is implemented in the Python binding. |
strict= mode | Deferred — the truthy form raises ConfigurationError. |
| Standalone tool (self-contained parser generation) | Deferred — the compiled bindings largely serve the same zero-dependency deployment need. |
use_bytes=True | Rejected by design — a core token’s TEXT is UTF-8 (a written .value may hold any object, but the lexed stream cannot be bytes); the truthy form raises. |
regex module (regex=True) | Deferred — the truthy form raises. The engine is not confined to re’s feature set, though: \p{...} property escapes compile on every target with no flag, where stock re cannot compile them at all. Constructs unique to the regex module — recursive subpatterns ((?&NAME)) above all — stay errors. |
edit_terminals= (mutate terminals before compiling) | Deferred — the truthy form raises. |
TextSlice inputs (parse a window of a larger string) | Deferred — pass the slice itself. |
| CYK parser | Not planned. |
How rejection works: options that hyperlark accepts as known Lark kwargs but
does not implement (strict, cache_grammar, regex, use_bytes,
edit_terminals) raise a clear ConfigurationError on their active (truthy)
value rather than silently doing nothing; their falsy/default value is an
accepted no-op. ordered_sets stays accepted as a no-op so drop-in code keeps
working. debug=True is not a no-op: like Lark’s, it raises the package
logger (hyperlark.logger) to DEBUG.
Correctness
Section titled “Correctness”Every supported feature is validated differentially against a pinned version of Lark: Lark is the answer key, never a runtime dependency. Lark’s own test suite runs against hyperlark unmodified — see Coming from Lark.