Skip to content

Feature matrix

What each target supports, tracked against the main branch (this page follows ongoing development, not a pinned release). Hyperlark is one Rust engine behind four surfaces; the engine is feature-complete for LALR + Earley, and the bindings expose progressively more of it. This page is the honest map of the gaps — see each binding’s own README, and the rest of this docs site, for its exact API.

Legend: ✓ supported · ◐ supported with caveats (see notes) · ✗ not yet · — not applicable to this target.

FeatureRustPythonTS / WASMC
LALR(1) parser
Earley parser (SPPF, resolve / explicit)
Lexers: basic / contextual
Lexers: dynamic / dynamic_complete
postlex / Indenter◐ ⁷ ⁸
Interactive / incremental parser✓ ¹✓ ¹ ⁹✓ ¹✓ ¹
Reduce-time (embedded) transformer◐ ²
Post-parse Transformer/Visitor/Interpreter✓ ³◐ ³
v_args, merge_transformers, Discard
Custom lexer
Tree cursors / find_data / find_token / pretty
propagate_positions
Unicode property escapes (\p{...}) in terminals
Save (serialize grammar)
Load (deserialize grammar)✓ ⁴✓ ⁴
Constructor cache (cache=)
%import (embedded stdlib · filesystem)✓ ⁵◐ ⁵
%import from an installed package (open_from_package / FromPackageLoader)
Grammar tooling (find_grammar_errors)
Browser / bundler / Deno build✓ ⁶

Notes

  1. Interactive parsing is LALR + built-in lexer only (Earley or a custom lexer raise). The LALR-only part mirrors Lark’s own behavior; the built-in-lexer restriction is hyperlark’s own, not inherited from Lark.

  2. C’s analogue is a fold-callback API keyed by rule and token rather than a Transformer class: lark_parse_fold folds a value during the parse, with no tree built.

  3. Rust and C ship primitives rather than the class toolkit. Rust has Cursor / Fold / EvalNode, covering all three patterns. C has the post-parse bottom-up fold (lark_fold_* + lark_fold_apply, the Transformer analogue) and a cursor walk for side-effect visiting, but no top-down Interpreter equivalent — hence ◐ rather than ✓.

  4. WASM/C can load a serialized grammar (Lark.fromJSON / lark_from_json); emitting the serialized form is a Rust/Python capability.

  5. WASM and C both resolve %import against the embedded stdlib; WASM also resolves filesystem imports through a loader you supply, while C is stdlib-only.

  6. Via the hyperlark/web entry (async init()), in addition to the Node build.

  7. A Python postlex composes with the contextual lexer: its process is resumed between parser steps, so each raw token is lexed against the live parser state and lexer="contextual" is honoured. A postlexer that reads AHEAD (pulling the next token before yielding the current one) outruns the parser, so terminals that only disambiguate contextually can fail for it — which is true of lark too, and is a property of lookahead rather than of the binding.

  8. A postlex session is re-driven, not rebuilt. Lark starts a fresh process() on every driving call — parse, resume_parse, exhaust_lexer, iter_parse — which for an Indenter also resets its paren and indent state. hyperlark rebuilds only after an error was delivered into process(), so a postlexer that carries state across tokens (a held-token lookahead, an indent stack) sees that state survive where lark’s is reset. Observable when a parse is driven more than once: under on_error, or by calling exhaust_lexer again after a rejection.

    Relatedly, an exception raised INSIDE process() is permanent here: every later driving call re-raises it, where lark starts a fresh process() and carries on. Deliberate — a retry that silently resumed a stream which had already failed would turn a failed parse into a truncated successful one.

  9. With an embedded (reduce-time) transformer, the interactive parser runs its callbacks at the FINISHING feed rather than per reduction. Lark transforms inline, so a side effect — or an exception — from an early rule appears while you are still feeding; here it appears when feed_eof completes the parse. parse() is unaffected: its flushes land before it returns, and a failing parse still runs the pre-error prefix.

    A REJECTED finishing feed shows none of them, because the pending batch is deliberately left intact — a rejected feed_eof is how a caller probes for completeness, so the session may continue. Nothing is lost if it does: the next successful finish delivers the same exception and the same side effects, in order. Only a caller that abandons the parse there sees the transformer’s exception replaced by the parse error. batch_size=1 gives lark’s exact timing. May be revisited.

Not yet implemented in any binding. These are peripheral to core parsing; the engine and the everyday surface do not depend on them.

FeatureStatus
Reconstructor (Tree → source text)Deferred — planned for a later release.
Tree templates (lark.tree_templates)Deferred.
ambiguity='forest' / forest transformerDeferred (Earley resolve/explicit are supported).
cache_grammar= (unanalyzed grammar in the cache blob)Deferred — its consumer, the reconstructor, is itself deferred. cache= is implemented in the Python binding.
strict= modeDeferred — the truthy form raises ConfigurationError.
Standalone tool (self-contained parser generation)Deferred — the compiled bindings largely serve the same zero-dependency deployment need.
use_bytes=TrueRejected by design — a core token’s TEXT is UTF-8 (a written .value may hold any object, but the lexed stream cannot be bytes); the truthy form raises.
regex module (regex=True)Deferred — the truthy form raises. The engine is not confined to re’s feature set, though: \p{...} property escapes compile on every target with no flag, where stock re cannot compile them at all. Constructs unique to the regex module — recursive subpatterns ((?&NAME)) above all — stay errors.
edit_terminals= (mutate terminals before compiling)Deferred — the truthy form raises.
TextSlice inputs (parse a window of a larger string)Deferred — pass the slice itself.
CYK parserNot planned.

How rejection works: options that hyperlark accepts as known Lark kwargs but does not implement (strict, cache_grammar, regex, use_bytes, edit_terminals) raise a clear ConfigurationError on their active (truthy) value rather than silently doing nothing; their falsy/default value is an accepted no-op. ordered_sets stays accepted as a no-op so drop-in code keeps working. debug=True is not a no-op: like Lark’s, it raises the package logger (hyperlark.logger) to DEBUG.

Every supported feature is validated differentially against a pinned version of Lark: Lark is the answer key, never a runtime dependency. Lark’s own test suite runs against hyperlark unmodified — see Coming from Lark.