Interactive parsing

Most of the time you hand hyperlark a whole string and get a tree back. The interactive parser turns that inside out: you drive an LALR parse one token at a time, and between tokens you can ask what the grammar would accept next, copy the parser’s state to explore a branch, and resume when you’re ready. That makes it the right tool for autocomplete, REPLs, editors, and error recovery — anywhere you’re parsing input that isn’t finished yet.

The moving parts:

• Create an interactive parser, optionally seeded with text (omit the text to feed tokens entirely by hand).
• Feed tokens with feed_token, or drive the built-in lexer over the seed text with iter_parse / exhaust_lexer.
• Inspect the current state: accepts() lists the terminals that fit next (including $END), and choices() gives the full shift/reduce action map.
• Fork the whole parser with copy — an independent branch for speculative feeds or backtracking. (One Python refusal: a parser carrying an embedded transformer= cannot fork — copy raises.)
• Finish with feed_eof (append $END and stop) or resume_parse (drive the rest of the seed text plus $END), both returning a parse tree.

(This mirrors Lark’s parse_interactive / InteractiveParser; Python adds peek_next / feed_next / last_token on top.)

LALR + built-in lexer only

Interactive parsing requires the LALR parser and the built-in lexer — an Earley instance or a custom lexer raises. The LALR-only half mirrors Lark; the built-in-lexer half does NOT — Lark drives a custom lexer interactively, and this is a hyperlark limitation. See the feature matrix for the full per-target support map.

The operations, per target

Operation	Python	Rust	TypeScript	C
Create (seed optional)	parse_interactive(text=None)	OwnedInteractiveParser::new	parseInteractive(text?)	lark_parse_interactive
Feed one token	feed_token	feed_token	feedToken	lark_interactive_feed_token
What fits next	accepts / choices	accepts / choices	accepts / choices	lark_interactive_accepts_count / _get
Drive the lexer	iter_parse / exhaust_lexer	feed_next / exhaust_lexer	iterParse / exhaustLexer	lark_interactive_feed_next
Finish	feed_eof / resume_parse	feed_eof / resume_parse	feedEof / resumeParse	lark_interactive_feed_eof / lark_interactive_resume_parse
Fork	copy	copy	copy	lark_interactive_copy
Release	(GC)	(drop)	(GC)	lark_interactive_free

A worked example

A self-contained calculator grammar — call it GRAMMAR below — so the same tokens parse identically on every target:

start: NUMBER (OP NUMBER)*
OP: "+" | "-" | "*" | "/"
NUMBER: /[0-9]+/
%ignore " "

Here we create a parser with no seed text, feed tokens by hand while watching accepts(), fork a branch, and finish. At the start only a NUMBER fits; after one number, an OP or end-of-input do — exactly the set you’d offer as autocomplete suggestions.

Python

import hyperlark as lark
p = lark.Lark(GRAMMAR, parser="lalr")      # interactive is LALR-only

ip = p.parse_interactive()                 # no seed: we feed tokens ourselves
print(ip.accepts())                        # {'NUMBER'} — only a number fits here

toks = list(p.lex("1 + 2"))                # any object with .type/.value feeds in
ip.feed_token(toks[0])                     # feed "1"
print(ip.accepts())                        # {'OP', '$END'} — operator or end

branch = ip.copy()                         # an independent fork to explore
for tok in toks[1:]:
    ip.feed_token(tok)                     # feed "+ 2"
tree = ip.feed_eof()                       # append $END -> a parse tree
print(tree.pretty())

Rust

use std::sync::Arc;
use hyperlark::{Lark, OwnedInteractiveParser, Token};

let p = Arc::new(Lark::lalr(GRAMMAR)?);    // interactive is LALR-only

// No seed text: feed tokens ourselves (pass a String to drive the lexer).
let mut ip = OwnedInteractiveParser::new(Arc::clone(&p), String::new(), None)?;
assert!(ip.accepts().iter().any(|t| t == "NUMBER"));   // what fits first

// feed_token takes a core Token; resolve its terminal id via the parse table.
let table = ip.parse_table();
ip.feed_token(Token::synthetic(table.token_id("NUMBER").unwrap(), "1"))?;
assert!(ip.accepts().iter().any(|t| t == "$END"));     // a lone number is complete

let _branch = ip.copy();                   // an independent fork to explore
let _tree = ip.feed_eof()?;                // append $END -> a ParsedTree (walk its arena)

TypeScript

import { Lark } from "hyperlark";
const p = new Lark(GRAMMAR, { parser: "lalr" });   // interactive is LALR-only

const ip = p.parseInteractive();           // no seed: we feed tokens ourselves
console.log(ip.accepts());                 // ["NUMBER"] — only a number fits here

const toks = p.lex("1 + 2");               // Token objects to feed
ip.feedToken(toks[0]);                      // feed "1"
console.log(ip.accepts());                 // ["OP", "$END"] — operator or end

const branch = ip.copy();                  // an independent fork to explore
for (const t of toks.slice(1)) ip.feedToken(t);
const handle = ip.feedEof();               // append $END -> a ParseHandle
console.log(handle.toJs(handle.rootPos())); // materialize the { data, children } tree

C

Lark *p = NULL;
lark_from_source(GRAMMAR, strlen(GRAMMAR), LARK_OPT_NONE, &p);  // the C ABI is LALR-only

// --- feed tokens by hand; inspect the accepted set at each step ---
LarkInteractive *ip = NULL;
lark_parse_interactive(p, "", 0, NULL, &ip);   // empty text -> feed by hand

// The accepted terminals in this state (sorted; "$END" only when it's a valid stop), by index.
for (size_t i = 0, n = lark_interactive_accepts_count(ip); i < n; i++)
    printf("accepts: %s\n", lark_interactive_accepts_get(ip, i));

lark_interactive_feed_token(ip, "NUMBER", "1", 1);   // feed one token: name + value

LarkInteractive *branch = NULL;
lark_interactive_copy(ip, &branch);            // an independent fork to explore

LarkParseResult *tree = NULL;
lark_interactive_feed_eof(ip, &tree);          // append $END -> a parse result
printf("%s\n", lark_result_pretty(tree));

// --- or STREAM the built-in lexer one token at a time (pull) ---
// Seat a SEPARATE handle over real input; feed_next drives that input's lexer.
// (The parse still builds its tree as it reduces — you'd take it via feed_eof;
//  for a genuinely tree-free stream, use a reduce-time fold instead.)
LarkInteractive *sp = NULL;
lark_parse_interactive(p, "1+2", 3, NULL, &sp);
LarkTokenEvent ev;
bool has_tok;
while (lark_interactive_feed_next(sp, &ev, &has_tok) == LARK_OK && has_tok)
    printf("token id %u = %.*s\n", ev.type_id, (int)ev.value_len, ev.value);

lark_result_free(tree);
lark_interactive_free(sp);
lark_interactive_free(branch);
lark_interactive_free(ip);
lark_free(p);

A few per-target notes:

• accepts() shape differs idiomatically — a Python set, a Rust Vec<String>, a TypeScript string[], and (in C) a count+index list (lark_interactive_accepts_count / _get) — but the members are the same terminal names. choices() returns the richer {name: (kind, id)} action map (Shift / Reduce); it is not exposed on the C surface.
• TypeScript — feedEof() / resumeParse() return a ParseHandle (the WASM retained-tree handle), so call handle.toJs(handle.rootPos()) for the plain { data, children } tree, or walk it with the handle’s cursors.
• Rust — OwnedInteractiveParser is the owning handle the Python, WASM, and C bindings all wrap; feed_eof / resume_parse hand back a ParsedTree you walk through its arena.
• C — every handle you create (including each copy fork) is freed with lark_interactive_free; the finished LarkParseResult with lark_result_free.

Driving the built-in lexer

Seed the parser with text and you don’t have to produce tokens yourself — drive the built-in lexer instead. iter_parse steps token by token so you can react per token — in Python it yields each token before feeding it (as Lark does), so retyping the yielded token steers the parse, while TypeScript’s iterParse feeds first and yields after; exhaust_lexer feeds them all at once; resume_parse runs the remainder plus $END straight to a tree. For example, in Python:

ip = p.parse_interactive("1 + 2")   # seat over text
ip.exhaust_lexer()                  # drive the lexer to the end of the seed
print(ip.accepts())                 # {'OP', '$END'} — a complete expression
tree = ip.feed_eof()

copy() forks at any point, so you can seed with a common prefix and branch to explore alternative continuations independently (in Python, unless the parser carries an embedded transformer= — that one combination refuses to fork).

Error recovery

The interactive parser also underpins error recovery: on the LALR parse path you can pass an on_error callback that receives the failure, inspects its accepts set and position, and decides whether to feed a corrective token and continue or let the error propagate. In Python that’s parser.parse(text, on_error=handler); in TypeScript, parser.parse(text, { on_error }) where on_error is an (err) => boolean (true resumes). The errors guide covers recovery in depth, including the Rust and C spellings.