Transformers

A parse tree is a means, not an end — you almost always want a value: a number, an AST node, a config object. Every tree-to-value job answers two questions, and hyperlark gives the same answers in every language:

1. When? Walk the tree after the parse, or fold during the parse — no tree is ever built (the fast path).
2. How? Bottom-up (a value per node), top-down (you drive the descent), or just for side effects.

Pattern	Python / TypeScript	Rust	C
Bottom-up value per node	Transformer	fold / fold_fn	lark_fold_*
Top-down, evaluate on demand	Interpreter	eval	—
Side effects only	Visitor	Cursor walk	cursor walk
Fold during the parse	transformer=	parse_fold_fn	lark_parse_fold

The class-based toolkit (Python, TypeScript) and the closure/primitive forms (Rust, C) express the same patterns: a Rust closure keyed on the node name is the idiomatic spelling of a Transformer method, and Rust’s lazy eval is the idiomatic spelling of an Interpreter. Pick your language; the shapes below line up row for row.

A running grammar

A four-function calculator whose tree evaluates to a number, self-contained (no imports) so it parses identically on every target — call it GRAMMAR below:

?start: sum
?sum: product
    | sum "+" product   -> add
    | sum "-" product   -> sub
?product: atom
    | product "*" atom  -> mul
    | product "/" atom  -> div
?atom: NUMBER           -> number
     | "-" atom         -> neg
     | "(" sum ")"
NUMBER: /[0-9]+/
%ignore " "

The -> aliases name each node (add, mul, number, …); the ? prefix inlines a rule with a single child, so operands flow straight into the operator nodes. You supply one function per node name.

Bottom-up: a value per node

Leaves first, then each rule after its children — so a rule’s function receives its children already turned into values. Return what the node is worth; the root’s value is the result.

Python

from hyperlark import Lark, Transformer, v_args

@v_args(inline=True)                 # children arrive as positional args
class Calc(Transformer):
    def number(self, tok): return float(tok)
    def add(self, a, b):   return a + b
    def sub(self, a, b):   return a - b
    def mul(self, a, b):   return a * b
    def div(self, a, b):   return a / b
    def neg(self, a):      return -a

parser = Lark(GRAMMAR, parser="lalr")
tree = parser.parse("2 + 3 * 4")
print(Calc().transform(tree))        # 14.0

TypeScript

import { Lark, Transformer, v_args } from "hyperlark";

class Calc extends v_args({ inline: true })(Transformer) {
  number(t) { return Number(t.value); }   // the child is the NUMBER token
  add(a, b) { return a + b; }
  sub(a, b) { return a - b; }
  mul(a, b) { return a * b; }
  div(a, b) { return a / b; }
  neg(a)    { return -a; }
}

const parser = new Lark(GRAMMAR, { parser: "lalr" });
const tree = parser.parse("2 + 3 * 4");
console.log(new Calc().transform(tree));   // 14

Rust

The idiomatic spelling is a closure keyed on the node name: leaf values each token, rule combines a node’s already-folded children. ParseResult::fold runs it iteratively — safe on trees of any depth.

use hyperlark::{fold_fn, Lark};

let parser = Lark::lalr(GRAMMAR)?;
let mut calc = fold_fn(
    |name, leaf| match name {
        "NUMBER" => leaf.text().parse().unwrap(),
        _ => 0.0,
    },
    |name, kids: Vec<f64>| match name {
        "add" => kids[0] + kids[1],
        "sub" => kids[0] - kids[1],
        "mul" => kids[0] * kids[1],
        "div" => kids[0] / kids[1],
        "neg" => -kids[0],
        _ => kids[0],                   // number: single child, passed through
    },
);
let value: f64 = parser.parse("2 + 3 * 4")?.fold(&mut calc);   // 14.0

C

Register a callback per rule name and per token type, then fold bottom-up. Values travel in an opaque 8-byte LarkValue (here .d, a double); unregistered single-child rules use the built-in first-child passthrough, so number and the inlined wrappers need no callback.

/* needs <stdio.h> (printf), <stdlib.h> (strtod), <stdbool.h> via hyperlark.h */
static LarkValue num(void *ud, LarkNode leaf) {
    /* Token values are not NUL-terminated, so copy once. A value too long for
       the buffer is REJECTED, not truncated, and a fold callback has no error
       channel — so record the failure through `ud` and check it after the
       fold. (`docs/examples/c/calc.c` is the complete, runnable version.) */
    char buf[64];
    if (lark_node_token_value_copy(leaf, buf, sizeof buf, NULL) != LARK_OK) {
        if (ud) *(bool *)ud = false;   /* NULL-safe if you lift this callback */
        return (LarkValue){ .d = 0 };
    }
    return (LarkValue){ .d = strtod(buf, NULL) };
}
static LarkValue add(void *u, LarkNode n, const LarkValue *c, size_t k)
    { (void)u; (void)n; (void)k; return (LarkValue){ .d = c[0].d + c[1].d }; }
static LarkValue mul(void *u, LarkNode n, const LarkValue *c, size_t k)
    { (void)u; (void)n; (void)k; return (LarkValue){ .d = c[0].d * c[1].d }; }
/* sub, div, neg follow the same shape */

bool ok = true;                            /* `num` clears this on failure */
LarkFold *fold = NULL;
/* REGISTRATION is a channel too, and the quietest one: a name the grammar does
   not have — a typo, a renamed terminal — returns LARK_INVALID_ARGUMENT right
   here. The leaf then folds through the built-in default, `lark_fold_apply`
   returns LARK_OK, `ok` stays true, and you print a number nobody computed. */
bool reg = lark_fold_new(lark, &fold) == LARK_OK;
reg &= lark_fold_token(fold, "NUMBER", num, &ok) == LARK_OK;  /* `num`'s userdata */
reg &= lark_fold_rule(fold, "add", add, NULL) == LARK_OK;
reg &= lark_fold_rule(fold, "mul", mul, NULL) == LARK_OK;
/* ... register sub / div / neg ... */
if (!reg) { fprintf(stderr, "fold setup: %s\n", lark_last_error()); return 1; }

LarkValue out = { .d = 0 };
/* Two channels remain: the status covers the fold itself (a result from a
   different Lark), `ok` covers a callback that could not convert. `out` is
   pre-zeroed and a failed leaf folds to 0, so ignoring either one reports a
   value that was never computed. */
if (lark_fold_apply(fold, res, &out) != LARK_OK)     /* res from lark_parse */
    fprintf(stderr, "fold failed: %s\n", lark_last_error());
else if (!ok) fprintf(stderr, "a literal did not fit\n");  /* don't exit 0 quietly */
else printf("%g\n", out.d);                /* 14 */
lark_fold_free(fold);

v_args — adapting the call shape (Python & TypeScript)

By default a rule method receives its children as one list. v_args changes that, per method or per class: inline spreads the children as positional args (used above), meta prepends the node’s Meta (positions), tree hands over the whole Tree. Discard (return it to drop a node from its parent) and merge_transformers (Python) round out the toolkit. Rust and C key on the node name directly, so they need no call adapter.

Top-down: evaluate on demand

Sometimes you want to visit the root first and decide what to descend into — to skip a dead branch or an unevaluated operand instead of folding the whole tree.

Python

An Interpreter visits top-down; call visit_children to descend.

from hyperlark.visitors import Interpreter

class Eval(Interpreter):
    def number(self, t): return float(t.children[0])
    def add(self, t): a, b = self.visit_children(t); return a + b
    def mul(self, t): a, b = self.visit_children(t); return a * b
    # sub, div, neg likewise

print(Eval().visit(tree))            # 14.0

TypeScript

import { Interpreter } from "hyperlark";

class Eval extends Interpreter {
  number(t) { return Number(t.children[0].value); }
  add(t) { const [a, b] = this.visit_children(t); return a + b; }
  mul(t) { const [a, b] = this.visit_children(t); return a * b; }
  // sub, div, neg likewise
}
console.log(new Eval().visit(tree));   // 14

Rust

ParseResult::eval is the same pattern, spelled lazily: each child is evaluated by n.eval(i), so a branch you never eval is never walked (more efficient than materializing every child first).

use hyperlark::EvalNode;

let value: f64 = parser.parse("2 + 3 * 4")?.eval(|n: &EvalNode<f64>| match n.name() {
    "add" => n.eval(0) + n.eval(1),
    "sub" => n.eval(0) - n.eval(1),
    "mul" => n.eval(0) * n.eval(1),
    "div" => n.eval(0) / n.eval(1),
    "neg" => -n.eval(0),
    "number" => n.eval(0),
    "NUMBER" => n.text().parse().unwrap(),
    // Every two-child rule needs its own arm: a catch-all `n.eval(0)` would
    // silently return the LEFT operand. Panic rather than guess.
    other => unreachable!("unhandled node: {other}"),
});

Side effects only

A Visitor walks the tree and returns it unchanged — reach for it to collect or check, not to rewrite. In Python and TypeScript it is a Visitor class; in Rust and C, walk a Cursor (see working with trees).

Python

from hyperlark import Visitor

class CountOps(Visitor):
    def __init__(self): self.ops = 0
    def add(self, tree): self.ops += 1
    def mul(self, tree): self.ops += 1

v = CountOps(); v.visit(tree); print(v.ops)   # 2

TypeScript

import { Visitor } from "hyperlark";

class CountOps extends Visitor {
  ops = 0;
  add() { this.ops++; }
  mul() { this.ops++; }
}
const v = new CountOps(); v.visit(tree); console.log(v.ops);   // 2

Fold during the parse (the fast path)

Hand the transform to the parser and it folds as the parser reduces — no parse tree is materialized. This is the fast path for straight tree-to-value work, and it is LALR-only in every target.

Python

Pass the Calc transformer from the top of this page as transformer= — one definition, two ways to run it. parse then returns the value directly:

parser = Lark(GRAMMAR, parser="lalr", transformer=Calc())
print(parser.parse("2 + 3 * 4"))     # 14.0 — no Tree built

Rust

parse_fold_fn folds at the reduce seam with the same two closures — no ParsedTree is allocated:

let value: f64 = parser.parse_fold_fn(
    "2 + 3 * 4", None,                 // default start symbol
    |name, leaf| match name { "NUMBER" => leaf.text().parse().unwrap(), _ => 0.0 },
    |name, kids: Vec<f64>| match name {
        "add" => kids[0] + kids[1], "sub" => kids[0] - kids[1],
        "mul" => kids[0] * kids[1], "div" => kids[0] / kids[1],
        "neg" => -kids[0], _ => kids[0],
    },
)?;                                    // 14.0

TypeScript

The reduce-time form is a plain object mapping each rule name to a function of its children array (v_args does not apply here):

const calc = {
  number: (c) => Number(c[0].value),
  add: (c) => c[0] + c[1],  sub: (c) => c[0] - c[1],
  mul: (c) => c[0] * c[1],  div: (c) => c[0] / c[1],
  neg: (c) => -c[0],
};
const parser = new Lark(GRAMMAR, { parser: "lalr" });
console.log(parser.parse("2 + 3 * 4", { transformer: calc }));   // 14

C

lark_parse_fold is the embedded form — a separate entry point from the post-parse lark_fold_* shown earlier, and the one to reach for here: it folds straight to a LarkValue during the parse, so no tree and no arena are ever built. Registration is by name, just as above, but the callback shapes differ — there is no tree, so nothing takes a LarkNode. A rule gets the reduced rule’s numeric display id; a leaf gets the token’s type id plus its (text, len) bytes:

/* also needs <string.h> for memcpy */
static LarkValue r_num(void *ud, uint32_t id, const char *text, size_t len) {
    (void)id;
    char b[64];
    if (len >= sizeof b) {          /* same hazard as the bottom-up tab: a value
                                       too long must NOT quietly become 0 */
        if (ud) *(bool *)ud = false;
        return (LarkValue){ .d = 0 };
    }
    memcpy(b, text, len); b[len] = '\0';                 /* not NUL-terminated */
    return (LarkValue){ .d = strtod(b, NULL) };
}
static LarkValue r_add(void *ud, uint32_t id, const LarkValue *c, size_t k)
    { (void)ud; (void)id; (void)k; return (LarkValue){ .d = c[0].d + c[1].d }; }

bool ok = true;
LarkReduceFold *fold = NULL;
/* Registration is the quiet channel here too — an unknown name is rejected
   here, not at fold time (see the bottom-up tab). */
bool reg = lark_reduce_fold_new(lark, &fold) == LARK_OK;
reg &= lark_reduce_fold_token(fold, "NUMBER", r_num, &ok) == LARK_OK;
reg &= lark_reduce_fold_rule(fold, "add", r_add, NULL) == LARK_OK;
/* ... */
if (!reg) { fprintf(stderr, "fold setup: %s\n", lark_last_error()); return 1; }

LarkValue out = { .d = 0 };
/* Two channels remain: the status covers a parse failure (*out is zeroed, so
   an ignored status evaluates to 0.0), `ok` covers a callback that could not
   convert its token. */
if (lark_parse_fold(lark, text, len, "start", fold, &out) != LARK_OK)
    fprintf(stderr, "parse failed: %s\n", lark_last_error());
else if (!ok) fprintf(stderr, "a literal did not fit\n");
else printf("%g\n", out.d);
lark_reduce_fold_free(fold);

See the feature matrix for the full per-target picture.