# EBNF Grammar
EBNF (Extended Backus-Naur Form) is XGrammar's native grammar format. A grammar written in this
format describes exactly which strings the LLM may generate: fixed strings, character sets,
alternatives, repetition, and recursion. Grammars are constructed with
[`xgr.Grammar.from_ebnf`](xgrammar.Grammar.from_ebnf), or by passing the grammar string directly
to [`xgr.GrammarCompiler.compile_grammar`](xgrammar.GrammarCompiler.compile_grammar).
EBNF is also XGrammar's common intermediate representation: every other frontend (JSON Schema,
regular expressions, structural tags, Lark) is converted to it internally, and printing any
[`xgr.Grammar`](xgrammar.Grammar) with `str(grammar)` produces an equivalent grammar in this
format that can be parsed again.
The syntax is compatible with the GBNF (GGML BNF) format used by
[llama.cpp](https://github.com/ggerganov/llama.cpp/blob/master/grammars/README.md), with many
extensions, such as repetition ranges and macros.
```python
import xgrammar as xgr
grammar = xgr.Grammar.from_ebnf(r"""
root ::= expr ("=" expr)*
expr ::= term ([-+*/] term)*
term ::= num | "(" expr ")"
num ::= [0-9]+
""")
```
This grammar accepts strings such as `1+2`, `(3*4)-5=7`, or `2=2=2`.
## Usage
```python
xgr.Grammar.from_ebnf(
ebnf_string: str,
*,
root_rule_name: str = "root",
) -> xgr.Grammar
```
- `ebnf_string` is the grammar source.
- `root_rule_name` is the name of the entry rule. By default the grammar must define a rule named
`root`; pass a different name to use another rule as the entry point.
The returned grammar is compiled and matched like any other grammar:
```python
tokenizer_info = xgr.TokenizerInfo.from_huggingface(tokenizer)
compiler = xgr.GrammarCompiler(tokenizer_info)
compiled = compiler.compile_grammar(grammar)
matcher = xgr.GrammarMatcher(compiled)
```
`compile_grammar` also accepts the EBNF string directly, in which case the string is used as the
cache key of the compiler's cache:
```python
compiled = compiler.compile_grammar('root ::= "a" | "b"')
```
Errors in the grammar raise `RuntimeError` with the line and the column of the offending
construct:
```text
EBNF parser error at line 1, column 10: Rule "a" is not defined
```
## Grammar Structure
A grammar is a sequence of rule definitions:
```text
name ::= expression
```
- A rule name must be the first token on its line. Apart from this requirement, whitespace and
line breaks are insignificant, so a rule body may span multiple lines:
```text
root ::= "a" |
"b" |
"c"
```
- Names consist of letters, digits, underscores (`_`), hyphens (`-`), and dots (`.`), and cannot
start with a digit. Unlike Lark, there is no case convention: EBNF has no separate terminal
concept, and every definition is a rule.
- Comments start with `#` and run to the end of the line.
- Each rule name may be defined only once. Rules may reference each other in any order, including
forward references. Both right recursion and left recursion are supported:
```text
root ::= root "a" | "a" # left recursion: accepts "a", "aa", "aaa", ...
```
## Elements
### String Literals
String literals use double quotes: `"hello"`. Matching is case-sensitive and exact. The empty
literal `""` matches the empty string, which is useful for making an alternative optional:
```text
sign ::= "" | "+" | "-"
```
Supported escape sequences:
| Escape | Meaning |
| -------------------------------------------- | --------------------------------------------------------------------- |
| `\"` `\\` `\/` `\'` `\?` | the character itself |
| `\n` `\r` `\t` `\b` `\f` `\a` `\v` `\0` `\e` | control characters (`\e` is ESC, `\0` is NUL) |
| `\xHH...` | code point from hex digits (arbitrary length, e.g. `\x41`, `\x1F600`) |
| `\uXXXX` | code point from exactly 4 hex digits |
| `\UXXXXXXXX` | code point from exactly 8 hex digits |
Non-ASCII characters may also be written directly: `"中文"`, `"😀"`. A literal cannot contain a
raw newline; write `\n` instead.
### Character Classes
`[...]` matches exactly one character, like in regular expressions:
- `[abc]` matches `a`, `b`, or `c`.
- `[a-zA-Z0-9]` matches one character from any of the ranges.
- `[^0-9]` is negated: it matches any character except a digit.
- `[^]` matches any character (nothing is excluded), including newline.
Characters and range endpoints may be any Unicode character (`[\u4e00-\u9fff]` matches one common
CJK character) and may use the same escapes as string literals. Additionally, the characters
`^ $ \ . * + ? ( ) [ ] { } | / -` can be escaped with a backslash; escape `-` as `\-` when it
should be a literal dash rather than a range separator. A raw newline is not allowed inside a
character class; write `[\n]`.
Regex-style shorthand classes such as `\d`, `\w`, `\s` are **not** supported inside character
classes and are rejected. Write the ranges explicitly (`[0-9]`, `[a-zA-Z0-9_]`, `[ \t\n\r]`).
### Rule References
Using a rule name inside an expression matches that rule:
```text
root ::= value ("," value)*
value ::= [0-9]+
```
Referencing an undefined rule is an error.
### Sequences, Alternatives, and Groups
| Form | Example | Meaning |
| ----------- | ------------------ | ------------------------------------------------ |
| Sequence | `"a" "b"` | Match the elements in order. |
| Alternative | `"a" \| "b"` | Match any one branch. Each branch is a sequence. |
| Group | `("a" \| "b") "c"` | Group a sub-expression; may carry repetition. |
| Empty group | `( )` | Matches the empty string. |
## Repetition
A repetition operator follows an element — a string literal, a character class, a rule reference,
or a parenthesized group:
| Form | Meaning |
| -------- | ----------------- |
| `x?` | zero or one |
| `x*` | zero or more |
| `x+` | one or more |
| `x{3}` | exactly 3 |
| `x{2,4}` | 2 to 4, inclusive |
| `x{2,}` | 2 or more |
The lower bound is required: `x{,4}` is not accepted (use `x{0,4}`). `x{0}` is allowed and
matches the empty string. Two operators cannot be applied directly to the same element (`"a"++`
is an error); wrap the element in parentheses instead: `("a"+)+`.
```text
root ::= identifier ("," identifier){0,4}
identifier ::= [a-zA-Z_] [a-zA-Z0-9_]*
```
## Rule Options
A rule may carry a comma-separated option list between its name and `::=`:
```text
value[max_tokens=32, max_chars=128, capture="answer", lazy, temperature=0.7] ::= [a-z]+
```
Each option may appear at most once. Options without a value, such as `capture` and `lazy`, are
written as bare names; the other options use `name=value`.
### Token Budgets
The `max_tokens` option gives each occurrence of a rule a token budget:
```text
root ::= "" reasoning "" answer
reasoning[max_tokens=512] ::= [^]*
answer ::= [0-9]+
```
Once the budget is exhausted, the token mask forces the occurrence to end if its body can end at
the current position. If it cannot end there, the budget is relaxed until the earliest valid
boundary, so the output remains grammar-valid. If this relaxation causes a derivation to consume a
token past its budget, the matcher logs a warning the first time, at most once per matcher. When
authoring a budgeted rule, prefer a body that can end at every possible budget boundary. For
example, `[^]*` can end at any position, so its budget is exact; bodies that cannot end where the
budget runs out remain best-effort.
The budget applies independently to every occurrence. To bound a whole repeated region, put the
option on a wrapper rule. Nested budgets use the smallest active budget. Use a positive decimal
integer for `N`.
`max_tokens` is enforced during token-mask-driven generation. `accept_string` has no token
boundaries and does not consume the budget. Rollback, reset, forking, and speculative decoding
preserve the budget state.
### Character Budgets
The `max_chars` option limits each rule occurrence by Unicode codepoints, the numeric values
assigned to text units by the Unicode standard:
```text
root ::= "" reasoning "" answer
reasoning[max_chars=2048] ::= [^]*
answer ::= [0-9]+
```
For example, `a`, `é`, and `中` each consume one character. A decomposed character such as `e`
followed by a combining accent consumes two. The count is independent of the number of bytes in
UTF-8, a variable-width Unicode encoding, or model tokens, the units generated by the model. This
also holds when one codepoint is split across multiple tokens.
Enforcement follows the same earliest-valid-boundary rule as `max_tokens`. If the body cannot end
when the character budget is exhausted, the matcher relaxes the budget until the earliest valid
end, so the output remains grammar-valid. If this relaxation causes a derivation to consume a
codepoint past its budget, the matcher logs a warning the first time, at most once per matcher.
When authoring a character-budgeted rule, prefer a body that can end at every possible character
boundary. For example, `[^]*` can end after every codepoint, so its budget is exact; bodies that
cannot end where the budget runs out remain best-effort.
The budget applies per occurrence, nested character budgets use the smallest active budget, and
`max_chars` may be combined with `max_tokens`; whichever boundary is reached first takes effect.
Unlike `max_tokens`, character budgets also apply to `accept_string`. Token acceptance, rollback,
reset, forking, and speculative decoding preserve the character count. Use an integer from 0 to
2,147,483,647. A zero budget closes the rule immediately when its body can end at the entry
position.
### Capture Groups
The `capture` option records the input span matched by a rule:
```text
root ::= item ("," item)*
item[capture] ::= [0-9]+
```
`capture` uses the rule name as the capture name. Use `capture="name"` to choose a different,
non-empty name:
```text
item[capture="number"] ::= [0-9]+
```
Retrieve completed captures from the matcher:
```python
matcher.accept_string("1,22,333")
matcher.get_captures()
# [("number", b"1"), ("number", b"22"), ("number", b"333")]
```
Captures are recorded when tokens or strings are accepted. Filling a token mask does not record
captures, and `rollback` also rolls them back. A repeated rule produces one capture for every
occurrence, in completion order.
The parser may keep several possible parses alive at once, so one occurrence can complete at
several candidate positions. By default, `get_captures(deduplicate=True)` keeps the longest
completion for each occurrence. This is exact when a following delimiter cannot also be matched
by the captured body. Use `get_captures(deduplicate=False)` to retrieve every raw completion
event.
### Committed-Shortest Matching
The `lazy` option makes a rule commit to its earliest possible completion:
```text
root ::= "<" name ">" rest
name[lazy] ::= [a-z]+
rest ::= [a-z]+
```
Here `name` consumes exactly one letter: as soon as `[a-z]+` can end, all alternatives in which
the same occurrence continues are discarded.
The body of a lazy rule must compile to a single terminal-like form made from string literals,
character classes, `Regex` expressions, and supported repetitions or alternatives of those
elements. Bodies that still require rule references, recursion, or repetition ranges after
normalization are rejected during compilation.
A lazy rule that can match the empty string always commits to the empty match. Each occurrence
commits independently, and `rollback`, `reset`, and `fork` restore the commit state.
### Sampling Temperature
The `temperature` rule option selects the sampling temperature while a rule is active:
```text
root ::= "answer:" value
value[temperature=0.7] ::= [a-z]+
```
`temperature` must be a finite non-negative number. An inner explicit temperature overrides an
inherited outer temperature. At ambiguous positions, different active temperatures produce a
warning once and `matcher.temperature` returns the maximum. If no active rule specifies a
temperature, it returns the matcher's `default_temperature`; if neither is configured, it returns
`None`.
Use `BatchGrammarMatcher.batch_fill_temperature` to fill the temperatures of multiple matchers
into a pre-allocated tensor. During speculative decoding, the optional `temperatures` tensor
passed to `GrammarMatcher.traverse_draft_tree` receives the effective temperature for each tree
node. In both tensors, `-1` means there is no effective temperature.
### Generated Suffix and Stop Metadata
The Lark frontend provides high-level `suffix`, `stop`, and `stop_capture` options for terminating
a lazy rule at a marker and controlling whether that marker appears in captures. EBNF does not
accept `suffix=...` or `stop=...` directly. Instead, `str(grammar)` lowers them to the following
EBNF rule options so that the printed grammar can be parsed again without losing capture
behavior:
- `capture_hidden_suffix_bytes=N` marks a trailing suffix marker as hidden from the annotated
rule's own capture while retaining it in enclosing captures. For a fixed marker, `N` is its byte
length.
- `capture_hidden_stop_bytes=N` marks a trailing stop marker as hidden from the annotated rule and
every enclosing capture. For a fixed marker, `N` is its byte length.
- `capture_hidden_body_rule_id=N` and `capture_hidden_marker_rule_id=N` identify helper rules used
to recover the boundary of a variable-length marker. They must be specified together.
- `stop_capture="name"` records the marker bytes under `name` before they are hidden from other
captures.
These options affect only capture materialization. The marker bytes remain in the accepted model
output. The numeric values are internal rule IDs and marker metadata generated by XGrammar;
changing rule order or editing the values can invalidate their meaning. They are intended for
`str(grammar)` / `Grammar.from_ebnf()` round trips rather than hand-authored grammars. Author
suffix and stop behavior through the
[Lark frontend](lark_grammar.md#the-suffix-and-stop-attributes).
## Macros
Macros look like function calls inside a grammar rule. For example:
```text
root ::= Regex("[0-9]{5}", json_string=false)
```
Here, `Regex` is the macro name, `"[0-9]{5}"` is a positional argument, and
`json_string=false` is a named argument. This macro matches exactly five digits.
Macros can also take rule names and tuples:
```text
root ::= TagDispatch(("", tool), loop_after_dispatch=false)
tool ::= "get_weather"
```
In this example, `("", tool)` pairs the trigger string `""` with the `tool` rule.
Each macro accepts its own set of arguments; the sections below describe them individually.
Macro calls can be used in sequences, alternatives, and groups like other grammar elements.
To add a new macro, see [Extending Macros](#extending-macros).
### Accepted Arguments
A macro call contains comma-separated positional arguments and named arguments:
```text
MacroName(value, option=value)
```
The parser accepts only these value forms:
- a double-quoted string literal, such as `"text"`;
- a decimal integer literal, such as `128010`;
- the boolean literal `true` or `false`;
- a rule name, such as `tool`;
- a tuple of values, such as `("trigger", tool)` or `(1, 2, 3)`. Tuples may be empty and may
have a trailing comma.
The macro call itself cannot have a trailing comma. Named arguments use `name=value`; they are
not allowed inside tuples. Character classes, grammar expressions, nested macro calls, floating
point numbers, lists, and objects cannot be passed as values.
### `Regex`
`Regex` matches text described by a regular expression.
```text
root ::= "ID-" Regex("[0-9]{5}")
```
This grammar accepts `ID-12345`.
**Arguments**
- `pattern` is the only positional argument. It is required and must be a string.
- `json_string` is an optional named argument. It must be a boolean and defaults to `false`.
- `flags` is an optional named argument. It must be a string of regular-expression flags, with
the same meaning as the flags of a Lark regex literal: `i` makes the match ASCII
case-insensitive, `s` makes `.` match newlines, and `u` is accepted as a no-op (patterns
always use Unicode codepoint semantics). Other flags raise an error. When `flags` is given,
`.` follows the standard semantics and does not match `\n` unless `s` is present; without the
argument, the pattern is used verbatim and `.` matches every codepoint.
- No other positional or named arguments are accepted.
`pattern` uses the same regular expression syntax as
[`xgr.Grammar.from_regex`](xgrammar.Grammar.from_regex). Backslashes must be escaped for the
surrounding EBNF string literal, so the regular expression `\d+\.\d+` is written as
`Regex("\\d+\\.\\d+")`.
```text
root ::= Regex("hello, world.", flags="is")
```
This grammar matches `hello, world` in any ASCII case, followed by any character including a
newline.
When `json_string=true`, the regular expression is intended to match the body of a JSON string.
Control characters, `"` and `\` are then excluded from every character match, preventing the
expression from consuming characters that must be escaped in JSON:
```text
string_body ::= Regex("\\S+", json_string=true)
```
### `Substring`
`Substring` matches every contiguous subsequence of a fixed list of chunks, including the empty
one.
```text
root ::= Substring("abc", "de", "fg")
```
This grammar accepts `""`, `"abc"`, `"de"`, `"abcde"`, `"defg"`, and `"abcdefg"`, but not `"ab"`
or `"cde"`: chunks are atomic and cannot be split.
**Arguments**
- Every positional argument must be a string. Chunks may be empty or repeated. Zero or more
chunks are accepted.
- No named arguments are accepted.
The chunk list is compiled into a suffix automaton, so the number of automaton states grows
linearly with the number of chunks. This is also the compiled form of the Lark
`%regex {"substring_chunks": ...}` and `%regex {"substring_chars": ...}` extensions.
### `TagDispatch`
`TagDispatch` implements the common tool-calling pattern: the model produces free text until a
trigger tag appears, then the output must follow the grammar associated with that tag, and
afterwards the model returns to free text.
```text
root ::= TagDispatch(
("", tool_call_body),
("", code_body),
loop_after_dispatch=true,
excludes=()
)
tool_call_body ::= "{" [^}]* "}"
code_body ::= "```" [^`]* "```"
```
**Arguments**
- Each positional argument must be a `(tag, rule)` pair. `tag` must be a non-empty string, and
`rule` must be the name of a defined rule. Zero or more pairs are accepted.
- `loop_after_dispatch` is an optional named argument. It must be a boolean and defaults to
`true`.
- `excludes` is an optional named argument. It must be a tuple of non-empty strings and defaults
to `()`.
- No other named arguments are accepted.
Dispatch then works as follows:
- Any text that does not contain a tag is accepted, and the match may end at any point in this
free-text state. An incomplete tag prefix at the end of the output (for example a final
`{"name": "get_weather"} Done.
no tool call at all
```print(1)```{}
```
`excludes` lists strings that must not appear in the free-text portion. This is typically used
together with an outer stop tag, so that the stop tag reliably terminates the dispatch loop
instead of being swallowed as free text:
```text
root ::= body ""
body ::= TagDispatch(
("", tool_call_body),
excludes=("",)
)
tool_call_body ::= "{" [^}]* "}"
```
An excluded string must not be a prefix of any trigger tag.
### `Token`
`Token` matches exactly one token whose ID is in a given set.
```text
root ::= Token(128010) content
content ::= [a-z]+
```
**Arguments**
- One or more positional arguments are required.
- Every argument must be a non-negative integer token ID.
- Named arguments are not accepted.
This macro matches token IDs in the model's vocabulary rather
than decoded text, making it useful for special tokens represented as single tokens. Token-level
and character-level elements can be mixed in one rule.
### `ExcludeToken`
`ExcludeToken` matches exactly one token whose ID is **not** in a given set.
```text
root ::= ExcludeToken(128010, 128011)
```
**Arguments**
- One or more positional arguments are required.
- Every argument must be a non-negative integer token ID.
- Named arguments are not accepted.
Duplicate token IDs are merged. Use repetition to accept more than one unrestricted token, or use
`TokenTagDispatch` for a dispatching loop.
### `TokenTagDispatch`
`TokenTagDispatch` is the token-level counterpart of `TagDispatch`: it accepts free tokens instead
of free text, and each trigger is a single token ID.
```text
root ::= TokenTagDispatch(
(128010, tool_call_body),
loop_after_dispatch=true,
excludes=(128020,)
)
tool_call_body ::= "{" [^}]* "}"
```
**Arguments**
- Each positional argument must be an `(id, rule)` pair. `id` must be a non-negative integer token
ID, and `rule` must be the name of a defined rule. Zero or more pairs are accepted.
- `loop_after_dispatch` is an optional named argument. It must be a boolean and defaults to
`true`.
- `excludes` is an optional named argument. It must be a tuple of non-negative integer token IDs
and defaults to `()`.
- No other named arguments are accepted.
Dispatch then works as follows:
- Before a trigger token appears, any token is accepted except those listed in `excludes`.
- When a trigger token is produced, matching dispatches to the corresponding rule; after the rule
completes, `loop_after_dispatch` behaves as in `TagDispatch`.
- Trigger token IDs must not overlap with `excludes`.
### Extending Macros
XGrammar does not currently provide a public registration interface for custom macros. Adding one
requires changing the C++ source. There are two implementation paths:
1. **Expand to existing grammar elements.** Add a parser function in `cpp/grammar_parser.cc`,
register its name in the parser's `kMacroFunctions` dispatch table, validate its arguments, and
use the internal `GrammarBuilder` helper to construct existing grammar elements. This is the
smallest approach. Printing the resulting grammar shows the expanded form rather than the
original macro call.
2. **Add a new internal grammar-element type.** Use this when the macro needs behavior that
existing elements cannot represent. In addition to parsing and registration:
- Add an entry and data layout to the internal `Grammar::Impl::GrammarExprType` enum in
`cpp/grammar_impl.h`, plus construction methods in `cpp/grammar_builder.h` and
`cpp/grammar_builder.cc`.
- Add printing support in `cpp/grammar_printer.h` and `cpp/grammar_printer.cc` so that
`str(grammar)` can be parsed again.
- Update the internal grammar traversal helper `GrammarFunctor` and every normalization or
analysis pass that switches on the grammar element type in `cpp/grammar_functor.h` and
`cpp/grammar_functor.cc`.
- Implement the runtime semantics. For byte- or token-matching behavior, this usually means
building a finite-state machine (an automaton that tracks accepted input) in
`GrammarFSMBuilder`; specialized behavior may also require updates to the compiler, parser,
or matcher.
- Add parser-error, printing round-trip, serialization round-trip, accepted-input, rejected-input,
and token-mask tests.
The implementations of `Regex`, `TagDispatch`, and `TokenTagDispatch` provide examples of native
grammar elements with different runtime behavior.
## A Complete Example
A grammar for JSON-style quoted strings, showing literals, character classes, repetition, and
recursion working together:
```python
import xgrammar as xgr
grammar = xgr.Grammar.from_ebnf(r"""
root ::= "\"" char* "\""
char ::= [^"\\\x00-\x1f] | "\\" escape
escape ::= ["\\/bfnrt] | "u" [0-9A-Fa-f]{4}
""")
```
This accepts `"hello"`, `"line1\nline2"`, and `"\u00e9"`, and rejects unterminated or badly
escaped strings.
As a larger example, the grammar below describes free-form JSON:
```text
root ::= basic_array | basic_object
basic_any ::= basic_number | basic_string | basic_boolean | basic_null | basic_array | basic_object
basic_integer ::= ("0" | "-"? [1-9] [0-9]*) ".0"?
basic_number ::= ("0" | "-"? [1-9] [0-9]*) ("." [0-9]+)? ([eE] [+-]? [0-9]+)?
basic_string ::= (([\"] basic_string_1 [\"]))
basic_string_1 ::= "" | [^"\\\x00-\x1F] basic_string_1 | "\\" escape basic_string_1
escape ::= ["\\/bfnrt] | "u" [A-Fa-f0-9] [A-Fa-f0-9] [A-Fa-f0-9] [A-Fa-f0-9]
basic_boolean ::= "true" | "false"
basic_null ::= "null"
basic_array ::= "[" ("" | ws basic_any (ws "," ws basic_any)*) ws "]"
basic_object ::= "{" ("" | ws basic_string ws ":" ws basic_any ( ws "," ws basic_string ws ":" ws basic_any)*) ws "}"
ws ::= [ \n\t]*
```
## Relation to Other Grammar Formats
- `str(grammar)` prints any grammar in this EBNF format. The printed form is normalized (for
example, `*` and `+` are expanded into recursive rules) and can be parsed again by `from_ebnf`.
- [`xgr.Grammar.from_json_schema`](xgrammar.Grammar.from_json_schema) and
[`xgr.Grammar.from_regex`](xgrammar.Grammar.from_regex) convert their inputs to EBNF
internally; pass `print_converted_ebnf=True` to inspect the result.
- [`xgr.Grammar.union`](xgrammar.Grammar.union) and
[`xgr.Grammar.concat`](xgrammar.Grammar.concat) combine grammars from any frontend, including
EBNF.
- The [Lark frontend](lark_grammar.md) offers an alternative notation with terminals, `%ignore`,
and inline JSON Schema, and converts to the same internal representation.