Parser Utilities

Every utility on this page is exported from the package root and from the @requence/tokenized-search/core entry point. The core entry contains only the parsing pipeline — no React or TipTap in its module graph (runtime or types) — making it safe to import in server-side code that needs to parse the same queries the <TokenizedSearch /> component produces:

import {
  parseTokenizedSearch,
  parseExpression,
  DEFAULT_MAX_NESTING,
  getOptionDisplayText,
} from '@requence/tokenized-search/core'

The pure parser types (TokenOption, TokenSegment, TextSegment, OperatorKind, OperatorSegment, TokenizedSearchSegment, SearchAst, ExpressionError, ExpressionErrorCode, ParsedExpression, ParseExpressionOptions) are exported from both entries as well. Both entries share one implementation, so client and server parse identically — just keep them on the same package version.

parseTokenizedSearch

Parse a raw search string into segments of recognized key:value tokens and plain text.

import { parseTokenizedSearch } from '@requence/tokenized-search'

const segments = parseTokenizedSearch<'status' | 'source'>(
  'status:active free text',
  ['status', 'source'],
)

Parameters

Parameter	Type	Default	Description
rawText	string	—	The raw search query
tokenKeys	string[]	—	Recognized token key names (and labels)
negationLabel	string	'not'	Negation prefix for detecting negated tokens
parseOperators	boolean	true	Parse AND / OR and ( / ) as operator segments. When false, they stay plain text and parens don’t delimit tokens

Returns

TokenizedSearchSegment[] — array of TokenSegment, TextSegment, and OperatorSegment objects. Operator segments ({ type: 'operator', op: 'and' | 'or' | 'not' | 'open' | 'close' }) are emitted for uppercase AND / OR / NOT and grouping parens ( / ).

---

parseExpression

Build a boolean expression tree from a flat segment stream. Combine it with parseTokenizedSearch to parse a raw query into an AST.

import { parseTokenizedSearch, parseExpression } from '@requence/tokenized-search'

const segments = parseTokenizedSearch('(type:bug OR type:feature) AND assignee:me', keys)
const { ast, valid, errors } = parseExpression(segments)

Parameters

Parameter	Type	Default	Description
segments	TokenizedSearchSegment[]	—	Segments from parseTokenizedSearch
options.maxNesting	number	5	Max parenthesis nesting depth before an error is recorded

Returns

ParsedExpression:

Field	Type	Description
ast	SearchAst | null	The boolean tree (null when there are no tokens)
segments	TokenizedSearchSegment[]	The input segments
expression	string	Human-readable rendering of the tree (key:value, AND(…), OR(…), NOT(…), ∅)
valid	boolean	true when the expression is well-formed
errors	ExpressionError[]	Problems found (unbalanced parens, dangling operators, empty groups, over-nesting), each with a code and character range

Parsing is tolerant: it never throws, and always returns a best-effort tree with details in errors.

---

getOptionDisplayText

Resolve the display text for a token option.

import { getOptionDisplayText } from '@requence/tokenized-search'

const text = getOptionDisplayText({ value: 'active', label: 'Active' })
// → 'Active'

Parameters

Parameter	Type	Description
option	TokenOption	The option to resolve

Returns

string — option.label ?? option.value.