Skip to content

cl-parser-kit

cl-parser-kit is a small, practical parser toolkit for Common Lisp. It focuses on the pieces that recur in real text-language parsers: tokenization, source spans, parser combinators, Pratt parsing, structured diagnostics, and AST/CST helpers — deliberately compact rather than a compiler framework.

New to cl-parser-kit?

Load it into any ASDF-visible checkout and tokenize your first source string in a few lines:

(asdf:load-system :cl-parser-kit)

(cl-parser-kit:tokenize-string
 "sum + 42"
 (cl-parser-kit:make-tokenizer
  :rules (list (cl-parser-kit:make-whitespace-rule :skip-p t)
               (cl-parser-kit:make-literal-rule :plus "+")
               (cl-parser-kit:make-number-rule)
               (cl-parser-kit:make-identifier-rule))))

Continue with Getting Started → Core Concepts.

Explore the docs

Status

This repository is in active development, but the core API is already usable. The current codebase includes:

  • a rule-based tokenizer with literal, keyword, identifier, number, string, predicate, char, whitespace, and line/block comment rules
  • span, token, and diagnostic types with structured notes and fix-its
  • parser combinators — sequencing, alternation, repetition, lookahead, failure propagation, and practical sep-by / preceded-by / between style helpers
  • a Pratt parser for prefix, infix, and postfix expression grammars
  • AST/CST constructors and inspection helpers (walk, find, map, collect, count, depth, stable *-node->sexp conversion)
  • exported resource-limit specials so hostile input fails gracefully instead of exhausting memory or the control stack
  • a test system wired into asdf:test-system, with parser-table invariants additionally checked as executable cl-prolog-kit/weave queries
  • runnable examples under examples/, regression-tested as user-facing workflows

The implementation is designed to stay small enough that the behavior is easy to audit from the tests. See Architecture for the layer model and Roadmap for the remaining public-facing work.

Guide Map

  • Getting Started — ASDF, Quicklisp, Ultralisp, and Nix install paths, the verification commands for a release-ready checkout, and the first runnable tokenizer and parser.
  • Recipes — parser combinators, Pratt parsing, seq helpers, lookahead, operator chains, and CST output in runnable snippets.
  • Diagnostics — rendering a parse failure with source excerpts, and building a diagnostic by hand.
  • Core Concepts — tokens, spans, tokenizers, the parser layer, the Pratt layer, diagnostics, and AST/CST helpers.
  • Parsing Patterns — how to choose the smallest stable layer for a grammar, the committed-failure contract, and how to upgrade an existing hand-written parser.
  • Examples — a map of every sample file under examples/ and the recommended reading order.
  • API Reference — the exported surface grouped by concern, with the common entry point for each layer.
  • Architecture — the layer model and dependency direction.
  • Roadmap — the near-term, mid-term, and explicit non-goals.
  • Compatibility — what v1.0.0 froze and what it did not.
  • Development — the nix flake check gate and the raw-checkout entry points.

Design Non-Goals

This project is deliberately not trying to be a CLI framework, a terminal/TTY layer, a Prolog engine, an event system, a dataflow engine, a generic utility package, large compiler infrastructure, a full language workbench, or an editor integration layer.

Nix Workflow

The flake.nix at the repository root packages cl-parser-kit as a Nix flake:

  • nix develop — a devShell with SBCL and Perl, with CL_PARSER_KIT_CL_WEAVE_ROOT and CL_PARSER_KIT_CL_PROLOG_KIT_ROOT pre-wired to the pinned test dependencies, and paredit-cli for structural S-expression lint checks.
  • nix flake check — the full reproducible CI gate: the library package build, the cl-weave/cl-prolog-kit test suite, the 90%/80% coverage gate, paredit-lint, the nixfmt formatting gate, and the --strict docs build, for x86_64-linux, aarch64-linux, and aarch64-darwin.
  • nix build .#docs — builds this documentation site with MkDocs (Material) in --strict mode, so broken links fail the build.
  • nix run .#test — the test suite alone, against the working tree.
  • nix fmt — formats Nix sources with nixfmt via treefmt.

Running direnv allow loads the devShell automatically.

Support

Use Support Policy for the canonical support boundary and release-readiness expectations.

Use private GitHub security advisories for vulnerability reporting — see Security Policy. Do not put exploit details in a public issue.

Project Operations

The contribution, conduct, governance and support policies are org-wide and published from the nerima-lisp/.github repository, so they are not duplicated here:

Repository-specific:

License

MIT. See LICENSE.