Pandoc Backend

Overview

The pandoc path is **parse with pandoc, then apply snapper** — not “guess markdown lines like the built-in parsers.”

  1. Pandoc reads the file (any format it supports) into its document AST.

  2. Snapper walks that AST and reflows only prose-bearing nodes (Para / Plain).

  3. Nodes pandoc already classified as structure (Header, CodeBlock, Table, …) are left alone.

  4. The mutated AST is written through an in-process FFI writer when the loaded library exports one, otherwise pandoc -f json --wrap=preserve.

This is not a splice of the original bytes.

That is how snapper can format everything pandoc can read (typst, asciidoc, docx, html, …) without inventing per-format line heuristics.

How to run step 1 (same step-2 walker):

  1. Auto — --pandoc-backend auto (default): prefer in-process FFI when libsnapper_pandoc loads, else CLI.

This amortizes the GHC RTS for parse. The shipped library is reader-only, so the writer still needs pandoc on PATH unless that library exports a writer (no silent spawn).

  1. CLI — --pandoc-backend cli: pandoc -t json (full reader set of the installed binary).

Explicit error if pandoc is missing.

  1. FFI — --pandoc-backend ffi: require libsnapper_pandoc (explicit error if missing).

Writer still needs pandoc on PATH when the library has no writer.

Requirements

**CLI backend.** Pandoc 2.x or 3.x on PATH. The CLI default uses pandoc when an FFI writer or pandoc on PATH is available. Otherwise it keeps the native line parsers (no error, no silent all-prose). With --use-pandoc, a missing or failing pandoc is an explicit error (no silent all-prose). Pass --native to force today’s line parsers.

**Writer.** The library libsnapper_pandoc is reader-only. The write step still needs pandoc on PATH (help and errors say so). A library that exports snapper_pandoc_write stays in-process for write too.

**FFI backend (dynamic).** Build native/snapper-pandoc, set SNAPPER_PANDOC_LIB to libsnapper_pandoc.so (or put it on the search path). A missing library with --pandoc-backend ffi errors with pandoc FFI library unavailable (fail-closed; never silent all-prose).

**FFI backend (co-linked / one binary).** Feature pandoc-colink is a build path: produce libsnapper_pandoc.a with native/snapper-pandoc/build-static.sh (ghc -staticlib), then Cargo absorbs that archive into a single snapper executable (--gc-sections). No SNAPPER_PANDOC_LIB discovery and no multi-hundred-MB libHS* RUNPATH graph.

cd native/snapper-pandoc && ./build-static.sh
export SNAPPER_PANDOC_LIB_DIR=$PWD/lib
cargo build --release --features "cli,pandoc,pandoc-colink"
snapper --use-pandoc --pandoc-backend ffi paper.md

See native/snapper-pandoc/README.md.

Usage

# Default: pandoc when FFI or pandoc on PATH is available
snapper paper.md
snapper --pandoc-backend auto paper.md
snapper --pandoc-backend cli guide.adoc
snapper --pandoc-backend ffi paper.md
# Require pandoc (error if missing)
snapper --use-pandoc paper.typ
# Force today's line parsers
snapper --native paper.md

The CLI default uses pandoc (auto FFI, then CLI) when an in-process writer or pandoc on PATH is available. Otherwise it keeps the native line parsers (no error, no silent all-prose). Pass --native to force today’s line parsers. Pass --use-pandoc to require the pandoc path (explicit error if FFI and pandoc are missing). The writer still needs pandoc on PATH unless the library exports a writer.

**Speed.** Warm in-process FFI parse is a few times native cost; a cold pandoc CLI process reloads the GHC RTS (tens of ms). The writer is that CLI spawn unless the FFI library exports a writer — it is never a surprise. Snapper also caches pandoc JSON ASTs by content hash (memory + disk under $XDG_CACHE_HOME/snapper/pandoc-ast). Re-formatting the same source skips pandoc parse. Disable with SNAPPER_PANDOC_CACHE=0; override dir with SNAPPER_PANDOC_CACHE_DIR.

When to use pandoc vs built-in parsers

  • **Pandoc** (CLI default when available): structure from pandoc’s model and coverage of formats without a built-in parser.

  • **Built-in** (--native): fast, source-oriented parsers for Org / LaTeX / Markdown / RST (e.g. full ATX lines as structure).

The two paths are different pipelines, not two ways to fake the same source lines.

Limitations

  • Pandoc’s reader deletes comments (RST .. comments, HTML comments) and therefore snapper:off / snapper:on.

  • --use-pandoc shields RST .. comments and snapper:off / snapper:on so the written output still contains that text and exits 0.

  • Other HTML / Org / LaTeX comments that are not snapper pragmas may still be dropped.

  • Round-trip is through the AST (not a perfect source-preserving rewrite of every markup character)

  • FFI library is optional; wasm/editor bundles do not embed GHC/pandoc

  • CLI pandoc versus editor native is the contract: the CLI default may use pandoc; VS Code, Obsidian, Word, wasm, and LSP stay on native parsers.