Visit the site 🌐 — Read the book 📖 — Playground 🎮

mq

Query. Filter. Transform Markdown.

mq is a command-line tool that processes Markdown using a syntax similar to jq.

It's written in Rust, allowing you to easily slice, filter, map, and transform structured data.

[!IMPORTANT] This project is under active development.

Why mq?

mq makes working with Markdown files as easy as jq makes working with JSON. It's especially useful for:

• LLM Workflows: Efficiently manipulate and process Markdown used in LLM prompts and outputs
• LLM Input Generation: Generate structured Markdown content optimized for LLM consumption, since Markdown serves as the primary input format for most language models
• Documentation Management: Extract, transform, and organize content across multiple documentation files
• Content Analysis: Quickly extract specific sections or patterns from Markdown documents
• Batch Processing: Apply consistent transformations across multiple Markdown files

Since LLM inputs are primarily in Markdown format, mq provides efficient tools for generating and processing the structured Markdown content that LLMs require.

Features

• Slice and Filter: Extract specific parts of your Markdown documents with ease.
• Map and Transform: Apply transformations to your Markdown content.
• Command-line Interface: Simple and intuitive CLI for quick operations.
• Extensibility: Easily extendable with custom functions.
• Built-in support: Filter and transform content with many built-in functions and selectors.
• REPL Support: Interactive command-line REPL for testing and experimenting.
• IDE Support: VSCode Extension and Language Server Protocol (LSP) support for custom function development.
• Debugger: Includes an experimental debugger (mq-dbg) for inspecting and stepping through mq queries interactively.
• External Subcommands: Extend mq with custom subcommands by placing executable files starting with mq- in ~/.local/bin/.

Installation

Quick Install

curl -sSL https://mqlang.org/install.sh | bash

The installer will:

• Download the latest mq binary for your platform
• Install it to ~/.local/bin/
• Update your shell profile to add mq to your PATH

Cargo

# Install from crates.io
cargo install mq-run
# Install from Github
cargo install --git https://github.com/harehare/mq.git mq-run --tag v0.6.0
# Latest Development Version
cargo install --git https://github.com/harehare/mq.git mq-run --bin mq
# Install the debugger
cargo install --git https://github.com/harehare/mq.git mq-run --bin mq-dbg --features="debugger"
# Install using binstall
cargo binstall mq-run@0.6.0

Binaries

You can download pre-built binaries from the GitHub releases page:

# macOS (Apple Silicon)
curl -L https://github.com/harehare/mq/releases/download/v0.6.0/mq-aarch64-apple-darwin -o /usr/local/bin/mq && chmod +x /usr/local/bin/mq
# Linux x86_64
curl -L https://github.com/harehare/mq/releases/download/v0.6.0/mq-x86_64-unknown-linux-gnu -o /usr/local/bin/mq && chmod +x /usr/local/bin/mq
# Linux arm64
curl -L https://github.com/harehare/mq/releases/download/v0.6.0/mq-aarch64-unknown-linux-gnu -o /usr/local/bin/mq && chmod +x /usr/local/bin/mq
# Windows (PowerShell)
Invoke-WebRequest -Uri https://github.com/harehare/mq/releases/download/v0.6.0/mq-x86_64-pc-windows-msvc.exe -OutFile "$env:USERPROFILE\bin\mq.exe"

Homebrew

# Using Homebrew (macOS and Linux)
brew install mq

Arch

# Using yay (ArchLinux)
yay -S mq-bin

Docker

$ docker run --rm ghcr.io/harehare/mq:0.6.0

Visual Studio Code Extension

You can install the VSCode extension from the Visual Studio Marketplace.

For VS Code compatible editors, it is also available on the Open VSX Registry.

Neovim

You can install the Neovim plugin by following the instructions in the mq.nvim README.

Zed

You can install the Zed extension from the harehare/zed-mq repository.

GitHub Actions

You can use mq in your GitHub Actions workflows with the Setup mq action:

steps:
  - uses: actions/checkout@v6
  - uses: harehare/setup-mq@v1
  - run: mq '.code' README.md

Web

Playground

The Playground lets you run mq queries in the browser with no install.

mq-web (npm)

mq-web is the official WebAssembly build for browser.

Language Bindings

Language bindings are available for Elixir, Python, Ruby, Java, and Go. See the Language Bindings documentation for details.

Usage

For more detailed usage and examples, refer to the documentation.

For a comprehensive collection of practical examples, see the Example Guide.

Basic usage

Usage: mq [OPTIONS] [QUERY OR FILE] [FILES]... [COMMAND]

Commands:
  repl  Start a REPL session for interactive query execution
  help  Print this message or the help of the given subcommand(s)

Arguments:
  [QUERY OR FILE]  
  [FILES]...       

Options:
  -A, --aggregate
          Aggregate all input files/content into a single array
  -f, --from-file
          load filter from the file
  -I, --input-format <INPUT_FORMAT>
          Set input format [possible values: markdown, mdx, html, text, null, raw, bytes, cbor, csv, hcl, json, psv, toml, toon, tsv, xml, yaml]
  -L, --directory <MODULE_DIRECTORIES>
          Search modules from the directory
  -M, --module-names <MODULE_NAMES>
          Load additional modules from specified files
  -m, --import-module-names <IMPORT_MODULE_NAMES>
          Import modules by name, making them available as `name::fn()` in queries
      --args <NAME> <VALUE>
          Sets a named string argument. NAME is accessible directly in queries, and also via ARGS."named" when --args or --argv is given
      --rawfile <NAME> <FILE>
          Sets file contents that can be referenced at runtime
      --stream
          Enable streaming mode for processing large files line by line
  -F, --output-format <OUTPUT_FORMAT>
          Set output format [default: markdown] [possible values: markdown, html, text, json, table, grep, raw, none]
  -U, --update
          Update the input markdown (aliases: -i, --in-place, --inplace)
      --unbuffered
          Unbuffered output
      --list-style <LIST_STYLE>
          Set the list style for markdown output [default: dash] [possible values: dash, plus, star]
      --link-title-style <LINK_TITLE_STYLE>
          Set the link title surround style for markdown output [default: double] [possible values: double, single, paren]
      --link-url-style <LINK_URL_STYLE>
          Set the link URL surround style for markdown links [default: none] [possible values: none, angle]
  -S, --separator <QUERY>
          Specify a query to insert between files as a separator
  -o, --output <FILE>
          Output to the specified file
  -C, --color-output
          Colorize markdown output
  -B, --before-context <NUM>
          Show NUM nodes before each match. Only effective with -F grep
      --after-context <NUM>
          Show NUM nodes after each match. Only effective with -F grep
      --context <NUM>
          Show NUM nodes before and after each match. Only effective with -F grep
      --list
          List all available subcommands (built-in and external)
      --doc
          Use the built-in reference document as input instead of a file
  -P <PARALLEL_THRESHOLD>
          Number of files to process before switching to parallel processing [default: 10]
      --argv [<ARGV>...]
          Positional string arguments, available as ARGS."positional" in queries
  -O, --optimize-level <OPTIMIZE_LEVEL>
          Optimization level for AST transformations (none = no changes, basic = constant folding and dead-branch elimination, full = all passes) [default: none] [possible values: none, basic, full]
  -h, --help
          Print help
  -V, --version
          Print version

# Examples

mq 'query' file.md
mq -f 'file' file.md        # read query from file
mq repl                     # start a REPL session

# Auto-parsing by file extension or -I flag

mq automatically imports the matching module based on the file extension.
Use -I <format> to force a specific format:

.cbor / -I cbor  import "cbor" | cbor::cbor_parse()  (reads as bytes)
.csv  / -I csv   import "csv"  | csv::csv_parse(true)
.hcl  / -I hcl   import "hcl"  | hcl::hcl_parse()
.json / -I json  import "json" | json::json_parse()
.psv  / -I psv   import "csv"  | csv::psv_parse(true)
.toml / -I toml  import "toml" | toml::toml_parse()
.toon / -I toon  import "toon" | toon::toon_parse()
.tsv  / -I tsv   import "csv"  | csv::tsv_parse(true)
.xml  / -I xml   import "xml"  | xml::xml_parse()
.yaml / -I yaml  import "yaml" | yaml::yaml_parse()

Use -I raw   to disable auto-parsing and receive the raw string.
Use -I bytes to read input as raw bytes without parsing.

# Passing arguments to queries (ARGS)

When --args or --argv is given, ARGS = {"positional": [...], "named": {...}}

mq -I null 'name' --args name Alice
mq -I null 'ARGS | ."named"' --args name Alice
# => {"name": "Alice"}

mq -I null 'ARGS | ."positional"' --argv x y z  # must come after query and files
# => ["x", "y", "z"]

mq -I null 'ARGS' file.md --args name Alice --argv x y z
# => {"positional": ["x","y","z"], "named": {"name": "Alice"}}

Here's a basic example of how to use mq:

# Extract all headings from a document
mq '.h' README.md

# Extract only h1 headings
mq '.h(1)' README.md

# Extract h1 and h2 headings
mq '.h(1, 2)' README.md

# Extract headings from level 1 to 3 using a range
mq '.h(1..3)' README.md

# Extract only Rust code blocks
mq '.code("rust")' example.md

# Extract code blocks containing "name"
mq '.code | select(contains("name"))' example.md

# Extract code values from code blocks
mq -A 'pluck(.code.value)' example.md

# Extract language names from code blocks
mq '.code.lang' documentation.md

# Extract URLs from all links
mq '.link.url' README.md

# Filter table cells containing "name"
mq '.[][] | select(contains("name"))' data.md

# Select lists or headers containing "name"
mq 'select(.[] || .h) | select(contains("name"))' docs.md

# Exclude JavaScript code blocks
mq '.code | select(.code.lang != "js")' examples.md

# Convert CSV to markdown table
mq 'include "csv" | csv_parse(true) | csv_to_markdown_table()' example.csv

# Extract a section by title
mq -A 'section::section("Installation")' README.md

# Filter sections by heading level (scalar or range)
mq -A 'section::sections() | section::by_level(2)' README.md
mq -A 'section::sections() | section::by_level(1..2)' README.md

Composing Workflows with Subcommands

mq subcommands are designed to work together via Unix pipes.

# Convert Excel report to Markdown, then extract all headings
mq conv report.xlsx | mq '.h'

# Convert a Word document and extract a specific section
mq conv document.docx | mq -A 'section::section("Summary")'

# Convert and view Markdown directly in the terminal
mq conv slides.pdf | mq view

Run mq --list to see all available subcommands (built-in and external).

External Subcommands

You can extend mq with custom subcommands by placing executable files starting with mq- in ~/.local/bin/ or anywhere in your PATH. This makes it easy to add your own tools and workflows to mq without modifying the core binary.

See the External Subcommands documentation for the full list and details.

Support

• 🐛 Report bugs
• 💡 Request features
• ⭐ Star the project if you find it useful!

License

This project is licensed under the MIT License. See the LICENSE file for details.