nail - Lightning-Fast Data Analysis CLI

nail is a high-performance command-line tool for analyzing, transforming, and exploring Parquet, CSV, JSON, and Excel files. Built with Rust, Apache Arrow, and DataFusion.

Process gigabyte-scale datasets in seconds • SQL-powered • Zero configuration • Works offline • Single binary.

Installation

Prebuilt binary (macOS/Linux — auto-detects your OS and architecture):

curl -fsSL https://raw.githubusercontent.com/Vitruves/nail-parquet/main/install.sh | sh

It installs to /usr/local/bin when writable, otherwise to ~/.local/bin. Set BINDIR to choose the location explicitly, and the installer tells you if the target dir is not on your PATH:

curl -fsSL https://raw.githubusercontent.com/Vitruves/nail-parquet/main/install.sh | BINDIR="$HOME/bin" sh

With Cargo:

cargo install nail-parquet

From source:

git clone https://github.com/Vitruves/nail-parquet
cd nail-parquet
cargo build --release
sudo cp target/release/nail /usr/local/bin/
nail --help

With nix:

nix shell nixpkgs#nail-parquet

Dependencies: macOS — none. Linux — pkg-config and openssl.

Commands

append        Concatenate multiple datasets
binning       Bin continuous variables into categories
convert       Convert between file formats
correlations  Calculate correlation matrices
count         Count total rows
create        Create new columns with expressions
dedup         Remove duplicate rows or columns
describe      Show global file overview and metadata
diff          Compare two datasets and show differences
drop          Remove columns or rows
fill          Fill missing values
filter        Filter rows by conditions
frequency     Calculate frequency distributions
head          Display first N rows
headers       Display column headers
id            Add unique identifier column
merge         Join two datasets
metadata      Show Parquet file metadata
optimize      Optimize Parquet files for better performance
outliers      Detect outliers in data
pivot         Create pivot tables with aggregations
preview       Preview random N rows
rename        Rename columns
sample        Extract data samples
schema        Display schema information
search        Search for values in data
select        Select specific columns or rows
shuffle       Randomly shuffle rows
size          Show data size information
sort          Sort data by columns with various strategies
split         Split data into multiple files
stats         Calculate descriptive statistics
tail          Display last N rows
transpose     Transpose rows and columns
unique        List distinct rows or per-column value counts
update        Check for newer versions
help          Print this message or the help of the given subcommand(s)

Run nail <command> --help for full usage.

Global Options

Available on all commands:

Examples

Explore a dataset:

nail describe sales.parquet
nail stats sales.parquet -c "revenue,profit" --percentiles "0.5,0.9,0.99"
nail correlations sales.parquet -c "price,volume,discount" --tests t_test
nail frequency sales.parquet -c "category,region"

Clean and enrich:

nail dedup raw.parquet --row-wise -c "id" -o unique.parquet
nail outliers unique.parquet -c "price" --method iqr --remove -o cleaned.parquet
nail create cleaned.parquet --column "margin=(price-cost)/price" -o enriched.parquet

Build an analysis pipeline:

nail optimize raw.parquet -o opt.parquet --compression zstd --sort-by "ts,customer_id" --dictionary
nail binning opt.parquet -c "age" -b "18,25,35,50,65" --method custom --labels "18-24,25-34,35-49,50-64,65+" -o binned.parquet
nail pivot binned.parquet -i "age_binned" -c "category" -l "revenue" --agg sum -o summary.parquet
nail stats summary.parquet --stats-type exhaustive -o summary_stats.json

Reshape and summarize:

nail unique sales.parquet -c "category"                 # distinct values of a column
nail unique sales.parquet -c "category,region" --count  # value counts, most frequent first
nail transpose metrics.parquet --header-column metric -o wide.parquet

Compare versions:

nail diff yesterday.parquet --compare today.parquet --keys "id" --changes-only

Piping (stdin/stdout)

Every command can read from stdin and write to stdout, so you can chain nail with itself or other tools. Use -o - on the producer to stream out, and - as the input on the consumer to read in. The input format is auto-detected.

Streaming to stdout defaults to Parquet, which preserves the full schema (types and nested List/Struct/Map columns) losslessly across the pipe. Use -f csv or -f json when you want text output for other tools.

# chain nail commands (Parquet by default — keeps all types)
nail filter sales.parquet -c "revenue > 1000" -o - | nail sort - -c revenue -o - | nail head - -n 10

# stream as CSV/JSON for other tools
nail select sales.parquet -c "id,revenue" -o - -f csv | grep -v '^0,'
nail sample sales.parquet -n 100 -o - -f json | jq '.revenue'

# read from stdin produced elsewhere
cat data.parquet | nail count -

Pipes that close early (| head, | less) are handled cleanly — no broken-pipe errors.

Performance Tips

• Prefer Parquet over CSV for analytical workloads.
• Scope operations with -c regex patterns.
• Use intermediate files for multi-step transforms.
• Tune -j to match your machine.
• Add --verbose to monitor long runs.

License

MIT — see LICENSE.

Contributing

Fork, branch, add tests, ensure cargo test and cargo clippy pass, open a PR.

Support

Issues and questions: https://github.com/Vitruves/nail-parquet/issues