Netcap (NETwork CAPture) converts network packets into structured, type-safe Protocol Buffer audit records — designed for security monitoring, forensic analysis, and machine learning. A single Go binary with 83 packet decoders, 40+ stream decoders, and 141+ audit record types, backed by a concurrent architecture and a built-in web UI.

Protocol hierarchy visualization in the Netcap web UI — more screenshots

Features

Protocol Analysis

• 83 packet-layer decoders — Ethernet, IPv4/6, TCP, UDP, DNS, DHCP, ARP, TLS ClientHello/ServerHello, ICMP, NTP, SIP, OSPF, BGP, MPLS, GRE, VXLAN, 802.11, and many more
• 40+ stream decoders — TLS, SSH, HTTP/2, QUIC, SMB, FTP, SMTP, POP3, IMAP, IRC, Kerberos, DCERPC, and more
• Industrial protocols — Modbus, S7Comm, DNP3, OPC-UA, PROFINET, BACnet, CIP, IEC 62351
• Full TCP/UDP stream reassembly with configurable limits

Web UI

Built-in React (Vite + TypeScript) dashboard in service mode with interactive visualizations:

• Sankey diagrams, treemaps, 3D scatter plots, geo maps, host communication graphs
• Record browsing with JSON/UI views and field-level filtering
• Protocol statistics, connection analysis, host profiling, alert management
• Related-evidence timeline on alerts and connections; Settings → Features switches optional features individually

See the Gallery for screenshots.

Security Analysis

• JA4 fingerprinting — BSD-licensed JA4 TLS client fingerprinting; external JA4+ support is available only in opt-in local source builds subject to the FoxIO License
• YARA rules — file scanning with compiled yara-x rules for malware detection
• Magika AI — Google's AI-based file type classification on extracted files
• Credential harvesting — configurable protocol-aware credential capture
• File extraction — extract files from HTTP, FTP, SMTP, POP3, IMAP, SMB, IRC with hashing (MD5, SHA1, SHA256) and MIME detection
• Detection rules — 30+ YAML rule categories covering reconnaissance, exfiltration, web attacks, industrial ports, and more. The expression engine supports source→distinct-destination cardinality (fan-out) detection, an approved-workstation allowlist (IsApprovedWorkstation), and time-of-day helpers (IsBusinessHours, HourOfDay)
• Hunting evidence — DNS query/response pairing with RTT, retransmission and unanswered/late status over UDP and TCP; DCE/RPC calls attributed to interface and operation with fault outcomes; periodic-connection (c2.beacon) detection; signed producer–consumer byte ratio per connection. See Hunting evidence
• Evidence linking — from any record, list the records of the same connection, the DNS answer that resolved its destination, connections opened to an answered address and alerts, as a timeline. See Evidence linking
• OT/ICS threat hunting — function-code level Siemens S7comm detection (write / logic download / logic theft / PLC stop / CPU restart) mapped to CISA AA26-231A; ships internal/rules/examples/s7comm_hunt.yml

Output Formats

• Protocol Buffers (default) — compact binary, accessible from any language
• CSV — configurable separators for data analysis pipelines
• JSON — human-readable structured output
• Elasticsearch — direct bulk indexing for ELK stack analysis

Enrichment

• DNS reverse resolution
• GeoIP geolocation and ASN: bundled DB-IP Lite with optional user-installed GeoLite2. Select providers using -geoProviders dbip,geolite2 (default order) or the Databases page; see GeoIP configuration.
• MAC vendor lookup
• Deep Packet Inspection (optional, via nDPI/libprotoident)
• Hyperscan / Vectorscan acceleration (optional) — multi-pattern regex prefilter for nmap service probes (~2.2× faster), CMS/web framework detection (~1.4×) and rule-engine MatchesPattern (up to ~6× on miss-heavy detection traffic), see docs/hyperscan.md

Integrations

• Prometheus + Grafana — real-time metrics and dashboards
• Elasticsearch + Kibana — full-text search and visualization
• Maltego — 45+ OSINT entity types and transforms

Distributed Capture

Agents stream audit records to a collection server over mutually authenticated TLS 1.3, with pinned keys, acknowledged delivery and reconnects. See docs/distributed-collection.md.

Quick Start

Pre-built binaries are available on the Releases page. To build from source:

# Build (requires libpcap)
go build -o net ./cmd/net/

# Build without DPI (fewer C dependencies)
go build -tags=nodpi -o net ./cmd/net/

# Build with Hyperscan / Vectorscan acceleration for service probes
# (requires libhs via pkg-config; e.g. `brew install vectorscan` on macOS)
# See docs/hyperscan.md for details.
CGO_ENABLED=1 go build -tags hyperscan -o net ./cmd/net/

# Capture from PCAP file
./net capture -read traffic.pcap

# Live capture
sudo ./net capture -iface en0

# Service mode (starts web UI)
./net capture -read traffic.pcap --service

# Service mode with hot reload (development)
air

Network detections

net capture enables capture-time network detections independently of baseline approval. -network-detection=false disables them; -network-detection-config policy.json loads bounded JSON thresholds, authorized source IPs/CIDRs and versioned indicators (internal/networkdetect/config.go).

Defaults: 60 s windows, 300 s alert deduplication, 10 distinct DNS names/TCP endpoints/large echoes, 5 SMTP hosts and 10 MiB SSH client payload. State retains at most 4,096 keys, 128 values per window and 1,024 flows with 4 KiB prefixes per direction. A stream gap stops signature/transfer claims for that direction and is reported. Indicator validity uses Unix nanoseconds at capture time. A missing status file or zero configured indicators is displayed as unavailable coverage.

internal/networkdetect/testdata/live/ contains 15 real FlightSim CLI captures at revision 3709d1c6905d9527885c62f66f49a955c7b0d191, generated with Docker --network none and local DNS/API/SSH/SFTP/IRC/Stratum fixtures. C2 intelligence is synthetic lab data; SSH qualification uses a 1 MiB transfer and a 512 KiB threshold. testdata/cases.json separately labels synthetic source-shape and benign controls. Scenario names never enter the detector. beacon has no FlightSim capture and is synthetic only.

Hunting evidence

These fields turn individual records into answers for common hunts: which resolvers are slow or failing, which remote procedures were actually called and whether they succeeded, and which hosts upload more than they download.

WebUI views:

• Domains shows per-domain median RTT and counts of retried, late, unanswered and reordered transactions.
• Connections shows the producer–consumer ratio with a sent/received label.
• Alerts shows the service, mean interval and jitter behind c2.beacon.

The bundled DCE/RPC rules match specific operations: DCSync (IDL_DRSGetNCChanges), SAMR account enumeration, remote service creation, remote registry writes and remote scheduled task/job creation. After Hours Data Transfer now applies its business-hours condition. Two rules were renamed for what they measure: Symmetric External Connection (byte symmetry, not timing) and DNS Multi-Question Message.

go test -race ./internal/networkdetect
go test -race -tags nodpi,noyara,nomagika ./internal/collector -run '^TestFlightSimCollectorReplay$'

The collector checks replay at 1/2/4/8 workers. The lab requires a 32 MiB tcpdump buffer and a 2 s drain; tests/flightsim-lab/run.sh rejects kernel drops and failed simulator modules before sealing capture hashes.

Evidence linking

Answers "what else happened in this connection or because of this lookup" without manual joins. Links are computed from the output directory when asked, so they work for finished and live captures and do not depend on worker count. With -capture-evidence (default true), packet ingress records a scope ledger in capture-manifest.json. After finalization, the linker qualifies records by capture RunID, local sensor, capture-wide interface index and VLAN stack. PCAPNG interface numbering is normalized across sections. DNS resolution links remain temporal candidates, not proof that a lookup caused a connection.

Per-packet records (TCP, TLSRecord, PacketContext, PKTAP) and records without a computed Community ID (fallback hashes) are not linked. Network-detection alerts carry evidence rather than a matched record, so they are not linked. Known Connection observations suppress fallback flow links when no unique observation contains the target. Duplicate type/Community ID/timestamp selectors are rejected; use the exact type and ordinal instead. Each directory's index has a 64 MiB accounted storage budget as well as its record limit.

The WebUI shows the timeline in alert details (for rule alerts) and in expanded Connections rows. The API selects a record by type+ordinal, by observationId (Connection), by type+communityId+time (Unix ns), or by the content-bound id returned by a previous query:

curl 'http://127.0.0.1:8080/api/evidence/related?observationId=<id>'
net investigate related -read out -type HTTP -ordinal 0

Each link states its kind and basis; notes explain truncation and when no connection contains the record. References include the audit-file fileSha256 and an id derived from type, file digest, capture RunID, scope-manifest digest and ordinal. captureManifestSha256 binds the selected capture/scope witness. Replacing or extending that file invalidates its old references rather than rebinding them. Index construction rejects files changing during the scan. Linked record buttons open another related-evidence timeline; Back returns to the original target. Links are not available to capture-time rules: records are written in a different order than they are captured, so a rule could not see the same links at every worker count.

Capture-time DNS context

-dns-resolution-context (NC_DNS_RESOLUTION_CONTEXT, default false) enables an independent capture-time snapshot on Connection records. The collector pairs UDP DNS at serialized ingress and copies context before dispatch; workers never query mutable cross-flow state. Fields 65–67 are DNSResolvedName, DNSResolvedAt (UTC ns) and DNSResolutionState (resolved or unobserved; empty when disabled). A new observation retains its first packet's context, including after later answers or cache eviction.

Only paired, successful, non-truncated, single-question IN A/AAAA responses on UDP port 53 with an ASCII direct matching answer name contribute. Context expires at the answer TTL, capped at 1 h, and is isolated by capture-wide interface/VLAN and client/destination IP. TCP DNS and CNAME chains are not contributors yet. The answer cache and pending query table each cap at 65,536 entries. unobserved is missing context, not a verdict that no resolution occurred.

type: Connection
expression: 'DNSResolutionMatches(DNSResolutionState, DNSResolvedName, DNSResolvedAt, TimestampFirst, "example.test", 60000000000)'

DNSResolutionMatches is a pure predicate on immutable record fields: exact DNS name comparison (case/trailing-dot normalized), nonnegative bounded age, and resolved state. Settings → Features has a separate switch applying to the next analysis; expanded Connections rows display the snapshot. Go/Rust qualification checks enabled/disabled fields and rule alerts at 1/2/4/8 workers.

Feature switches

Settings → Features lists optional features with their flag and environment variable. GET /api/features returns them; POST /api/features {"name":"evidence-links","enabled":false} switches one until the process exits. Query features (evidence linking) apply to the next request; capture features (network detections) apply to the next analysis started from the WebUI.

Behavioral monitoring

capture -behavior and agent -behavior enable passive, sensor/interface/VLAN- scoped observations. Capture defaults to <output>/Behavior.json; agents use the user-config netcap/behavior/<interface>/ directory. Select an existing baseline explicitly with -behavior-baseline <path>.

./net capture -read traffic.pcap -out capture -behavior -http localhost:8080
./net capture -iface en0 -out live -behavior -behavior-sensor office-router

The synthetic packet-to-SSE gate (TestBehavioralPacketToSSELatency, 100 samples) measured p95 51.85 ms including decode on Darwin ARM64 / Apple M5 Max, Go 1.27.0, at 19.99 synthetic packets/s. TestBehavioralBrowserDelivery measured receipt to React-render p95 99.10 ms in Chrome 154.0.8037.98, with 1,000 approved facts, 100 decisive SYNs over 5.998 s and 58,000 background SYNs at 9,586 packets/s; zero fact/window overflow. Browser approval, reconnect, evidence and suppression history are asserted against the real backend. TestBehavioralCollectorPressure processed 100,000 stable SYNs through 4 protocol workers and audit persistence in 0.406 s (246,310 packets/s), with zero missing observations or state overflow. The browser workload uses synchronous synthetic ingress; the collector workload uses an offline PCAP. Neither measures kernel-capture drops.

Run the opt-in gates after pnpm install --frozen-lockfile && pnpm build in cmd/capture/webui/frontend; the browser gate also requires Google Chrome:

NETCAP_BEHAVIOR_BROWSER=1 go test -race ./cmd/capture/webui -run '^TestBehavioralBrowserDelivery$' -count=1 -timeout 120s
NETCAP_BEHAVIOR_PRESSURE=1 go test ./internal/collector -run '^TestBehavioralCollectorPressure$' -count=1 -timeout 90s

PCAP oracles in behavior_replay_test.go cover discovery/DNS, rates, DHCP transitions, IPv6 conflicts, geography and lateral patterns with equivalent evidence at 1/2/4/8 workers. These scoped results do not establish latency on arbitrary traffic or hardware. Endpoint visibility is limited to traffic at the selected interface; encrypted SSH/RDP connection patterns do not establish authentication failures.

BenchmarkBaselineDurableStorage compares atomic JSON checkpoints with SQLite WAL / synchronous=FULL on the same Mac (100 iterations, 2026-10-06):

Live state remains bounded in memory with portable JSON checkpoints every 1 s; the measured 10,000-fact checkpoint costs 10.987 ms. The SQLite row result is a lower bound for one observation, not a complete lifecycle/state-store port. Alerts remain separately synced per event. TestCorruptBaselineRecoveryFromValidatedTemplate verifies recovery preserves approval and both the damaged file and backup; cmd/agent/behavior_test.go verifies TLS delivery of identical retained alert evidence after a collector outage and local-engine restart.

internal/behavior/testdata/qualification/ pins fixture contract 1: schema-3 approved snapshots, monitoring PCAPs, exact detector/endpoint/evidence oracles and SHA-256 manifests for discovery/DNS, address changes, lateral patterns, rates and geography. The geography case includes synthetic DB-IP MMDBs. NETCAP_BEHAVIOR_EXPORT=<absolute fresh directory> plus NETCAP_BEHAVIOR_REFERENCE=<Go commit> exports the same corpus while running go test ./internal/collector -run 'TestBehavioral.*(Oracle|WorkerCounts)$' -count=1. Export refuses an existing case directory. Decision timestamps are generated at approval; consumers verify the committed bytes against each manifest.

Subcommands

Investigation tools

net investigate flows --read Connection.ncap.gz \
  --start-ns "$START_NS" --end-ns "$END_NS" \
  --filter 'InSubnet(SrcIP, "192.0.2.0/24")' --group-by srcIP

net investigate packet-evidence --read exercise.pcapng \
  --bpf 'host 192.0.2.1 and tcp port 80' --out evidence.zip

net investigate collect-flows --listen 127.0.0.1:2055 --out ./fresh-output --duration 1m
net investigate exported-flows --read ./fresh-output/FlowExports.jsonl \
  --exporter 192.0.2.10:50000 --format netflow-v9 --domain 7 \
  --start-ns "$START_NS" --end-ns "$END_NS" --time-basis receive

START_NS and END_NS are inclusive UTC nanosecond integers. The WebUI API exposes the same flow engine at GET /api/flows/query with startNs, endNs, filter, groupBy, sortBy and limit. Connection downloads accept format=evidence, a TCP/UDP protocol, and optional paired time bounds.

flows --window-mode overlap|contained|start|end selects interval semantics. Optional --bucket-ns emits uniformly interpolated byte/rate estimates, capped at 4,096 bins; raw group counters remain whole-observation counts. Reports also include byte shares and nearest-rank size/packet/duration distributions. The Investigation Evidence page exposes both controls and labels estimated values.

Exports refuse replacement and fail on truncation or exceeded limits. Packet exports cap captured packets at 1 MiB and PCAPNG blocks/metadata at 16 MiB; flow queries cap records at 4 MiB and decoded input at 256 MiB. The 38 book-derived workflow families are qualified in testdata/investigation-coverage.json, with exact fixture references and mandatory integration commands. Coverage is scoped to those fixtures and declared limits; external endpoint, proxy and organizational evidence remains explicitly external.

capture --flow-exports normalizes UDP NetFlow v5/v9, IPFIX and sFlow v5 on --flow-export-ports (default 2055,4739,6343,9995,9996). It records raw datagrams, normalized observations and issues in FlowExports.jsonl before worker dispatch. FlowExportsHealth.json binds final health to the record-file SHA-256. Existing flow artifacts are refused; use a fresh output directory.

capture --conns also writes stream-evidence/stream-*/{client.bin,server.bin,manifest.json}. These bytes contain no ANSI markup. The manifest preserves per-direction offsets, capture times, TCP gaps and UDP datagram boundaries. A gapped stream must not be parsed as contiguous data; the legacy display transcript remains separate.

capture --capture-evidence writes capture-manifest.json with input/configuration hashes, ingress/admission/drop counters and nullable kernel statistics. Add --retain-packets --packet-segment-mb 32 --packet-retention-mb 512 for rotating PCAPNG retention. Expired segments remain recorded as tombstones. A fresh output directory is required; retention failure is an error, not successful collection. The Investigation evidence page (/investigation-evidence) displays this coverage, scoped flow reports and directional stream downloads.

net investigate exchange --spec exchange.json executes one bounded TCP/UDP experiment. Byte fields use JSON base64; Framing supports length-prefix, delimiter and fixed. Response variables capture fresh negotiation values. TLS verifies certificates and hostnames; optional rootCAFile and paired clientCertificateFile/clientKeyFile configure experiment trust and mTLS.

net investigate protocol-fields --spec grammar.json --read client.bin --compare other-client.bin validates framed streams against declared byte/UTF-8/signed/unsigned fields and reports changes by frame ordinal and field name. Fields include offsets, raw bytes and hypothesis descriptions. Frame insertions can shift alignment; a successful parse does not prove the hypothesized application semantics.

net investigate tls-capture --read tls.pcapng --key-log secrets.log --stream 0 uses installed tshark for offline TLS dissection. Output binds input, key-log and plaintext hashes to the tool version; key material is not included. Node ordering follows tshark's reported endpoints, not an assumed client role. NETCAP_REQUIRE_TLS_ADAPTER=1 go test ./internal/protocoltest -run TestTLSCaptureAdapter requires the TLS 1.2/1.3 adapter fixtures instead of skipping when tshark is absent.

net investigate protocol-proxy --spec proxy.json --listen 127.0.0.1:9000 handles one framed TCP connection. Mutations select direction/frame and byte range, with optional length-prefix repair, drop, duplication and delay. Output records original/transmitted bytes. Both tools enforce time/frame/byte limits; response matches are protocol observations, not proof of endpoint effects.

Use each command's --help for the versioned JSON/input contract. FTP data extraction waits for control/data readers to drain; FTPDataHealth.json records associations and exclusions. TCPReassemblyHealth.json records checksum policy and rejection counts. Both sidecars are shown in the Investigation Evidence page and retained by Pro projects.

Docker

Pre-built images are available for multiple configurations:

See the docker/ directory for all Dockerfiles and build variants.

Documentation

• Documentation — full usage guide
• Homepage — project homepage
• DeepWiki — AI-powered codebase exploration
• Thesis — original research paper

Contributing

Contributions welcome — from protocol decoder additions to core framework improvements.

Development Setup:

• macOS Development Setup Guide
• Installation Guide

Please use the bug report template for issue reports.

License

Netcap is licensed under the GNU General Public License v3, which is a very permissive open source license, that allows others to do almost anything they want with the project, except to distribute closed source versions. This license type was chosen with Netcap's research purpose in mind, and in the hope that it leads to further improvements and new capabilities contributed by other researchers on the long term.