Skip to main content

Pipe Mode (No Server)

LynxDB works without a running server. The CLI can query local files and stdin using the full LynxFlow engine with zero network overhead. Think of it as grep with the power of a full analytics engine.

How It Works​

When you use --file or pipe data through stdin, LynxDB:

  1. Creates an ephemeral in-memory engine
  2. Ingests the data (auto-detects JSON, NDJSON, or plain text)
  3. Runs the full LynxFlow pipeline
  4. Prints results
  5. Exits

No daemon, no config file, no data directory. Everything happens in-process.

Query Local Files​

# Single file
lynxdb query --file access.log 'stats count() by status'

# Glob patterns
lynxdb query --file '/var/log/nginx/*.log' 'where status >= 500 | top 10 uri'

# Multiple file types
lynxdb query --file '/var/log/*.log' 'stats count() by source'

Pipe from Stdin​

# Kubernetes logs
kubectl logs deploy/api | lynxdb query 'stats avg(duration_ms), p99(duration_ms) by endpoint'

# Docker logs
docker logs myapp 2>&1 | lynxdb query 'where contains(_raw, "OOM") | stats count() by container'

# System logs
cat /var/log/syslog | lynxdb query 'where level == "ERROR" | stats count() by service'

# Compressed files
zcat archive.log.gz | lynxdb query 'where status >= 500 | top 10 path'

# Filter before querying
grep "2026-01-15" /var/log/app.log | lynxdb query 'stats count() by level'

Combine with Unix Tools​

LynxDB plays well with the Unix pipeline:

# Output as CSV and sort with system tools
lynxdb query --file access.log 'stats count() as count by status' --format csv | sort -t, -k2 -rn

# Chain with jq for JSON post-processing
lynxdb query --file app.json 'stats count() as count by endpoint' --format json | jq '.count'

# Feed results into another LynxDB query
lynxdb query 'from main | where status >= 500' --since 1h \
  | lynxdb query 'stats count() by path'

# Export to file
lynxdb query --file access.log 'where level == "ERROR"' > errors.json

Output Behavior​

LynxDB is TTY-aware:

  • Terminal (interactive): Colorized JSON output with query statistics
  • Pipe (non-TTY): Plain NDJSON for scripting
# Terminal: colorized output with stats
lynxdb query --file access.log 'stats count() by status'

# Pipe: raw JSON, stats go to stderr
lynxdb query --file access.log 'stats count() by status' | jq .
tip

Metadata and statistics are always written to stderr, so they never pollute piped output. Only result data goes to stdout.

Memory Management​

For large files, you can limit memory usage:

lynxdb query --file huge.log 'stats count() by level' --max-memory 512mb

When the memory limit is reached, LynxDB spills intermediate results to disk automatically.

Real-World Examples​

Debug a Production Issue​

# Find the most common errors in the last hour of logs
tail -n 100000 /var/log/app.log | lynxdb query '
  where level == "ERROR"
  | stats count() as count by message
  | sort -count
  | head 10'

Analyze Nginx Access Logs​

# Top slow endpoints with error rates
lynxdb query --file /var/log/nginx/access.log '
  stats count() as total,
        count(where status >= 500) as errors,
        avg(duration_ms) as avg_lat,
        p99(duration_ms) as p99_lat
    by uri
  | extend error_rate = round(errors / total * 100, 1)
  | where error_rate > 5
  | sort -error_rate
  | keep uri, total, errors, error_rate, avg_lat, p99_lat'

Monitor Kubernetes Pods​

# Error rate by service across all pods
kubectl logs -l app=api --all-containers --since=1h | lynxdb query '
  stats count() as total, count(where level == "error") as errors by service
  | extend error_pct = round(errors / total * 100, 1)
  | sort -error_pct'

Next Steps​