Waytermirror

Real-time Wayland screen mirroring to a terminal or native window using Unicode braille characters, half‑blocks, ASCII, sixels, kitty graphics, framebuffer, KMS direct rendering, native GUI window, or hybrid rendering. Includes bidirectional input forwarding, audio streaming (PipeWire), zooming, focus-follow, and optional NVIDIA CUDA acceleration (server-side).

Table of contents

• Waytermirror
  • Table of contents
  • What it is
  • Demonstration
  • How it works (short)
  • Quickstart
  • Build & install
    • Prerequisites
    • Arch Linux-based
    • Debian-based
    • Building
    • Artifacts
    • Packaging (Arch)
  • Runtime requirements
  • Usage
    • Server
      • Full server options
    • Client
      • Full client options
  • Examples
  • Keyboard shortcuts (client)
    • Session control
    • Input control
    • Zoom control
    • Rotation
    • Rendering
    • Output & FPS
    • Audio
    • Quick usage tips
  • Network ports
  • Performance tuning
    • Hardware-accelerated pixel rendering (best visual quality)
    • Unicode rendering (terminal-based)
    • Encoder-specific tuning
  • Troubleshooting
  • Security & limitations
  • Design notes & behavior
    • Rendering architecture
    • Hardware acceleration
    • Compression
    • Hybrid renderer
    • GUI renderer
  • Contributing
  • License

What it is

• A client/server application:
  • Server runs on the Wayland host (captures screen & audio, performs rendering, injects input).
  • Client runs locally (receives frames, displays them, captures input, sends it to server).
• Rendering modes:
  • Terminal-based: braille, half-blocks, ASCII, hybrid (auto-select per cell)
  • Pixel-based: sixels, kitty graphics, framebuffer (/dev/fb0), KMS direct rendering, native Wayland window (GUI)
• Color modes: 16, 256, truecolor (24‑bit). (applies to terminal-based renderers only)
• Optional CUDA acceleration for server-side Unicode rendering (NVIDIA only).
• Hardware-accelerated H.265/HEVC encoding for pixel-based renderers with automatic hardware detection (NVENC, QuickSync, VAAPI, AMF, VideoToolbox).
• Native GUI mode: Client creates a Wayland window for a native application experience (no terminal required).

Demonstration

https://github.com/user-attachments/assets/089d1c60-c502-4422-892c-fb83e392149a

How it works (short)

• Screen capture: wlr-screencopy/PipeWire → frame buffer → renderer (CPU/CUDA)
• Unicode renderers (braille/blocks/ascii/hybrid): ANSI escape sequences → LZ4 → TCP → client terminal
• Pixel renderers (sixel/kitty/framebuffer/KMS/GUI): RGB24 → H.265 encoder (hardware-accelerated) → TCP → client decoder → display
• Input: Client libinput capture → TCP → server → virtual pointer/keyboard (Wayland) on host
• Audio: System audio (server→client) and microphone (client→server) via PipeWire with optional Opus compression
• Hardware encoder priority: NVENC (NVIDIA) → QuickSync (Intel) → VAAPI (Intel/AMD) → AMF (AMD) → VideoToolbox (macOS) → software fallback (x265)
• Hardware decoder priority: CUDA (NVIDIA) → VAAPI (Intel/AMD) → QuickSync (Intel) → VDPAU (NVIDIA Legacy) → VideoToolbox (macOS) → multi-threaded software fallback

Quickstart

1. Build (CPU-only)

   git clone https://github.com/cyber-wojtek/waytermirror.git
   cd waytermirror
   meson setup builddir
   meson compile -C builddir -j$(nproc)
   

   This produces waytermirror_server and waytermirror_client.

2. Run the server on your Wayland desktop:

   ./waytermirror_server
   

3. Run the client:

   • Terminal mode (default - works anywhere):

     ./waytermirror_client -H <host>
     

   • Native GUI window (best experience):

     ./waytermirror_client -H <host> -R gui
     

   • Framebuffer mode (TTY/console - requires root/video group):

     sudo ./waytermirror_client -H <host> -R framebuffer
     

   • KMS direct rendering (TTY/console - requires root/video group):

     sudo ./waytermirror_client -H <host> -R kms
     

Tip: Run ./waytermirror_client --help or ./waytermirror_server --help to see all available flags and defaults.

Build & install

Prerequisites

• Core: gcc/g++, make, wayland, wayland-protocols, libinput, pipewire, lz4, rapidjson, systemd, argparse, libsixel, libpng, opus, libudev, libdrm, gbm
• Video encoding: FFmpeg (libavcodec, libavutil, libswscale, libavformat) for H.265/HEVC encoding/decoding with hardware acceleration support
• Optional: NVIDIA CUDA toolkit for GPU rendering (nvcc) — see NVIDIA CUDA Toolkit

Arch Linux-based

sudo pacman -S base-devel git gcc wayland wayland-protocols libinput pipewire lz4 rapidjson systemd argparse libsixel libpng ffmpeg opus libdrm mesa meson
# Optional for CUDA:
sudo pacman -S cuda

Debian-based

sudo apt install build-essential git gcc libwayland-dev wayland-protocols libinput-dev libpipewire-0.3-dev liblz4-dev rapidjson-dev libsystemd-dev pkg-config meson libargparse-dev libsixel-dev libpng-dev libavcodec-dev libavutil-dev libswscale-dev libavformat-dev libopus-dev libudev-dev libdrm-dev libgbm-dev
# For CUDA: install NVIDIA CUDA toolkit from https://developer.nvidia.com/cuda-toolkit

Building

• CPU-only (default):

  meson setup builddir
  meson compile -C builddir -j$(nproc)
  

• CUDA-enabled (if you have nvcc and CUDA libs):

  meson setup builddir -Dnvidia_cuda=true
  meson compile -C builddir -j$(nproc)
  

• The provided PKGBUILD auto-detects nvcc. Override with WAYTERMIRROR_CUDA=1 or WAYTERMIRROR_NO_CUDA.

Artifacts

• waytermirror_server
• waytermirror_client

Packaging (Arch)

• AUR: Install directly from the waytermirror-git AUR package:

  yay -S waytermirror-git
  # or
  paru -S waytermirror-git
  

• Manual build: Use the included PKGBUILD with makepkg:

  mkdir -p waytermirror
  cd waytermirror
  wget https://github.com/cyber-wojtek/waytermirror/raw/refs/heads/main/PKGBUILD
  makepkg -si
  

Runtime requirements

• Audio: PipeWire (for system audio streaming and microphone capture).
• Input: Access to input devices (user must be in the input group or run with sufficient privileges to read /dev/input/*).
• GUI mode: Running Wayland compositor with xdg-shell support.
• Framebuffer/KMS modes: Root access or proper permissions for /dev/fb0 and /dev/dri/card*.
• Hardware video encoding (server): Supported encoders auto-detected at runtime:
  • NVIDIA: NVENC (most performant, lowest latency)
  • Intel: QuickSync Video or VAAPI
  • AMD: AMF (Windows/Linux) or VAAPI (Linux)
  • Apple: VideoToolbox (macOS)
  • Software fallback: libx265 (very slow, not recommended for real-time)
• Hardware video decoding (client): Supported decoders auto-detected at runtime:
  • NVIDIA: CUDA
  • Intel/AMD: VAAPI or QuickSync
  • Software fallback: multi-threaded libavcodec

Usage

Server

./waytermirror_server [options]

Full server options

For the authoritative list use ./waytermirror_server --help.

Capture backend notes:

• wlr: Uses wlr-screencopy protocol directly (compositor must support wlr-screencopy-unstable-v1)
• pipewire: Uses PipeWire + xdg-desktop-portal for screen capture (works on more compositors including GNOME/KDE)
  • Important: When using PipeWire backend, select screens in their logical index order (0, then 1, then 2, etc.) to match the output indices used by the client's -o option.
• auto: Automatically detects and prefers wlr-screencopy if available, falls back to PipeWire

Input backend notes:

• virtual: Uses Wayland virtual input protocols (zwlr_virtual_pointer_v1, zwp_virtual_keyboard_v1) — requires compositor support
• uinput: Uses Linux uinput (/dev/uinput) — works on any compositor but requires proper permissions
• auto: Automatically selects virtual protocols if available, falls back to uinput

Client

./waytermirror_client -H <server_ip> [options]

Full client options

For the authoritative list use ./waytermirror_client --help.

Connection

Video & rendering

Renderer descriptions:

• braille: Unicode braille patterns (2×4 pixels per cell, works in any terminal)
• blocks: Half-block characters (1×2 pixels per cell, better color support)
• ascii: ASCII art style rendering (1×1 pixel per cell)
• hybrid: Automatically selects braille or blocks per cell for optimal quality
• sixel: Sixel graphics protocol (pixel-perfect, requires sixel-compatible terminal)
• kitty: Kitty graphics protocol (pixel-perfect, requires kitty or compatible terminal)
• framebuffer: Direct Linux framebuffer (/dev/fb0) rendering (requires root, works on TTY)
• kms: Direct KMS/DRM rendering (requires root/DRM permissions, best TTY performance)
• gui: Native Wayland window (no terminal needed, resizable, best user experience)

Quality setting notes:

• For pixel renderers (sixel/kitty/framebuffer/kms/gui): Controls H.265 encoding quality
  • 0-40: Fast encoding, lower bitrate, acceptable quality
  • 50-70: Balanced quality/performance (recommended)
  • 80-100: Maximum quality, slower encoding, higher bitrate
• For Unicode renderers (braille/blocks/ascii/hybrid): Controls pattern search precision

Input (local client input capture / forwarding)

Audio

Zoom / viewport

Examples

Basic LAN streaming (terminal)

# Server (desktop)
./waytermirror_server -F 60

# Client (terminal)
./waytermirror_client -H 192.168.1.100 -F 60 -M true -R hybrid

Native GUI window (best experience for graphical applications)

./waytermirror_client -H 192.168.1.100 -R gui -Q 75
# Creates a native Wayland window, resizable, hardware-accelerated
# No terminal required, works like a native application

High quality sixel with hardware encoding

./waytermirror_client -H 192.168.1.100 -R sixel -Q 85 -M true
# Server auto-detects best hardware encoder (NVENC/QuickSync/VAAPI/AMF)

Framebuffer output (TTY/console)

sudo ./waytermirror_client -H 192.168.1.100 -R framebuffer -Q 70
# Mirrors to /dev/fb0, hardware-accelerated H.265 encoding

KMS direct rendering (best TTY performance)

sudo ./waytermirror_client -H 192.168.1.100 -R kms -Q 80
# Direct KMS/DRM rendering with hardware acceleration

Low bandwidth (Unicode with compression)

./waytermirror_client -H server.example.com -c -L 12 -F 15 -M 256 -d 30 -R braille

Follow focused window

./waytermirror_client -H 192.168.1.100 -o follow

Input-only (control remote desktop without video)

./waytermirror_client -H 192.168.1.100 -n -x

4K GUI window with audio

./waytermirror_client -H 192.168.1.100 -R gui -K 3840x2160 -Q 85 -a -b 128

Zoom mode with GUI

./waytermirror_client -H 192.168.1.100 -R gui -z -Z 3.0 -X 1920 -Y 1080

Keyboard shortcuts (client)

All client shortcuts use the Ctrl+Alt+Shift modifier prefix, so normal keys are forwarded to the remote session. Press Ctrl+Alt+Shift+H at any time to display the full shortcut list with current toggle states.

Session control

Input control

Zoom control

Rotation

Rendering

Output & FPS

Audio

Quick usage tips

• GUI mode: Ctrl+Alt+Shift+R cycles to GUI renderer, creating a native resizable Wayland window
• Zoom panning: When zoomed (Ctrl+Alt+Shift+Z), use arrow keys to pan the viewport
• Rotation: Use [ and ] to rotate in 5° steps, T/Y for 90° jumps, \ to reset
• Renderer cycling: Press R repeatedly to cycle through all 9 renderers
• Pixel modes (sixel/kitty/framebuffer/kms/gui): Use hardware-accelerated H.265 encoding automatically
• FPS adjustment: Use J to increase and K to decrease FPS by 5
• Audio compression: Press 5 and 6 to toggle audio and microphone Opus compression

Network ports

• (base port) — TCP — video frames
• (base port + 1) — TCP — input events
• (base port + 2) — TCP — system audio (server → client)
• (base port + 3) — TCP — configuration/control
• (base port + 4) — TCP — microphone (client → server)

Default base port is 9999 (see -P / --port).

Performance tuning

Hardware-accelerated pixel rendering (best visual quality)

Native GUI window (recommended)

-R gui -Q 70

• Creates native Wayland window (no terminal needed)
• Hardware-accelerated H.265 encoding/decoding
• Resizable, works like any native application
• Best user experience for desktop mirroring

Sixel mode

-R sixel -Q 70

• Pixel-perfect graphics in sixel-compatible terminals
• Hardware H.265 encoding

Kitty mode

-R kitty -Q 70

• Kitty graphics protocol with H.265 compression
• Requires Kitty or compatible terminal

Framebuffer mode

sudo -R framebuffer -Q 80

• Direct framebuffer rendering
• Works on physical TTYs
• Requires root access

KMS mode (best TTY performance)

sudo -R kms -Q 85

• Direct KMS/DRM rendering
• Zero-copy rendering path
• Best performance on TTY
• Requires root/DRM permissions

Quality ranges:

• 50-70: Balanced performance (recommended)
• 80-100: Maximum quality (higher bitrate)

Hardware encoder auto-detection priority:

1. NVENC (NVIDIA) — lowest latency, best performance
2. QuickSync (Intel) — excellent performance
3. VAAPI (Intel/AMD) — good performance
4. AMF (AMD) — good performance
5. Software (x265) — fallback, very slow (not recommended)

Unicode rendering (terminal-based)

Maximum quality

-r cuda -R braille -d 100 -Q 100 -M true

Smooth video

-R hybrid -d 30 -F 60

Low bandwidth

-c -L 9 -M 256 -F 15 -d 30

Low latency

-F 60 -Q 0 -d 50

Encoder-specific tuning

• NVENC (NVIDIA): Ultra-low latency, CBR mode, no B-frames
• QuickSync (Intel): Low delay mode, minimal look-ahead
• VAAPI (Intel/AMD): Quality-based encoding
• Software fallback: Not recommended for real-time (install hardware drivers)

Troubleshooting

"Failed to initialize libinput"

sudo usermod -aG input $USER
# Log out and back in

PipeWire issues (audio not working)

systemctl --user status pipewire
systemctl --user restart pipewire

CUDA errors / GPU verification

nvcc --version
nvidia-smi

H.265 encoding issues

• Check server logs for encoder selection messages
• NVENC not available: Ensure NVIDIA drivers are installed
• QuickSync not available: Install Intel media drivers (intel-media-driver on Arch)
• Software encoder very slow: This is expected — install hardware acceleration

Sixel/Kitty/GUI rendering artifacts

• Increase quality: -Q 80 or higher
• Check encoder logs for errors
• Try different renderer: -R gui for native window

GUI window not responding

• Ensure Wayland compositor is running
• Check that xdg-shell protocol is available
• Look for "GUI" messages in client logs
• Try another renderer: -R sixel or -R kms

GUI window won't resize

• This is a known issue being fixed
• Restart client to reset window size
• Use -K WxH to set specific resolution

Permissions to /dev/input

# Check group membership
groups $USER

# Add to input group if missing
sudo usermod -aG input $USER

Framebuffer/KMS permission denied

# Framebuffer
sudo chmod 666 /dev/fb0  # Temporary
# Or run with sudo

# KMS/DRM
sudo usermod -aG video $USER  # Permanent
# Log out and back in

Logs & debugging

• Server prints encoder selection: Look for "HEVC ENCODER" in logs
• Client prints decoder selection: Look for "HEVC DECODER" in logs
• Compositor detection: Check server startup messages
• Network issues: Verify firewall allows ports 9999-10003

Security & limitations

Security considerations

• No built-in authentication: Uses raw TCP streams
• Do not expose to untrusted networks: Use SSH tunnel or VPN for remote access
• Example SSH tunnel:

  ssh -L 9999:localhost:9999 -L 10000:localhost:10000 -L 10001:localhost:10001 \
      -L 10002:localhost:10002 -L 10003:localhost:10003 user@server
  # Then connect client to localhost
  

Limitations

• Input injection requires elevated privileges or input group membership
• Framebuffer/KMS modes require root access or proper permissions
• H.265 hardware encoding: Software fallback (libx265) is very slow
• GUI mode: Currently has some window resize handling issues (being fixed)
• PipeWire backend: Must manually select screens in correct order

Recommended setup

• Run on trusted local network or VPN
• Use hardware acceleration when possible
• Keep software up to date
• Monitor system resources during operation

Design notes & behavior

Rendering architecture

Unicode modes (braille/blocks/ascii/hybrid)

• Rendering performed server-side
• Client displays ANSI escape sequences
• Optional LZ4 compression
• CUDA acceleration available (server-side)

Pixel modes (sixel/kitty/framebuffer/kms/gui)

• Server: RGB24 → H.265 encoding (hardware-accelerated)
• Network: Compressed H.265 stream
• Client: H.265 decoding (hardware-accelerated) → display
• Much more efficient than LZ4 for pixel data

Hardware acceleration

Server (encoding)

• Automatic encoder detection prioritizes best available
• NVENC > QuickSync > VAAPI > AMF > software
• Logs show selected encoder at startup

Client (decoding)

• Automatic decoder detection: CUDA > VAAPI > QuickSync > VDPAU > software
• Multi-threaded software fallback when hardware unavailable
• Sequence number tracking prevents frame drops

Unicode rendering (optional)

• CUDA acceleration for pattern search (server-side)
• Only when built with CUDA=true
• Significantly faster braille/blocks rendering

Compression

Unicode modes

• Optional LZ4 compression of ANSI strings
• Levels 0-12 (0=fast LZ4, 1-12=HC)
• Toggle with Ctrl+Alt+Shift+U

Pixel modes

• H.265 video compression (much more efficient)
• Hardware-accelerated for real-time performance
• Quality controlled by -Q flag

Audio

• Optional Opus compression (both directions)
• Configurable bitrate/complexity
• Toggle with Ctrl+Alt+Shift+5 (playback) or 6 (microphone)

Hybrid renderer

• Chooses per-cell between braille and half-blocks
• Adaptive Unicode rendering for best quality
• CPU or CUDA accelerated

GUI renderer

• Native Wayland window using xdg-shell protocol
• Hardware-accelerated H.265 decoding
• Resizable window (some issues being fixed)
• Best user experience for desktop mirroring
• No terminal emulator required

Contributing

• Bug reports, feature requests, and PRs welcome
• Join discussions in Issues

License

MIT License — see LICENSE file for details.

---

Project Status: Active development