veil

"Your apps, your terminal."

Terminal GUI renderer. Capture Wayland applications and encode them as ASCII art with luma-based character selection, edge detection, and AT-SPI text overlay. Runs natively on Niri.

Overview

Veil renders graphical Wayland applications inside your terminal by:

1. Launching GUI apps under Niri (Wayland compositor)
2. Detecting window position via Niri IPC (niri msg --json windows)
3. Capturing frames with grim (Wayland screenshot utility)
4. Converting to luma-based ASCII with hysteresis and edge detection
5. Overlaying actual text from accessibility APIs (AT-SPI)

Result: Your desktop GUI apps render as live ASCII art in the terminal. Works with any GTK4/Qt/native Wayland application.

Quick Start

Launch a TUI app:

veil run my-terminal-app

Launch a GUI app (renders to terminal):

veil run-gui nautilus      # File manager
veil run-gui zen-browser   # Web browser

Exit: Ctrl+C

Check terminal capabilities:

veil probe

Install

Requirements

• Niri — Wayland compositor (required for window detection)
• grim — Wayland screenshot utility
• python3 — For AT-SPI text overlay
• Rust 1.70+ (to build)

Build from source

git clone https://github.com/viewerofall/veil
cd veil
cargo build --release
./target/release/veil run-gui nautilus

Install globally

cargo install --path .
# or with git directly:
cargo install --git https://github.com/viewerofall/veil

Usage

Basic Commands

Command	Description
veil run <app>	Launch a TUI/terminal application
veil run-gui <app>	Launch a GUI application (renders to terminal)
veil probe	Show terminal capabilities and active config

Overrides

Control rendering parameters on the fly:

veil run-gui --override fps=15,quality=ascii zen-browser

Parameter	Description	Default
fps	Target frames per second (render loop)	30
quality	Rendering quality (ascii_luma, ascii_edge)	ascii_luma

Architecture

Crate Structure

veil/
├── veil-cli/          CLI interface, render loops
├── veil-compositor/   Window detection, capture pipeline
├── veil-render/       Character encoding (luma, edge detection)
├── veil-config/       Lua configuration
└── veil-capture/      Zig library (future GPU capture)

Rendering Pipeline

For GUI apps (run-gui):

Niri window detection (IPC)
    ↓ Extract window geometry & PID
    ↓ grim fullscreen capture (PNG)
    ↓ Decode to RGBA
    ↓ Compute luma (luminance) per cell
    ↓ Apply hysteresis (noise rejection)
    ↓ Edge detection (neighboring cell luma deltas)
    ↓ Map luma to ASCII characters (sparse to dense)
    ↓ AT-SPI text overlay (actual widget text)
    ↓ Dirty-row optimization (only redraw changed rows)
    ↓ Terminal output

Components

veil-cli

Command-line interface and main render loops. Two paths:

• run: TermCompositor — connects to existing TTY, manages PTY for TUI apps
• run-gui: GuiCompositor — Niri IPC detection, grim capture, luma pipeline

veil-compositor

Window detection and frame capture:

• gui.rs — Niri window detection, grim capture, luma computation
• capture_shm.rs — Read from /dev/shm (future LD_PRELOAD path)
• wayland_screencopy.rs — Wayland screencopy framework (May 1st GPU work)

veil-render

Terminal encoding:

• Luma-to-character mapping (70-character ramp)
• Edge detection (Sobel-style neighboring cell deltas)
• Hysteresis (18-unit threshold for character change)
• Dirty-row optimization (only output changed rows)
• AT-SPI text overlay (accessibility tree integration)

veil-capture

Zig library for future GPU rendering:

• shm_out.zig — /dev/shm output module
• intercept.zig — Wayland wl_shm pool interception
• egl_intercept.zig — EGL GPU capture skeleton (future)

Roadmap

Current (April 2026)

• ✅ Basic rendering (luma-based ASCII)
• ✅ Window detection via Niri IPC
• ✅ grim fullscreen capture
• ✅ AT-SPI text overlay
• ✅ Dirty-row optimization
• ✅ App lifecycle management (exit detection)

May 1st (GPU Rendering Path)

• GPU capture via wl_shm interception
• EGL glReadPixels support
• Proper window region extraction (Niri tile coordinate mapping)
• Multi-output support

Mid-May (Beta)

• Quality ladder (kitty, sixel, ASCII fallback)
• Performance tuning for weak hardware
• Configuration Lua refinement

Mid-June (v1)

• Smithay rewrite (full compositor)
• X11 support
• TTY boot support
• AUR/package distribution

Known Limitations

Current

• Niri-only: Window detection uses Niri IPC. Other compositors not yet supported (planned for v1 with Smithay rewrite).
• Fullscreen capture: grim captures entire screen including desktop background. Window region visible behind app on Niri.
• No window positioning: App fills entire terminal; can't move it within the render area.
• CPU-bound: Luma computation and encoding run on CPU. GPU path (May 1st) will fix this.

Why These Exist

Niri coordinates: Niri is a tiling compositor. Windows don't have fixed screen coordinates — they exist in a layout tree. Full Wayland coordinate calculation requires knowing tile grid dimensions, output scaling, and workspace offsets. Fullscreen capture + downscaling is pragmatic for now; proper solution requires Wayland screencopy protocol (in progress for May 1st).

GPU rendering: Rendering 1920×1080 to 80×24 luma computation on every frame is expensive on weak hardware. GPU path will push capture and initial downscaling to VRAM, freeing CPU for encoding.

Workarounds

• Close other windows to minimize background noise in render
• Use a dark, solid desktop background (#0a0010 recommended)
• Lower FPS if CPU usage is high: --override fps=15

Performance Notes

CPU Budget

• Capture: grim fullscreen (100–300ms per frame, system-dependent)
• Luma computation: 1920×1080 → 80×24 cells (1–5ms)
• Encoding: luma-to-chars, edge detection, overlay (1–3ms)
• Terminal I/O: dirty-row write (1–10ms)

Default 30 FPS target assumes ~33ms budget per frame. Adjust --override fps=N for your hardware.

Memory Usage

• Frame buffers: ~2–3 MB (RGBA + luma + chars)
• AT-SPI tree: varies by app (typically <1 MB)
• Zig/Rust runtime: ~5–10 MB

Minimal footprint suitable for embedded systems and remote SSH sessions.

Contributing

GitHub: viewerofall/veil

Areas for contribution:

• GPU rendering path (wl_shm interception, EGL hooks)
• Additional compositor support (Sway, Hyprland)
• Quality improvements (better character mapping, color support)
• Testing on weak hardware (RPi, older systems)

License

MIT

Created by viewerofall. GitHub: viewerofall/veil