text-swipe

Demo site: paste text on one device and have it relayed to another device over WebSocket. Built for testing the gesture-driven "swipe to send" flow (the gesture-recognition module lives in src/text_swipe/swipe_detecting/ and the face-identification module lives in src/face_recognition/face_detecting/; neither is part of this demo).

Structure

text-swipe/
├── run.sh                  # one command: launches backend (:8000) + frontend (:5173)
├── backend/                # FastAPI WebSocket server (2-device limit, relay)
├── frontend/               # React + Vite + TypeScript UI
└── src/
    ├── text_swipe/         # Python package (gesture module: swipe_detecting/)
    └── face_recognition/   # Python package (face-identification module: face_detecting/)

Run

./run.sh          # or: bash run.sh

This starts both servers:

• backend — FastAPI WebSocket on http://0.0.0.0:8000 (endpoint /ws)
• frontend — Vite dev server on http://0.0.0.0:5173
• socket relay — JSON-lines TCP relay on 0.0.0.0:8765

Open http://localhost:5173. From a phone on the same network, open http://<this-computer's-ip>:5173.

To run the camera gesture detector in a second terminal:

cp .env.example .env
uv run python -m text_swipe.main

Configure the local .env to select the WebSocket role/address (IS_SERVER, HOST, PORT), V4L2 camera node and capture options (CAMERA_DEVICE, CAMERA_FOURCC, CAMERA_WIDTH, CAMERA_HEIGHT, CAMERA_FPS, CAMERA_BUFFER_SIZE), and preview-window name and initial size (WINDOW_NAME, WINDOW_WIDTH, WINDOW_HEIGHT). The real .env is ignored by Git; use .env.example as the documented template.

Use a device path rather than an OpenCV numeric index: UVC cameras can expose several /dev/video* nodes, while an OpenCV index does not reliably identify the capture node. Find the usable node with v4l2-ctl --list-devices. For the external Web Camera in this setup, use CAMERA_DEVICE=/dev/video2. CAMERA_FOURCC=MJPG allows its 1280×720 and 1920×1080 modes to run at 30 FPS; the uncompressed YUYV Full HD mode is limited to 5 FPS. The detector prints the format, resolution, and FPS actually selected by the camera after startup; set CAMERA_FOURCC=YUYV only if your camera does not support MJPG.

The preview window uses a resizable OpenCV window with preserved frame ratio. Under Hyprland it runs through XWayland (QT_QPA_PLATFORM=xcb) and takes the size assigned by the compositor.

When the detector recognizes a completed swipe (unclenching), it sends a {"type":"swipe", "timestamp": ...} event to the TCP relay. The detector connects to 127.0.0.1:8765 by default; set TEXT_SWIPE_SOCKET_HOST and TEXT_SWIPE_SOCKET_PORT to use a relay on another computer.

A second computer can subscribe to the relay with:

nc <relay-computer-ip> 8765

Each event is one JSON object per line. The relay accepts multiple local-network clients and broadcasts each swipe event to all of them.

(The Vite output prints the network URL.)

http://localhost:5173 is still the local URL; other devices use the host computer's IP address.

On first run run.sh installs frontend deps (npm install) automatically. Python deps are managed with uv (uv sync).

Face recognition demo

uv run python -m face_recognition.face_detecting.demo

Run it with -m — invoking the file by path does not put the package on sys.path. On first run the buffalo_s model is downloaded to ~/.insightface/models/.

• e — enroll the currently detected face (type a name in the terminal)
• f — forget an enrolled face by name
• c — switch to the next camera (shown only when more than one is found)
• q — quit

Enrolled faces are saved to faces.npz next to the module (path set by FACES_PATH in the config) and loaded on the next run. The file is gitignored since it holds personal biometric data. Embeddings from different model packs are not comparable, so the model name is stored alongside them and a file written by another model is ignored rather than silently mismatched.

Cameras are discovered by scanning /dev/video* for nodes that actually deliver frames — a UVC webcam exposes several nodes and OpenCV's integer indices do not line up with them. The active camera's name is drawn in the bottom-left corner.

Frames are grabbed on a background thread: V4L2 queues frames, so reading straight from cv2.VideoCapture in a slow loop returns progressively older ones (over 30 frames of backlog build up after a few seconds). Cameras report different native resolutions (1280x720 and 1920x1080 here) and ignore requests to change them, so frames are scaled to a fixed width — otherwise the window resized on every camera switch.

Settings live in src/face_recognition/face_detecting/config.py: model pack, detector input size, match threshold, frame width, window size and thread count. Both onnxruntime and OpenCV default to spreading across every core and then contend with each other — capping them at two threads cut usage from 7.5 cores to 2.6 and made detection roughly twice as fast.

The window is created with WINDOW_NORMAL, so it can be resized; calling imshow without namedWindow produces a fixed-size WINDOW_AUTOSIZE window instead. Under a tiling compositor the window takes whatever size the compositor assigns and the frame is scaled to fit.

Enrolled faces get a green box with the name and match score; unrecognized faces get a red unknown box. Enrollments live in memory only and are lost on exit. The demo forces the xcb Qt platform because the opencv-python wheel ships no Wayland plugin.

How it works

• On page load the browser connects to ws://<host>:8000/ws.
• The server accepts at most 2 devices. A 3rd connection is told "room full" and closed.
• Connected devices appear in a vertical list (yours is highlighted).
• Text typed in the input is sent over the WebSocket and relayed to the other connected device.

The text-transfer here is a simple relay scaffold. The actual trigger (a hand swipe) will come from the gesture module and is out of scope for this demo.