swish
Control your Mac with mid-air hand gestures — webcam only, on-device
- Stars
- 8
- Language
- Python
- License
- MIT
- Updated
Swish 🖐️
Control your Mac with mid-air hand gestures — no extra hardware, webcam only, fully on-device.
A hand shape picks the command group; the direction you swipe picks the action.
| Hand shape | Group | Swipe ← | Swipe → | Swipe ↑ |
|---|---|---|---|---|
| Open palm | Window tiling | snap left | snap right | maximize |
| Two fingers (V) | Spaces | prev desktop | next desktop | Mission Control |
| Fist | App switch | prev app | next app | — |
| Shaka (thumb+pinky) | Media/volume | vol down | vol up | play/pause |
Runs as a menu-bar app. Always-on, with strong anti-misfire safeguards and a global kill-switch hotkey (⌥⌘G).
Install
python3.11 -m venv .venv && source .venv/bin/activate
pip install -e .
On first run, Swish downloads the MediaPipe hand model (~7 MB) to ~/.cache/swish.
Permissions
Grant these in System Settings → Privacy & Security to the app running Swish (your terminal, or the packaged app):
- Camera — to see your hand.
- Accessibility — to move windows and send keystrokes.
Run
Tune and sanity-check gestures first with the live overlay (shows landmarks, the detected shape, and what fires):
swish --debug # press q to quit, p to pause
swish --debug --dry-run # overlay only, never touches the OS
Then run it as a background menu-bar app (the daily driver):
swish # 🖐️ appears in the menu bar; click to Pause/Quit
swish --dry-run # logs gestures without acting
How it works
Webcam → MediaPipe Hands → features (shape + motion) → recognizer → action
Always-on, with anti-misfire safeguards: high-confidence gate, a shape must be held
stable before motion counts, swipes must clear both distance and speed thresholds,
and a cooldown + hand-reset is required between gestures. Tune everything in
src/swish/config.py.
Develop
pip install -e '.[dev]'
pytest # 30 tests; the recognizer + pipeline are fully unit-tested
The pure core (features.py, gestures.py) has no camera/OS dependencies, so the
gesture logic — including the critical "don't fire on normal hand movement" cases —
is tested deterministically.
Stack
Python · MediaPipe Tasks (HandLandmarker) · OpenCV · pyobjc (Quartz + Accessibility) · rumps
Platform
macOS, Apple Silicon, built-in webcam.
License
MIT