AI & Data

Gesture lets you control a laptop with hand movements, using only its built-in camera

The free app for Mac and Windows turns eight simple gestures into clicks, scrolls and shortcuts, and processes video on the computer without uploading it.

A new desktop app called Gesture aims to give people another way to use their computers: by moving their hands in front of the camera already built into their laptop. Point a finger and the cursor follows; pinch and it clicks.

Gesture was developed by Jacob Diameh, co-founder and editor-in-chief of AfricTech News. It runs on macOS and Windows and is free to download.

How it works

The app uses Google's MediaPipe hand-tracking technology to find 21 points on the user's hand, including fingertips and joints. From the position and movement of those points it works out which gesture is being made, then passes the matching command to the computer as if it came from a mouse or keyboard. No extra hardware or special camera is needed.

Eight gestures

Users can adjust tracking sensitivity and change which action each gesture triggers.

Privacy first

Because the app watches through the camera, it is built to keep video on the device. Camera frames are processed in the computer's memory and thrown away straight after; they are not saved or sent anywhere. Gesture needs no account and no internet connection, and its settings and logs stay on the computer.

Who it's for

The app is aimed at presenters and teachers who want to move through slides away from the keyboard, as well as students, creative professionals and anyone who finds a mouse or trackpad awkward. It is not meant to replace them, but to add another option.

Gesture supports macOS 11 or later on Apple Silicon and Intel Macs, and Windows 10 and 11. It runs in the background, starts when you sign in and picks up again if another app has been using the camera. Under the hood it combines MediaPipe with a background service written in Go and C++ code for system input control.

Gesture is available at gesture.promintal.com.

Disclosure: Gesture was developed by Jacob Diameh, co-founder and editor-in-chief of AfricTech News.

Sources: Gesture