A virtual board for writing, drawing, picking colors and erasing with hand gestures in front of a camera, built in Python with MediaPipe hand tracking and OpenCV.
A project to develop an interactive virtual board that lets you write, draw, erase, and pick colors with hand movements in front of a camera, without touching a screen or using a stylus, relying on computer vision and hand tracking to turn gestures into commands that appear directly on the virtual board. A team of students carried out the project with technical assistance from Techno Enjaz.
The prototype was built in Python using MediaPipe to track finger positions, OpenCV for video processing, and NumPy for data processing, within an Anaconda environment. Real use cases were tried, such as drawing, changing color, pausing interaction, and partial or full erasing, and drawing updates on screen were close to real time in the test environment, with no published numerical measurements of accuracy or response time.
| Item | Details |
|---|---|
| Project type | Interactive gesture-control prototype |
| Field | Educational technology and computer vision |
| Project status | Prototype developed and functionally tested |
| Techno Enjaz role | Technical support and assistance to the students during project development |
| Core technologies | Python, MediaPipe, OpenCV, NumPy, Anaconda, VS Code |
| Outputs | Digital board for gesture-based writing and drawing, color palette, partial and full erase, pause state |
The traditional blackboard is still the main tool in many classrooms. It forces the teacher to turn their back to the students while writing, which reduces eye contact and makes it harder to follow their reactions, and students in the back rows sometimes struggle to read what is written. The project proposes a digital board that the teacher controls with hand movements in front of a camera, writing, changing colors, and erasing without touching any surface while still facing the students.
The project suggests the idea could extend beyond classrooms to training centers, virtual meetings, and presentations.
The project was carried out academically by a team of students, with technical assistance and support from Techno Enjaz during development. It is presented among the office's work as an example of its support for prototypes that combine hand tracking, video processing, and interactive interfaces.
The prototype was designed with two interfaces merged into one: an interface that captures and tracks hand motion through the camera, and a black board layer containing interaction tools such as the color palette and erasing. This merge shows the effect of each gesture directly in the drawing area while the video is being processed.
| State | How it was performed in testing |
|---|---|
| Drawing and writing | Moving the index and middle fingers held close together, turning the motion into digital strokes on the board |
| Color selection | Choosing a color from the palette (red was selected in the test) and continuing to draw with it |
| Pausing interaction | Opening the hand fully, so the system performs no drawing or erasing command, limiting unintended interactions |
| Partial erase | A dedicated erase gesture removes a selected part of the writing or drawing |
| Full erase | A Clear button on the board removes all content to start over |
| Tool | Role |
|---|---|
| Python | Main programming language |
| Anaconda | Creating an isolated environment for the project and reducing library compatibility issues |
| VS Code | Code editor |
| MediaPipe | Hand tracking and finger position detection |
| OpenCV | Video capture and image and frame processing |
| NumPy | Data-processing operations |
This architecture handles the camera's video stream, analyzes hand motion, and then updates the board interface according to the detected gesture.
Testing revealed several practical challenges related to real-time motion tracking:
These challenges were an important part of the experiment, because they showed the factors to consider when moving from an experimental prototype to a more stable application for everyday use.
The experiment showed that hand-motion tracking can be integrated with a digital writing and drawing interface and that the prototype's core functions work. In the test environment, drawing and writing updates on screen were close to real time, while performance remained dependent on lighting, device specifications, and the speed of movement in front of the camera.
The project does not publish numerical measurements of accuracy, response time, or the number of users tested, so this page focuses on the functions that were actually implemented and observed rather than using undocumented performance figures.
The project set out several recommendations that could be explored in later versions:
Broader future ideas were also proposed, such as integrating augmented and virtual reality to display 3D models, voice commands such as "clear the screen" and "change color", a phone and tablet app that works offline, and support for people with disabilities and sign language, but these remain ideas for future development, not functions implemented in the current version.
If you are working on a prototype based on computer vision or gesture interaction and need technical help developing the idea or building a prototype, you can contact the Techno Enjaz office to discuss the project's requirements and the appropriate scope of support.
The board lets you write, draw, erase, and pick colors with hand movements in front of the camera, without touching a screen or using a stylus. It was designed mainly so a teacher can write and control content while facing the students, and was built by a team of students with technical assistance from Techno Enjaz.
The prototype uses Python as the main language, with Anaconda for environment management and VS Code as the editor. MediaPipe tracks the hand and finger positions, OpenCV handles image and video processing, and NumPy is used in data-processing steps.
You draw by moving the index and middle fingers held close together, pick a color from the palette, erase part of the writing with a dedicated gesture, and clear the whole board with a Clear button. When the hand is fully open, the system pauses and executes no drawing or erasing command, limiting unintended interactions.
Tracking accuracy dropped with fast or complex finger movements, and performance suffered under very low or very bright lighting. Other people, moving objects, or reflections in the camera's view caused noise, and a slight delay appeared on lower-spec devices.
No. The project documents functional results: drawing, color selection, pausing, and erasing worked, with near-real-time drawing updates in the test environment. It reports no numerical accuracy or response-time measurements, and voice control, augmented reality, and a phone app are future ideas.
Reach out to Techno Enjaz engineering team to discuss technical feasibility, prototype hardware, and development scope.
Discover more engineering prototypes and academic implementations
A prototype that detects people with YOLOv8 and tracks them with DeepSORT in defined work areas, counts presence time in seconds, and exports records to Excel.
An exam-monitoring prototype using YOLO, MediaPipe and OpenCV to analyze a student's head direction, flagging head turns in red for review and forward gaze in green.
A face-recognition access control prototype using OpenCV and face-recognition that logs authorized entries, saves images of unknown faces, and sounds an alarm.
