Project Name: Mouse Commander, Visual mouse control using a low resolution webcam
CS585
Advisor: Prof. Magrit Betke

Overview: Computers were initially designed to automate mundane tasks and as time went by it became obvious that they could do more than just that. Next came personal PCs and software such as Microsoft Windows that took computers out of the hands of large corporations into homes. The computer became personal. Soon, typewriters became obsolete as it was now possible to type on a PC and word processors followed shortly after. However despite all these advances there was little or no attention paid to the severely disabled community who unfortunately through no fault of their own were unable to use a standard desktop computer.

Computers were designed to be controlled by humans. By design they are interactive, starting it up required a push of the power button; using any of the software calls for ability to move and press down on a mouse. Therefore those who cannot perform any of the above tasks were precluded.

The camera mouse is a software that bridges the gap between the severely disabled and the computer. It works by allowing the user to control the computer within their limited ability by moving the part of their body that they can still control: their head. The user controls the mouse and mouse clicks by translating or rotating their head. In its current implementation, the software must be initialized (selection of feature to track for mouse control) by an individual manually thereby mandating that the severely disabled still have to rely on another person prior to using the software. Their independence was not feasible under this design.

The existing Camera mouse software though revolutionary is limited in its functionalities. It can only track facial features and utilizes dwell time for clicking. While this offers basic mouse control, precise control can be implemented by utilizing eye blinks to control the mouse. Furthermore, once the feature being tracked is lost the system has to be re-initialized. This limits the control the users have over the mouse since they have to wait for an able assistant to perform the re-initialization.

Goal:

The goal of this project is to give the severely disabled a modicum of independence by designing the system to self initialize. The aim is to do away with the selection of the feature to track by monitoring eye blinks. Mouse movement will be controlled by moving (translating) the head and mouse clicks performed by blinking of the eyes. The drawback is that the user must be able to control their head movement and eye blinks. Delibrate eye blinks will be necessary to successfully utilize the software.

For this project I will implement the following enhancements:

• As a step towards total independence between manual mouse control and facial mouse control, users will be able to control the mouse with their eyes. Moving the head moves the mouse, dwelling over an icon/menu for a certain amount of time (customizable) will highlight the item as a region of interest and deliberate eye blinks (as opposed to involuntary eye blinks) will be used to double click.
• Currently, once the tracked feature is lost the system has to be reinitialized. With this enhancement, users will no longer need an assistant or caregiver to reinitialize the camera mouse. Rather the system will re-initialize itself.

Deliverables:
There will be two major deliverables:

1. The Camera Mouse executable, including well-documented and clearly written C++ source code.
2. The project write-up including a user guide and technical documentation.

Development Plan:

• Research similar subjects on image processing involving normalized correlation, feature tracking, segmentation and skin detection.
• Development - Implement system initialization via eye blinks. Utilize existing feature tracking code to track head movement from one frame to the next. Detect tracking feature loss via a confidence factor. When the factor is low, the system automatically re-initializes the system via blink detection.
• Prototype the above code
• Testing
• In-class presentation
• Review and documentation.

Timeline

Process	Date Due
Research	March 23, 2008
Development/Prototype	April 15, 2008
Testing	April 25, 2008
Presentation	May 1, 2008