The Eye Mouse
Project Plan
Project:
sdmay324
Client:
Senior Design
Advisors:
Dr. J. Davidson
Dr. A. Dogandžić
Team Members:
Brian Figueroa
John Guy
James Loftus
Brian Reed
Jeff Tott
October 8, 2002
i
1. List of Figures ii
2. List of Tables ii
3. Introductory Material 1
3.1 Abstract 1
3.2 Definitions of Terms 1
4. Project Plan 2
4.1 Introduction 2
4.1.1 General Background 2
4.1.2 Technical Problem 2
4.1.3 Operating Environment 2
4.1.4 Intended Users and Uses 2
4.1.5 Assumptions and Limitations 3
4.2 Design Requirements 3
4.2.1 Design Objectives 3
4.2.2 Functional Requirements 4
4.2.3 Design Constraints 4
4.2.4 Measurable Milestones 5
4.3 End-Product Description 5
4.4 Approach and Design 5
4.4.1 Technical Approaches 5
4.4.2 Technical Design 6
4.4.3 Testing Description 6
4.4.4 Risks and Risk Management 7
4.4 Financial Budget 7
4.5 Personal Effort Budget 8
4.7 Project Schedule 9
5. Closure Material 10
5.1 Project Team Information 10
5.2 Summary 11
1. List of Figures
Figure 3.2: Electro-oculography setup
Figure 4.7: Gantt chart
2. List of Tables
Table 4.5: Financial Budget
Table 4.6: Personal Effort Budget
i
3. Introductory Material
3.1 Abstract
Use of the human eye as a input device for personal computing remains a largely untapped field. This design project shall attempt to explore this field further by developing a low-cost input device to be usable by an average PC user. The team shall attempt to combine electro-oculography (potential difference across the face) measurements and optical image processing to emulate a basic PC mouse.
An apparatus shall be designed and built to perform this tracking and emulate a standard PC mouse via serial, PS/2, or USB interface. This device will comprise electrodes attached to the subject’s face to measure potential difference changes along with a camera component to provide auxiliary data to a signal-processing box. Hopefully, such an apparatus will allow for alternative pointer control options for the PC.
3.2 Definitions of Terms
Electro-oculography: measurement of changes in electrical potential difference across a subjects face due to eye movement, measured via electrodes attached to compass points on the face (see figure below)
Figure 3.2: Electro-oculography setup
i
4. Project Plan
4.1 Introduction
4.1.1 General Background
This project is intended to develop a prototype computer “mouse” system that shall be completely controlled by the user’s eye movement. It shall be usable by both handicapped and non-handicapped people alike. The envisioned system shall allow a person to use applications such as Microsoft Word to perform the usual functions easily (possibly with the help of an included, small, on-screen keyboard to enter text into the computer). Four electrodes shall be placed on the user’s face to detect horizontal and vertical eye movements using electro-oculography. An imaging device will be used to detect if the user blinks or moves his or her head away from the screen. Blinking shall simulate a user “clicking” the mouse.
4.1.2 Technical Problem
Some specific technical challenges include:
· how to make the system accurate within one character
· how to implement auto-recalibration
· how to deal with head movements
· how to turn the cursor off when the user wishes to read text
· how to analyze and interpret the data optimally from the electrodes.
This design shall attempt to address these issues in the form of a cost-effective, reasonably easy to use apparatus consisting of electrical and optical transducers interfaced to PC-based driver software via A/D converter hardware.
4.1.3 Operating Environment
The imaging device shall be attached to the computer monitor. The product shall only be used in normal household conditions on a personal computer running Microsoft Windows.
4.1.4 Intended Users and Uses
Although the end product shall be useable by both handicapped and non-handicapped persons, guidance shall be required for some. The current design would not provide easy self-setup for paralyzed subjects. Severely disabled people may greatly benefit from this device, especially people suffering from so-called “locked-in” syndrome since they shall be able to communicate with their surroundings and control their environment. The requirement to keep the user’s head still (unfortunately) is fulfilled in this case and these severely disabled people shall have the greatest success with this product.
4.1.5 Assumptions and Limitations
The result the end-user shall have with the eye mouse depends on certain assumptions regarding the user:
· The user has normal, healthy eyes that can be detectable as any other eye by the imaging device, can focus on a point for any amount of time, can blink, and can move smoothly.
· If the user is paralyzed, they will have a person around to attach the electrodes to the user’s face.
· The user has a computer with a USB, PS/2, or serial port.
· The imaging device refreshes often enough to catch a blink the majority of the time from the user.
Success of the design may also be limited by the following factors:
· The imaging device does not have an infinite refresh rate and may not process some inputs.
· The mouse needs to be re-calibrated every time the user moves his or her head.
· A severely handicapped user will need assistance to use the device.
4.2 Design Requirements
4.2.1 Design Objectives
This design effort shall attempt to complete the following objectives:
· Emulation of PC mouse: The design team shall attempt to make usage of the “eye mouse” device emulate it’s conventional namesake as closely as possible, with some changes to accommodate special needs (such as auxiliary virtual keyboard, special pointer visibility behavior, and modified “clicking” via captured signals).
· Cost-effective hardware: Due to limited budget, all possible attempts shall be made to find the cheapest hardware which meets the needs of the design.
· Combine electro-oculography with image processing: Reliable and accurate tracking may be best achieved by combining the basic directional measurements possible via electro-oculography with auxiliary data gathering through image processing, such as facial position with respect to the PC screen and blinking.
· Intuitive / automatic calibration routines: The system will likely need calibration for each new user, and possibly may require periodic re-calibration during a single-user session. The design will attempt to make this process as painless as possible.
4.2.2 Functional Requirements
The following functionality shall be provided by the design:
· Movement of cursor via eye tracking alone: At the most basic level, this design shall provide the ability for a user to influence position of the PC mouse cursor using only eye movement. No other user input will be required to move the cursor.
· Clicking: End users shall have some mechanism to “click” the emulated PC mouse – that is, some user input such as blinking or pressing an auxiliary button, shall result in the same input to the PC as would result from clicking a standard mouse. Some distinction shall be made between single and double clicks, and in the case of blinking, normal (single) blinks must be ignored.
· Interface for calibration / configuration: Users shall be able to invoke a calibration routine as well as adjust sensitivity and click behavior to individual tastes. These tools shall be exposed for manual use and may be invoked automatically when the system initializes.
4.2.3 Design Constraints
The device must conform to the following constraints:
· Comfortable for long-term use: An input device may be considered fairly useless if it is not ergonomic enough to be comfortably usable, possible for long periods of time. Thus, the device must not be too heavy or irritating for general use.
· Suitable for office use: To be useful, it must be feasible to utilize this device in a fairly average office environment. For example, power should not require special considerations – it shall draw power from the PC itself if possible, or else use standard wall plug AC.
4.2.4 Measurable Milestones
· Capture electro-oculography data to PC (40%): The most basic task in this design project shall be to capture changes in electro-ocular potential differences and translate these signals into pointer movements onscreen.
· Add calibration routines (20%): For ideal operation, the apparatus will need both automatic and manual calibration abilities. This logically follows the basic data capturing abilities.
· Add image processing component (20%): Further accuracy and improved intuitive operation may be achieved through adding image-processing capabilities. This functionality may be considered auxiliary to the basic input.
· Polish software and hardware for release (20%): Before such a device may be useful for the general public, much testing and tweaking shall be done to ensure reliable and intuitive function.
4.3 End-Product Description
Many users of personal computers either need or desire alternative forms of input to accomplish computing tasks. The “eye mouse” tracking device will offer a viable alternative to traditional manual mice for manipulation of popular PC graphical user interfaces. The device shall utilize both direct measurement of electrical activity produced by movement of a user’s eyes as well as a machine vision camera system to track movements visually and shall interface to common “Wintel” PCs via USB, PS/2 or serial ports. Hardware shall consist of electrodes for direct measurements, a camera fixture, and a small external box containing signal processing and PC interface components.
4.4 Approach and Design
4.4.1 Technical Approaches
There are several approaches to eye tracking that can be considered in order to design the optimal eye mouse. One of the least expensive and simplest eye tracking technologies is recording from skin electrodes. The retina is electrically active compared to the rest of the eyeball, so there is a measurable potential difference between it and the cornea. Electrodes are placed on the skin around the eye socket, and can measure changes in the orientation of this potential difference. It can cover a wider range of movement than other tracking technologies, but gives poor accuracy. In combination with software and the optical approach below, we can heighten the accuracy of responses -- if the user looks left for an extended period of time, the cursor on the screen will travel far left. If the user just looks left for a second, the cursor will move one space.
Another approach that can be used to implement an eye mouse is based on a non-contacting camera that observes the eyeball plus image processing techniques to interpret the picture. Tracking one of its visible features can identify the position of the eyeball. For example, the boundary between the sclera (white portion of the front of the eye) and iris (colored portion) is easy to find, but only a portion of it usually remains visible at one time. The outline of the pupil against the iris is a good choice if implemented properly. This can be alleviated with sophisticated image processing and pattern recognition techniques or, more simply, by illuminating the eye with a light that is coaxial with the camera. We would need to stabilize the user’s eyes in order to eliminate any error caused by head movement.
In choosing the approach, either of the two described above can be chosen, or both may be used in parallel. The most important criteria in selecting our approach is find the method that is least costly to us, yet still offers a sufficient amount of accuracy in tracking eye movement. Also, the approach that is chosen should be able to be designed within the timeframe given.
4.4.2 Technical Design
In reporting the actual there are several areas that will need to be discussed:
· The actual eye tracking method used will is very important in understand the eye mouse system. The one chosen will have to be for reasons such as accuracy and cost.
· The software written for the system is also critical because this is what interprets the data captured by the hardware. What does each particular eye movement represent to the system? Which movements should be made in order to make it the most comfortable and compatible with the user’s intuitions? Should special, non-intuitive behavior be included, requiring user training?
· Calibrating the system is also a big factor in the design. When the user’s head moves, the system has to be able to detect that and either re-calibrate itself to continue working, or it should be able to process that information and just keep on functioning. In either case, the one less obtrusive to the user should be used.
4.4.3 Testing Description
Several aspects of this design will require testing considerations. First, the software written must be tested to ensure that there are no bugs. If the code is rigorously tested before it is integrated with the eye tracking device, errors found in the cursor movement can be narrowed down to the source. Once the software and eye tracking device is integrated, more thorough testing must be made. Testing in different eye movements must be made to see how it affects the cursor on the screen. If the user is looking left, does the cursor move left as well? Testing on head movement must also be made. If the user’s head moves, can the eye mouse system re-calibrate itself correctly so that the user isn’t hassled with waiting for the device to respond normally.
4.4.4 Risks and Risk Management
In any project, there are a number of risks that can be encountered. Some risks that we may encounter through this project include:
· As this project is going to take two semesters, there is a definite risk of losing a team member. In order to maintain all of the information gained by all members, each member must record any data or thoughts relating to the project. If that team member leaves, all the information that team member had about the project can then be retained by the other members.
· The software that is to be written for the project or any documentation relating to the project can be erased, deleted, or lost. In order to save ourselves from the hassle of this risk, we can back up all material so that there are multiple copies of everything. So, in case some time is lost, there will always be a redundant back up system in place.
· Any hardware or other eye tracking devices used for this project can be stolen or destroyed. To stay clear from this risk, we must ensure that all materials are properly and safely kept so that only the team members have access to it. In handling the hardware, we must ensure to be very careful, especially in light of our lack of budget for replacements.