Eye Tracking via Webcam
The mouse has undoubtedly become an integral part of personal computing as we know it. Be it a laptop or a desktop, it seems no workstation is complete without a keyboard and mouse. But just how efficient is it to use a mouse to control a cursor? What if you could control your mouse cursor with just your eyes? You would never have to take your right hand (or left) off the keyboard! It’s a simple idea, and it makes sense.
When moving the cursor, I’ve noticed that people usually do one of two things:
- They follow the cursor with their eyes until it reaches the intended destination (where they want to click), or
- They look at the intended destination (where they want to click) and wait for their hand to get the cursor there.
I’d say #2 is more common among frequent computer users, but either way, there is a delay between deciding what to click and actually getting the cursor there to click it. The summation of this lag time certainly adds up. I’d have to do a little more thinking to come up with a rough estimate of just how much time could be saved, but this isn’t my motivation. My motivation is the fact that this is just a really cool and inexpensive-to-test idea.
The Idea
So anyway, the idea is to be able to determine where exactly on the screen the user is looking just by examining the position of the eyes (as seen from a webcam). But what about clicks? Well, click events could be triggered by many things; maybe a quick head nod, an emphatic blink, or simply a dedicated key on the keyboard. There are many possible solutions for triggering left/right/middle mouse clicks. I’ll just have to try out various methods and see which ones work the best.
Source: Unknown
A simple Google search will show that extensive research has already been done on the topic of eye tracking. However, it seems that modern approaches require the user to wear some sort of head gear to track eye movement. This can be costly, inconvenient, and likely requires a less-than-trivial amount of calibration –The exact opposite of the convenience that humans naturally seek.
This idea may be better suited for laptops than desktops, because majority of laptops today have built in webcams. Webcams that can’t move relative to the screen. This means extrinsic camera calibration would only need to be performed once per laptop. However, with standalone webcams (commonly used with desktop computers) it may be necessary to recalibrate every time the webcam is moved -or even bumped for that matter.
Applications
Aside from the convenience of being able to move a cursor with just your eyes, this technology has potential for many applications. One of which is analyzing what us humans spend the most time looking at when shown pictures, videos, or text. A heat map can be generated to represent areas of an image (or a website) that someone is most interested in. This has been done in many variations in the past, but providing a way for this analysis to be performed by consenting users in the comfort of their own homes (without fancy headgear) could certainly lead to an expansion in this field of research.
Conclusion
I still need to do more research on the topic. I may find that the required precision/accuracy needed for effectively controlling a mouse cursor with eye-movement alone is beyond the reach of current webcams.
Interesting Notes
- I may also be able to determine how close a user is sitting to the computer monitor (within a reasonable degree of error). This can be calculated as the height of the triangle formed by two eyes and a point of interest on screen. Assuming, of course, that the user is focusing on point that lies on the plane defined by the computer screen.
I have two thoughts.
First, many webcams are capable of panning and zooming. Software that keeps the camera trained on the user’s eyes could increase significantly increase tracking precision.
Second, when you offered the “two ways people move the cursor”, I’m concerned you forgot a significant one.
While I typically look at the position where I want the mouse to go when controlling a mouse, the cursor sometimes moves independent of my eyes.
For example, if I grab the scroll bar on the side of a internet browser window and use it to scroll the page, I’m typically reading the page…not looking at the scroll bar.
The arrow keys can replace the functionality of the scroll bar in some cases, but there are bound to be situations where the eyes and the mouse disagree and there are no obvious alternatives.
This could present a hurdle.
Have you heard of Tobii? I am unsure if you mean to do something else, but they already make software that can make a heat map from your web browsing. They also don’t use a head-mounted apparatus. They have camera-bars that attach to a monitor and track your eyes from there. Here is an interesting topic that I found for a class I am taking: http://research.microsoft.com/en-us/um/people/cutrell/sigir2010-buscher-goodbadrandom.pdf
I had not heard of Tobii. I wrote about this idea without really researching first. The main thing I’m interested in is producing high-resolution eye to screen mappings. I haven’t given the paper a good read yet, but in Section 3.2 it says they were able to get 0.5 degree accuracy at 50Hz. Assuming I’m sitting 2 feet away from my monitor, that translates to a circular area on the screen of radius 24inches*cos(0.5deg) = ~0.2inches. Roughly 0.1 inch radius increase per 1 foot increase in distance from the monitor. Good to know, thanks for the info. (So I guess for now all I need to do is find a large, low dpi monitor. Ha!)
I’m curious though, what class are you taking?