How to Operate a Computer without Using Your Hands

Introduction

My story in short: after years of computing (mousing and typing), playing guitar and general hand abuse, I developed severe repetitive strain in both my forearms. This debilitated me to such a degree that I couldn't type or use a mouse for any meaningful period. As a result I had to investigate ways to interact with the computer that did not require hands. The primary solutions I discovered were:

Voice Dictation Software

Most people have an understanding of the basics of voice dictation software. You speak (into a microphone) and the words you say appear on the screen. You can also use your voice to issue commands that open up particular files or programs, or even to click a particular point on the screen.

There are other voice dictation software tools that can be found on the web, though I know little about them. Both the current versions of Windows and the Mac OS have some kind of built-in voice dictation functionality which I presume to be underwhelming, though I'm open to being convinced otherwise. But, even a cursory examination of the available products will make clear that Dragon NaturallySpeaking is your best bet.

I use NaturallySpeaking for three types of hands-free functions: typing, running applications, and navigating the file system and the Web. I provide some detail on each of these below.

Hands-Free Typing

This is really what voice dictation was designed for. I do most of my writing in Notepad and Dragon does a pretty good job of translating what I say to words on the screen. When you first install the software, you go through a training phase where it acclimates to your voice. I find that even if you skip the training (not that I recommend it), you get pretty good recognition right out of the gate. So, if I say "I would like to order a Thai iced tea," that's what shows up on the screen.

You can also use voice commands to navigate through your text. If I want to insert a word 10 words back, I will say "move back 10 words" and the cursor will do so. If I want to move up six lines, I say "move up six." If I want to move down five paragraphs, I say "move down five paragraphs." I also use a lot of the standard search functionality to navigate through documents. If I know the word "Aardvark" is near a section I want to edit, I will launch the Find tool by speaking the keyboard shortcut and then run the search, as in "Press Ctrl F, Aardvark, Click Find, Next." (More details on the "Press" and "Click" commands below.)

The program doesn't always flawlessly transcribe what I dictate. When I spoke the word "Aardvark" above, it translated it as "artwork." Nonetheless, accuracy is very high and when Dragon makes a mistake, you have the opportunity to train it to better understand what you're saying. You can also create custom commands for frequently used text. For instance, you can, create a command called "My Address" and every time you use it it will print out something like...

3452 Smith street
#123
Nowheresville, Virginia
12345-9876

At this point in my usage, I believe that voice dictation "typing" is actually superior to regular typing. (This varies by individual, of course. If you were an incredible typist before starting to use voice dictation, you may never get up to your original speed. I was always a pretty crappy typist.) One obvious advantage is that as long as Dragon "hears" the right word, it's going to spell it correctly. (Consequently, my spelling has become much worse.) I also find that the process of writing with voice dictation forces me to think through what I'm saying in such a way that my writing is clearer and more precise.

Hands-Free Application Use

If using computers was only about producing text, voice dictation would make the process blissfully simple. Of course, there's more to it than that. Some of our time with computers is spent opening up applications like Word, Excel, Firefox, Photoshop etc. For the most part, we don't use typed text to perform these operations, though one can use keyboard shortcuts. Once the programs are open, we spend time trolling through pulldown menus and submenus and clicking various icons. How does one use voice dictation in these scenarios? Well, it ain't always easy. It is true that anything you can do with the keyboard, you can do with voice dictation. If you're not a "power user" you may be unaware of how much of the functionality of a computer program can be accessed using keyboard shortcuts like "CTRL-P" (to print) or "ALT-F"* (to open the File menu in many Windows programs). Dragon has a "Press" command that can be used to access this functionality. So, to print a webpage I'm looking at, I say "Press Control P" and it launches the print dialog. When I first started using voice dictation, I printed out all the keyboard shortcuts for a program I use quite often, the web design tool Dreamweaver, and taped them to my computer screen. It wasn't always pleasant, but I was able to do quite a bit of web work with no hands.

You can, of course, also use the Tab, Enter, Shift, Delete, Backspace, up and down arrow keys and any similar keys with the "Press" command.

Many programs, however, have functionality accessible only by clicking a button or icon, not by using a keyboard shortcut. This is shoddy software interface design, and any programmers related to the product should be flailed, but it's an imperfect world. For these situations, Dragon has a "Click" command. This command has the effect of a mouse click on whatever icon or button you name. For example, if I'm looking at the main screen in Outlook and have an e-mail selected and then say "Click Reply," a new window with a "reply" e-mail pops up. After I dictate my reply, I can then say "Click Send" which has the effect of clicking the Send button on the e-mail toolbar.

There are a couple problems with the "Click" command. It tends to work reliably in standard Windows applications like Word or Outlook, less so in third-party apps like Photoshop or Dreamweaver. Secondly, some applications will have buttons or icons that don't actually have a name and you can't say "click the funny icon in the left-hand corner that has a ball surrounded with what appears to be winged lizards with halos."

What do you do when the "Click" command has failed you? At this point we turn to the mouse grid. Before describing how it works, I will refer you to this link which has a quick animation demonstrating the functionality. Basically, saying "mouse grid" overlays a 3 x 3 grid onto the screen. Each grid square has a number, and saying that number redraws a smaller 3 x 3 grid in just that square. You can repeat this process an infinite number of times, making the square smaller and smaller and focusing in on your target. It sounds pretty inefficient, but in reality isn't that bad. You quickly learned to mentally map out the screen to specific grid of coordinates so you can look at a point and say "mouse grid 348" and your mouse will move to that point. Having said that, there's no way you could play a video game or use a mouse intensive application like Photoshop with this method.

Hands-Free Navigation

I use the term "navigation" to refer to the process of finding and opening a file or application on your computer, or navigating the Internet in a browser. In this area, Dragon is certainly not as efficient as using your hands, but it's functional. Being that we can do most of this navigation using keyboard shortcuts, it's easy enough to apply voice dictation. It's also easy enough to interface with the operating system to navigate. If you were using your hands to open a specific file you would probably go to the start menu, go to My Computer, drill down through the file structure to find the folder with your file, open your file. This process can be translated to speech as "click start, my computer, local disk C, press enter, Dell (presuming you have a folder called "Dell"), press enter, service guide.pdf, press enter." (The end result of this being the opening of a likely unhelpful Dell service guide.)

Let's say you wanted to open up Internet Explorer and go to a quality website like acidlogic.com. The full speech command would be, "click start, Internet (wait for Internet Explorer to launch), go to address (this is a specific Dragon command which puts focus on the address bar of the browser), acid no space (to run words together you have to use the "no space" command) logic.com, press enter.)

Of course, some intelligent organization of your computer (for example, putting shortcuts to often used files on your desktop) can cut down your navigation time tremendously.

Downsides

Microphones

One can't underestimate the value of a good microphone when using Dragon (or any voice dictation software.) I use the KnowBrainer Hands-Free Headset Microphone which is of good quality and is quite affordable ($50.)

Add-ons

It's worth noting that there are a number of third-party add-on tools that can be used in conjunction with Dragon NaturallySpeaking. The main advantage of these tools is that they provide additional commands not available with out-of-the-box Dragon. Probably the best known is the KnowBrainer product. I have to confess that I haven't purchased it myself, but it looks very interesting and promising. I will also say that the KnowBrainer speech recognition forums are fantastic --- possibly the best Internet forum I've ever used for any subject. I've gone in there with several baffling problems, and all of them have been courteously and graciously solved by forum members.

There's also a tool called Vocola which appears to be something of an add-on language framework for Dragon. Haven't tried it myself, but it is positively reviewed. (Update: October 9, 2010. I have installed Vocola and have been playing with it. Very useful for automating a lot of recurring navigation tasks. I definitely recommend it. Installing vocola involves installing several different programs; there's a downloadable package at the ergoarchitect site that walks you through the full install.)

Voice Strain

One final note about voice dictation. Just as you can overuse your hands while mousing and typing, you can overuse your voice while dictating. I haven't had any problems but some people have, including this fellow. It's something to keep in mind.

I don't entirely agree with the design of the product which requires that you lift your leg up slightly to move the mouse on the pad. I think a better design would be to essentially take a rollerball mouse, flip it up side down, then attach a slipper to that, and your leg could roll the slipper around without having to fight the effects of gravity.

The FootTime mouse itself is a decent enough product. It connects by USB port to the computer. I've had a couple of the lasers in the mouse slipper die, but I was living in Los Angeles where the product is manufactured, so I just drove over to the company office, and the main guy (whose name I've forgotten) gave me a new one.

Downsides

Obviously the idea of a foot mouse has certain flaws. Your foot just doesn't have the fine motor control of your fingers, so you can't do the kind of delicate mousing operations sometimes needed while doing graphic design in Photoshop. But it's adequate for most situations, and faster than using voice dictation to navigate around the screen.

There is one big drawback to a foot mouse. When I first got it and started using it extensively, I very quickly got muscle strain in my leg, much more painful than any of the pain I had experienced in my hands. I had to lay off it for several days, and even now I am careful about its use. But it can be helpful.

Other Hands-Free Mice

There is another foot mouse I discovered while researching this article called the No Hands Mouse. It's not a slipper, but a foot pad which allows you to direct the mouse by pushing in various directions. More information at www.footmouse.com. It's worth watching this demo video of the product if only to hear the appalling "No Hands Mouse" theme song.

Ergoarchitect.com has the StepAssist Foot Pedal which duplicates the "mouse click" functionality of a mouse.

Of course, a hands-free mouse doesn't have to involve the feet. There are other mousing devices using the eyes, the chin, and various parts of the body. Their efficiency is clearly not comparable to using a hand mouse, but might be worth investigating depending on the degree of your disability. I have not tried any of these myself, and cannot vouch for their quality. I merely offer them as areas of further exploration.

The Eye Mouse from www.eyetechds.com This tool tracks your eye movement and moves the mouse accordingly. Demo video here.

The Hands-free touchpad from www.handsfreetouchpad.com is a "chin powered" product which looks rather dubious to me, and destined to create neck issues. I leave you to make your own judgment. Demo video here.