How I speak to Unix boxes using NaturallySpeaking

I have tried many methods of trying to control Unix machines by voice, all have worked with some success but I have never totally happy until I stumbled across an idea of using multiple instances of an application to provide me with snappy and reliable response to what I was saying.

Naturally Speaking automatically switches between what you can say in each application by working it out from the title bar.

To connect to different Unix machines I use a program called  PuTTY and by creating multiple copies of PuTTY and renaming the PuTTY executables to have unique names for each machine, each machine ends up with unique title bar name.  In doing so each machine can have it's own vocabulary. This mechanism does not only speed up and improve the recognition of the speech software but it also allows you to automatically switch between different Unix vocabularies of the fly.

A brief description of PuTTY

PuTTY is bit of software that allows you to connect to other machines. It’s a bit like a telnet client for windows but more intelligent and far more secure. You can get PuTTY from here.

A proof of concept:

Here is an example of how I control two Unix boxes, one called Sunbox, which runs the Solaris operating system, and the other called Intelbox, which runs a copy of Redhat. The idea is to completely control the Unix machine handsfree but to also control the vocabularies on the fly so that Redhat system specify commands can only be spoken to the machine running Redhat and only Solaris commands can be spoken to the machine running Solaris.

 

Once it is set up it work like a dream - you can say things like:

And every time you jump between machines you automatically swap Unix vocabularies on the fly! So saying, "install package" would produce the appropriate command depending on what platform you were currently on!

I told you it was sexy, didn't I?

1st things first - setting up multiple instances of PuTTY

First you need a copy of the application called PuTTY. Simply copy the program into a spare directory and create two copies called Sunbox.exe and Intelbox.exe.

So for this example we have two executable called:

 
D:Utilities\sunbox.exe
D:Utilities\intelbox.exe

And that's it, basically you need a copy of PuTTY for each machine you wish to speak to.

The Underlying Macros

This is the "glue" that makes it all work and the good news is that you only need to set up two global macros to get things working!

The two global macros are:

 
"Log Onto <Machines>"
 
IF (_arg1 = "sunbox") THEN
ShellExecute "D:Utilities\sunbox.exe"
Wait 750
SendKeys "sunbox.csi.cam.ac.uk"
Wait 500
SendKeys "{Alt+h}"
Wait 500
SendKeys "{Alt+o}"
Wait 2000
AppSwapWith "sunbox"
Wait 500
SendKeys "My-username{Enter}"
Wait 1000
SendKeys "My-password{Enter}"
Wait 1000
END IF
IF (_arg1 = "intelbox") THEN
ShellExecute "D:Utilities\intelbox.exe"
Wait 250
SendKeys "intelbox.linuxmachines.org"
Wait 500
SendKeys "{Alt+h}"
Wait 500
SendKeys "{Alt+o}"
Wait 2000
AppSwapWith "intelbox"
Wait 750
SendKeys "My-Username{Enter}"
Wait 1000
SendKeys "My-password{Enter}"
Wait 1000
END IF
IF (_arg1 = "1") THEN
ShellExecute "D:Utilities\1.exe"
Wait 750
SendKeys "1.cam.ac.uk"
Wait 500
SendKeys "{Alt+h}"
Wait 500
SendKeys "{Alt+o}"
Wait 2000
AppSwapWith "1"
Wait 500
SendKeys "My-username{Enter}"
Wait 1000
SendKeys "My-password{Enter}"
Wait 1000
END IF

This macro simply allows you to log onto each machine. You may or may not want to include to password in the macros! The additional entries can be used to add new machines when needed.

 
"Jump to <machines>"
 
if _arg1 = "sunbox " then AppBringUp "sunbox"
if _arg1 = "intelbox" then AppBringUp "intelbox"
if _arg1 = "1" then AppBringUp "1"
if _arg1 = "1" then AppBringUp "1"
if _arg1 = "1" then AppBringUp "1"

This macro simply changes the focus to the correct machines. The additional entries can be used to add new machines to your set up when needed.

Don't forget you will need to define a "machines" list.

Talking Unix

To speak to a Unix machine is just a simple matter of creating the appropriate macros in each machine's unique vocabulary. Macros basically map your spoken words to unix commands, a simple macro is "List Directory", saying "List Directory" will enter "ls" on the command line and hit the return key for you. Another example of a macro would be: "Grep For Inetd" SendKeys "ps -elf |grep inetd {Enter}". The possibilities are endless but one very important thing to remember is to try to use one utterance to perform one unix task, it saves your voice! Here are my core unix commands

Talking to Unix Applications

I very nice feature of this multiple PuTTY technique is the ability to log onto a machine, automatically start a program and change to this application's vocabulary on the fly. I use this technique to read my mail.

I can say things like:

 
"Log onto Hermes" [This is the machine I read my mail from, this command logs me on and starts Pine]
"Index Folder"
"Read message"
"Reply to message"

Here are my Pine macro that I use to read my mail.