MouseClicker: Exploring Tactile Feedback and Physical Agency for People with Hand Motor Impairments
In short
MouseClicker is a low-cost device that presses the buttons of an ordinary mouse when I make a facial expression, and buzzes my fingers at the same moment. It lets someone who cannot press a mouse button use one, and feel the click.
Why this work
Assistive technology usually aims at what a tool does, and stops there. Hands-free input gives the function of a click but none of the touch.
I could already click through my trackpad, which I use with one thumb. What I did not have was the feel of a real mouse, or a physical connection to the device I was using. We call the wider goal holistic inclusion: the sensory and emotional side of using something counts alongside the function.
What we built
An ordinary two-button mouse sits under two small levers, each driven by a servo motor that presses a button. A webcam recognizes my facial expressions, and I chose which expression makes a single click, a double click or a hold, on the left or the right button.
Two coin-sized vibration motors on my index and middle fingers buzz with each press, and the mouse makes its own click sound. The cursor stays on my trackpad. The parts cost about 100 US dollars.
How we studied it
First we explored which vibration signals feel like a click. Then ten people without hand motor impairments, and I, judged vibrations at eight positions on two fingers, at five strengths, with the click sound on or off.
Finally I used the full system at home for several hours a day, and reflected on it after a month.
What we found
People judged the strength of a vibration consistently, but where they felt it best varied from person to person. So intensity can be standardised, while location should be personal. Unexpectedly, the fingernail was the position people identified most reliably. The click sound made little difference, and my own sense of touch was in line with the others’.
After a month I was making my clicks without assistance. Before, a right-click meant asking for help or using slower macro software. It felt as if I were pressing the buttons with my own fingers.
Why it matters
The paper argues for physical agency: acting on a real object yourself, and owning the whole loop from action to feedback. Using the same mouse as my non-disabled peers also made me feel included.
Assistive technology should be judged by how it feels to use, as well as by whether the task gets done.
Cite this paper
Atieh Taheri, Carlos Gilberto Gomez-Monroy, Vicente Borja, and Misha Sra. 2024. MouseClicker: Exploring Tactile Feedback and Physical Agency for People with Hand Motor Impairments. ACM Transactions on Accessible Computing 17, 1, Article 5 (March 2024), 31 pages. https://doi.org/10.1145/3648685
BibTeX
@article{10.1145/3648685,
author = {Taheri, Atieh and Gomez-Monroy, Carlos Gilberto and Borja, Vicente
and Sra, Misha},
title = {MouseClicker: Exploring Tactile Feedback and Physical Agency for
People with Hand Motor Impairments},
journal = {ACM Trans. Access. Comput.},
volume = {17},
number = {1},
articleno = {5},
numpages = {31},
year = {2024},
issue_date = {March 2024},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
issn = {1936-7228},
doi = {10.1145/3648685}
}