AruControls – Reimagining Personalised Interactions on Your Digital Devices

National University of SingaporeSingapore

Disrupting, Expressing, Optimizing

Concept / Student


Carina Lim, Choo Yuan Jie, Prasanth Kumaar Kunasilan


Most of us have used the keyboard and mouse to interact with the digital world. As they are immutable objects, they cannot be modified to suit the user’s needs. But what if anyone could create their own set of customisable physical modules to interact with the digital world?
AruControls is an app that connects such DIY physical modules to a particular digital action, such as a keyboard press. It consists of 6 core modules made with AruCo markers, which enable the detection of a physical interaction using Computer Vision. When a marker is detected, the computer will respond according to a digital action, such as a keyboard press, as indicated on the AruControls app.
By coding the app according to marker detection methods, AruControls also expands the possibilities of physical interactions: anyone can personalise their own productivity board with keyboard shortcuts; create their own interfaces as embodied learning tools; or construct their own controllers for new gameplay experiences. More concretely, the app lets users pick a module according to common input archetypes (like a button, toggle, or slider), followed by the desired corresponding computer keypress.
AruControls is also open-source and designed with affordance, tactile feedback, and ease of construction. Find construction instructions at, and access the app at

The Interaction Awards are an initiative of the Interaction Design Association (IxDA), a global community of more than 120,000 individuals worldwide dedicated to the professional practice of Interaction Design. Find out how to join your local group and get involved at

© 2012 - 2023 Interaction Design Association