Multi-User Motion Matching Interaction for Interactive Television using Smartwatches

As continuation on WaveTrace, a novel interaction technique based on Motion Matching Input, we developed a demo to highlight, inspire and spark discussion on the usage of the the developed technique.

We demonstrate an implementation of motion matching input using smartwatches for interactive television, that allows multi-user input using bodily movements and offers new interaction possibilities by means of a second screen as extension on TV display. Besides the advantages of their integrated IMU’s, smartwatches are worn (most of the time) on the user’s wrist, introducing new opportunities. For example, multiple users in the same household can wear smartwatches, and thus interact simultaneously, allowing for multiple interactions occur in parallel. Multiple connected watches concurrently makes the TV controller decentralized, though it remains visible when someone interacts. This adds to the swiftness of interaction as well as the social translucence within a multiuser environment. Furthermore, the smartwatch allows for haptic feedback during interaction and can function as a second screen.

Our demo revolves around a Video on Demand (VOD) application. The VOD presents a variety of interaction possibilities. For example, users are able to navigate through movies using discrete input (going up or down) and continuous input (scrolling right and left). The VOD also varies in visual representation of moving targets and system feedback. For example, the volume is represented in the trajectory of the volume targets that orbit around a speaker icon. Alternatively, the targets for scrolling through time orbit around the horizontal center, with a separate timeline to indicate its progress. Other action possibilities show the system’s behavior if multiple users interact (e.g. add or subtract for continuous input or first- come first-choice for discrete input) and the ability to pull additional data onto the smartwatch screen.


This demo (and corresponding paper) was accepted for publication in TVX ’17 Adjunct Proceedings on Interactive Experiences for Television and Online Video. The paper was co-written by Vassilis-Javed Khan, Augusto Esteves and Saskia Bakker as part of my Master thesis.


  • Share post