ModulAR: Eye-Controlled Vision Augmentations for Head Mounted Displays

Jason Orlosky, Takumi Toyama, Kiyoshi Kiyokawa, Daniel Sonntag

Research output: Contribution to journalArticlepeer-review

39 Scopus citations


In the last few years, the advancement of head mounted display technology and optics has opened up many new possibilities for the field of Augmented Reality. However, many commercial and prototype systems often have a single display modality, fixed field of view, or inflexible form factor. In this paper, we introduce Modular Augmented Reality (ModulAR), a hardware and software framework designed to improve flexibility and hands-free control of video see-through augmented reality displays and augmentative functionality. To accomplish this goal, we introduce the use of integrated eye tracking for on-demand control of vision augmentations such as optical zoom or field of view expansion. Physical modification of the device's configuration can be accomplished on the fly using interchangeable camera-lens modules that provide different types of vision enhancements. We implement and test functionality for several primary configurations using telescopic and fisheye camera-lens systems, though many other customizations are possible. We also implement a number of eye-based interactions in order to engage and control the vision augmentations in real time, and explore different methods for merging streams of augmented vision into the user's normal field of view. In a series of experiments, we conduct an in depth analysis of visual acuity and head and eye movement during search and recognition tasks. Results show that methods with larger field of view that utilize binary on/off and gradual zoom mechanisms outperform snapshot and sub-windowed methods and that type of eye engagement has little effect on performance.

Original languageEnglish (US)
Article number7164337
Pages (from-to)1259-1268
Number of pages10
JournalIEEE Transactions on Visualization and Computer Graphics
Issue number11
StatePublished - Nov 15 2015
Externally publishedYes


  • Augmented reality
  • augmented vision
  • eye tracking
  • hands-free interaction
  • modular display
  • visualization

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'ModulAR: Eye-Controlled Vision Augmentations for Head Mounted Displays'. Together they form a unique fingerprint.

Cite this