Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras

Alexander Plopski; Jason Orlosky; Yuta Itoh; Christian Nitschke; Kiyoshi Kiyokawa; Gudrun Klinker

doi:10.1109/ISMAR.2016.16

Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras

Alexander Plopski, Jason Orlosky, Yuta Itoh, Christian Nitschke, Kiyoshi Kiyokawa, Gudrun Klinker

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

15 Scopus citations

Abstract

Properly calibrating an optical see-through head-mounted display (OST-HMD) and maintaining a consistent calibration over time can be a very challenging task. Automated methods need an accurate model of both the OST-HMD screen and the user's constantly changing eye-position to correctly project virtual information. While some automated methods exist, they often have restrictions, including fixed eye-cameras that cannot be adjusted for different users.To address this problem, we have developed a method that automatically determines the position of an adjustable eye-tracking camera and its unconstrained position relative to the display. Unlike methods that require a fixed pose between the HMD and eye camera, our framework allows for automatic calibration even after adjustments of the camera to a particular individual's eye and even after the HMD moves on the user's face. Using two sets of IR-LEDs rigidly attached to the camera and OST-HMD frame, we can calculate the correct projection for different eye positions in real time and changes in HMD position within several frames. To verify the accuracy of our method, we conducted two experiments with a commercial HMD by calibrating a number of different eye and camera positions. Ground truth was measured through markers on both the camera and HMD screens, and we achieve a viewing accuracy of 1.66 degrees for the eyes of 5 different experiment participants.

Original language	English (US)
Title of host publication	Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016
Editors	Wolfgang Broll, Hideo Saito, J. Edward Swan, Wolfgang Broll
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	94-99
Number of pages	6
ISBN (Electronic)	9781509036417
DOIs	https://doi.org/10.1109/ISMAR.2016.16
State	Published - Dec 12 2016
Externally published	Yes
Event	15th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016 - Merida, Yucatan, Mexico Duration: Sep 19 2016 → Sep 23 2016

Publication series

Name	Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016

Conference

Conference	15th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016
Country/Territory	Mexico
City	Merida, Yucatan
Period	9/19/16 → 9/23/16

Keywords

Eye pose estimation
OST-HMD calibration

ASJC Scopus subject areas

Computer Science Applications
Computer Graphics and Computer-Aided Design
Modeling and Simulation

Access to Document

10.1109/ISMAR.2016.16

Cite this

Plopski, A., Orlosky, J., Itoh, Y., Nitschke, C., Kiyokawa, K., & Klinker, G. (2016). Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras. In W. Broll, H. Saito, J. E. Swan, & W. Broll (Eds.), Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016 (pp. 94-99). Article 7781771 (Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ISMAR.2016.16

Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras. / Plopski, Alexander; Orlosky, Jason; Itoh, Yuta et al.
Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016. ed. / Wolfgang Broll; Hideo Saito; J. Edward Swan; Wolfgang Broll. Institute of Electrical and Electronics Engineers Inc., 2016. p. 94-99 7781771 (Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Plopski, A, Orlosky, J, Itoh, Y, Nitschke, C, Kiyokawa, K & Klinker, G 2016, Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras. in W Broll, H Saito, JE Swan & W Broll (eds), Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016., 7781771, Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016, Institute of Electrical and Electronics Engineers Inc., pp. 94-99, 15th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016, Merida, Yucatan, Mexico, 9/19/16. https://doi.org/10.1109/ISMAR.2016.16

Plopski A, Orlosky J, Itoh Y, Nitschke C, Kiyokawa K, Klinker G. Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras. In Broll W, Saito H, Swan JE, Broll W, editors, Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 94-99. 7781771. (Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016). doi: 10.1109/ISMAR.2016.16

Plopski, Alexander ; Orlosky, Jason ; Itoh, Yuta et al. / Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras. Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016. editor / Wolfgang Broll ; Hideo Saito ; J. Edward Swan ; Wolfgang Broll. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 94-99 (Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016).

@inproceedings{1f7a6f02bd2c4144a9ca1135a32b7242,

title = "Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras",

abstract = "Properly calibrating an optical see-through head-mounted display (OST-HMD) and maintaining a consistent calibration over time can be a very challenging task. Automated methods need an accurate model of both the OST-HMD screen and the user's constantly changing eye-position to correctly project virtual information. While some automated methods exist, they often have restrictions, including fixed eye-cameras that cannot be adjusted for different users.To address this problem, we have developed a method that automatically determines the position of an adjustable eye-tracking camera and its unconstrained position relative to the display. Unlike methods that require a fixed pose between the HMD and eye camera, our framework allows for automatic calibration even after adjustments of the camera to a particular individual's eye and even after the HMD moves on the user's face. Using two sets of IR-LEDs rigidly attached to the camera and OST-HMD frame, we can calculate the correct projection for different eye positions in real time and changes in HMD position within several frames. To verify the accuracy of our method, we conducted two experiments with a commercial HMD by calibrating a number of different eye and camera positions. Ground truth was measured through markers on both the camera and HMD screens, and we achieve a viewing accuracy of 1.66 degrees for the eyes of 5 different experiment participants.",

keywords = "Eye pose estimation, OST-HMD calibration",

author = "Alexander Plopski and Jason Orlosky and Yuta Itoh and Christian Nitschke and Kiyoshi Kiyokawa and Gudrun Klinker",

note = "Publisher Copyright: {\textcopyright} 2016 IEEE.; 15th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016 ; Conference date: 19-09-2016 Through 23-09-2016",

year = "2016",

month = dec,

day = "12",

doi = "10.1109/ISMAR.2016.16",

language = "English (US)",

series = "Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "94--99",

editor = "Wolfgang Broll and Hideo Saito and Swan, {J. Edward} and Wolfgang Broll",

booktitle = "Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016",

}

TY - GEN

T1 - Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras

AU - Plopski, Alexander

AU - Orlosky, Jason

AU - Itoh, Yuta

AU - Nitschke, Christian

AU - Kiyokawa, Kiyoshi

AU - Klinker, Gudrun

PY - 2016/12/12

Y1 - 2016/12/12

N2 - Properly calibrating an optical see-through head-mounted display (OST-HMD) and maintaining a consistent calibration over time can be a very challenging task. Automated methods need an accurate model of both the OST-HMD screen and the user's constantly changing eye-position to correctly project virtual information. While some automated methods exist, they often have restrictions, including fixed eye-cameras that cannot be adjusted for different users.To address this problem, we have developed a method that automatically determines the position of an adjustable eye-tracking camera and its unconstrained position relative to the display. Unlike methods that require a fixed pose between the HMD and eye camera, our framework allows for automatic calibration even after adjustments of the camera to a particular individual's eye and even after the HMD moves on the user's face. Using two sets of IR-LEDs rigidly attached to the camera and OST-HMD frame, we can calculate the correct projection for different eye positions in real time and changes in HMD position within several frames. To verify the accuracy of our method, we conducted two experiments with a commercial HMD by calibrating a number of different eye and camera positions. Ground truth was measured through markers on both the camera and HMD screens, and we achieve a viewing accuracy of 1.66 degrees for the eyes of 5 different experiment participants.

AB - Properly calibrating an optical see-through head-mounted display (OST-HMD) and maintaining a consistent calibration over time can be a very challenging task. Automated methods need an accurate model of both the OST-HMD screen and the user's constantly changing eye-position to correctly project virtual information. While some automated methods exist, they often have restrictions, including fixed eye-cameras that cannot be adjusted for different users.To address this problem, we have developed a method that automatically determines the position of an adjustable eye-tracking camera and its unconstrained position relative to the display. Unlike methods that require a fixed pose between the HMD and eye camera, our framework allows for automatic calibration even after adjustments of the camera to a particular individual's eye and even after the HMD moves on the user's face. Using two sets of IR-LEDs rigidly attached to the camera and OST-HMD frame, we can calculate the correct projection for different eye positions in real time and changes in HMD position within several frames. To verify the accuracy of our method, we conducted two experiments with a commercial HMD by calibrating a number of different eye and camera positions. Ground truth was measured through markers on both the camera and HMD screens, and we achieve a viewing accuracy of 1.66 degrees for the eyes of 5 different experiment participants.

KW - Eye pose estimation

KW - OST-HMD calibration

UR - http://www.scopus.com/inward/record.url?scp=85010399619&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85010399619&partnerID=8YFLogxK

U2 - 10.1109/ISMAR.2016.16

DO - 10.1109/ISMAR.2016.16

M3 - Conference contribution

AN - SCOPUS:85010399619

T3 - Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016

SP - 94

EP - 99

BT - Proceedings of the 2016 IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016

A2 - Broll, Wolfgang

A2 - Saito, Hideo

A2 - Swan, J. Edward

A2 - Broll, Wolfgang

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 15th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2016

Y2 - 19 September 2016 through 23 September 2016

ER -

Automated Spatial Calibration of HMD Systems with Unconstrained Eye-cameras

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this