[Enlarge image]Left: Artist’s rendering of a lightweight AR headset powered by étendue-expanded holography, including the fabricated neural étendue expander. Right: Holograms generated without and with neural étendue expansion. The increase in étendue appears as an increase in the field of view of the holographic display.
Holographic displays can generate light fields by dynamically modulating the wavefront of a coherent light beam using a spatial light modulator (SLM), promising rich virtual-reality (VR) and augmented-reality (AR) applications.1,2 However, the limited spatial resolution of existing dynamic SLMs imposes a tight bound on the diffraction angle. In work published this year, we offered a path to usable, wearable and ultrathin AR displays through neural étendue expanders. These elements, enabled by artificial intelligence (AI) techniques, can effectively increase étendue—a key performance parameter in these visual systems—by two orders of magnitude.3 Compared with existing state-of-the-art hand-engineered approaches,4,5 neural étendue expanders produce high-fidelity, full-color holograms for complex scenes.
Holographic displays use optical elements that are small enough to fit into a regular pair of glasses, and they project images that are seamlessly integrated into a user’s normal field of view. Headsets that rely on a monitor, on the other hand, tend to be bulky, as they must accommodate a screen and the hardware necessary to operate it. Thus, holographic displays have the potential to become ubiquitous, transforming how we interact with our environments, in tasks ranging from getting directions while driving to monitoring a patient during surgery to accessing plumbing instructions while doing a home repair.
Étendue is a physical quantity associated with any optical system that sets an upper bound on the product of field-of-view and eyebox size. The SLMs that produce digital holograms possess a limited étendue that is orders of magnitude lower than that of analog recording devices, resulting in either a narrow field of view or an unusably small eyebox. Unfortunately, there is no foreseeable path toward building a high-étendue electronic SLM in the near future due to fundamental issues such as pixel crosstalk, power consumption and device size. Thus, compact passive elements that integrate with the SLM are poised to make the biggest impact on common practice in this active area of research and development.
In our work, we turned to AI to learn an étendue-expanding element. This allowed us to close the gap in étendue without sacrificing other criteria, such as hologram fidelity and form factor. The approach improved the quality of the holographic image, making it larger and clearer and easily viewable for the user. We believe that our work breaks through the étendue frontier long established by fundamental limitations of dynamic SLMs.
Researchers
Ethan Tseng, Praneeth Chakravarthula and Felix Heide, Princeton University, Princeton, NJ, USA
Seung-Hwan Baek, Princeton University, Princeton, NJ, USA, and Pohang University of Science and Technology, Pohang, Republic of Korea
Grace Kuo, Nathan Matsuda, Andrew Maimone, Florian Schiffers and Douglas Lanman, Reality Labs Research, Meta, Redmond, WA, USA
Qiang Fu and Wolfgang Heidrich, King Abdullah University of Science and Technology, Thuwal, Saudi Arabia
References
1. M. Gopakumar et al. Nature 629, 791 (2024).
2. L. Shi et al. Nature 591, 234 (2021).
3. E. Tseng et al. Nat. Commun. 15, 2907 (2024).
4. G. Kuo et al. ACM Trans. Graphics 39, 66 (2020).
5. J. Park et al. Nat. Commun. 10, 1304 (2019).