Ahead of the unveiling of Apple’s Vision Pro, researchers had been probing the efficiency of what eventually became the primary control system for the headset. The Vision Pro, showcased at WWDC, innovatively combines gaze detection and pinch movement to interact with the display. This intuitive approach, dubbed ‘Gaze and Pinch’, allows users to select and interact with objects within the virtual environment seamlessly.
Revolutionizing AR & VR with ‘Gaze and Pinch’
Prior to Apple’s revelation, researchers had been conducting tests on this ‘Gaze and Pinch’ concept back in 2022. The method was pitted against a ‘Handray’ system, requiring users to initially point at an object before pinching to activate it.
Professor Ken Pfeuffer of Aarhus University shed light on these research findings in a Twitter thread. The Gaze and Pinch technique was found to be superior in terms of speed and exerted less physical effort, contingent on the eye-tracking accuracy of the headset.
While real-world efficacy can only be confirmed after Vision Pro hits the market, initial assessments report promising gaze tracking capabilities. It’s noteworthy that Apple has numerous related patents under its belt, including one from 2019.
Comparative analysis: ‘Gaze and Pinch’ vs ‘Handray’
Despite the overall superiority of the ‘Gaze and Pinch’ technique, the ‘Handray’ option outperformed in terms of error minimization. According to Professor Pfeuffer’s summary:
- Speed: Gaze+Pinch (2.5s) > Handray (4.6s)
- Error: Handray (1%) > Gaze+Pinch (3%)
- Preference: Gaze+Pinch (6/16) > Handray (0/16)
These results indicate that while the Gaze and Pinch approach is faster, the Handray method is more accurate. However, user preference clearly leaned towards the Gaze and Pinch method.
Academic references and future developments
The findings referenced in this thread originate from a research paper titled “A Fitts’ Law Study of Gaze-Hand Alignment for Selection in 3D User Interfaces.” Professor Pfeuffer also directed attention towards another study, “PalmGazer: Unimanual Eye-hand Menus in Augmented Reality.”
With the Vision Pro’s release scheduled for the coming months, Apple continues to refine the product. It’s improbable that a foundational aspect like user interaction would change at this point, given the extensive research presumably carried out by Apple over its development years.
What are your thoughts on this intriguing development in the world of AR and VR technology? Feel free to share your opinions in the comments section below!
