XR / OpenXR — VR Applications in Unity

VR/AR (Unity calls this XR — eXtended Reality) is a large universe of its own. Unity has historically been one of the leaders in the VR-development market. This chapter is a short entry point: what you need to know to run a basic scene on Meta Quest, Valve Index, or PCVR.

The XR stack in Unity 6

Three key packages:

1. OpenXR Plugin (com.unity.xr.openxr) — the lower layer. OpenXR is an open standard from the Khronos Group for VR/AR. It replaced the Oculus SDK, SteamVR, Vive Wave, and others.
2. XR Interaction Toolkit (com.unity.xr.interaction.toolkit, XRI) — a high-level system: ray-pointers, grab, teleport, locomotion, UI interaction. The main package for the developer.
3. XR Hands (com.unity.xr.hands) — hand-tracking (without controllers) for Quest 3, Vision Pro.

Additionally:

• AR Foundation (com.unity.xr.arfoundation) — for AR (mobile ARCore/ARKit, Vision Pro pass-through).

Basic setup

1. Edit → Project Settings → XR Plug-in Management → check OpenXR for the platforms you need (Windows / Android for Quest).
2. OpenXR feature groups: enable the Meta XR feature group for Quest, Khronos for Index/SteamVR.
3. Interaction profiles: Oculus Touch Controller, Khronos Simple Controller, HTC Vive Controller — enable all the ones you are targeting.
4. Package Manager → install XR Interaction Toolkit.
5. Add an XRI Sample → XR Origin (XR Rig) to the scene.

XR Origin — the root node of a VR scene

XR Origin (XR Rig)            ← position and rotation track the player's floor
├── Camera Offset
│   ├── Main Camera           ← XR camera, tracked
│   ├── Left Controller        ← XR Controller (Direct + Ray Interactor)
│   └── Right Controller       ← XR Controller
└── Locomotion System
    ├── Teleportation Provider
    └── Continuous Move Provider

XR Origin is a GameObject that corresponds to “the user’s position in real space”. When the player walks around the room, their Main Camera inside the XR Origin moves. When you want to teleport the player, you move the XR Origin.

Simple grab and ray-interaction

XRI ships with ready-made components:

• XR Ray Interactor — a ray from the controller for distant selection.
• XR Direct Interactor — pick up an item by hand on touch.
• XR Grab Interactable — on an object, so that it can be grabbed.
• XR Simple Interactable — on a button/door for a tap-action.

using UnityEngine;
using UnityEngine.XR.Interaction.Toolkit;
using UnityEngine.XR.Interaction.Toolkit.Interactables;

[RequireComponent(typeof(XRGrabInteractable))]
public class PickupSword : MonoBehaviour
{
    private XRGrabInteractable _grab;

    private void Awake() {
        _grab = GetComponent<XRGrabInteractable>();
        _grab.selectEntered.AddListener(OnPickedUp);
        _grab.selectExited.AddListener(OnDropped);
    }

    private void OnPickedUp(SelectEnterEventArgs args) {
        Debug.Log($"Picked up by {args.interactorObject}");
    }

    private void OnDropped(SelectExitEventArgs args) {
        // Possibly return it to its place after 5 seconds
    }
}

XRGrabInteractable handles physics, attach point, and throw velocity by itself — you only subscribe to the events.

Teleport-locomotion

The standard VR pattern that “doesn’t cause motion sickness”:

XR Origin
└── Locomotion System
    ├── Teleportation Provider

Teleport Area (on the floor)
└── Teleportation Area (XR Interactor target)
    Collider: Plane or Mesh

XR Right Controller
├── XR Ray Interactor (with TeleportRequest line type)

The player points the ray at the Teleport Area → clicks → the XR Origin jumps there. XRI handles target highlighting and the animation.

Alternatively, use Continuous Move Provider for smooth stick-driven movement (for experienced players who don’t get sick in VR).

Hand-tracking (without controllers)

With the XR Hands package, you can use your hands instead of controllers (Quest 3, Vision Pro):

1. Install com.unity.xr.hands.
2. In Project Settings → OpenXR → activate the Hand Tracking feature.
3. In the scene, use XR Hand instead of XR Controller. It has a skeleton with 26 joints (knuckles, fingertips).
4. Ready-made gesture-detection (point, pinch, fist) — via XRHandTrackingEvents.

using UnityEngine.XR.Hands;

public class PinchDetector : MonoBehaviour
{
    [SerializeField] private XRHandTrackingEvents handEvents;

    private void OnEnable() {
        handEvents.poseUpdated.AddListener(OnPoseUpdated);
    }

    private void OnPoseUpdated(XRHandJointsUpdatedEventArgs args) {
        if (args.hand.GetJoint(XRHandJointID.IndexTip).TryGetPose(out Pose indexPose) &&
            args.hand.GetJoint(XRHandJointID.ThumbTip).TryGetPose(out Pose thumbPose)) {
            float pinch = Vector3.Distance(indexPose.position, thumbPose.position);
            if (pinch < 0.02f) {
                Debug.Log("Pinching!");
            }
        }
    }
}

Performance — the main rules

VR requires a minimum of 72 FPS on Quest, 90 FPS on PCVR, 120 FPS on Index/Vision Pro — with no drops. This is stricter than a regular game.

• Forward+ or Mobile renderer is mandatory.
• MSAA 4× is the standard for VR (anti-aliasing is critical).
• Single-Pass Instanced rendering is mandatory (Project Settings → XR Plug-in Management → OpenXR → Render Mode).
• Foveated rendering — reducing quality at the periphery (on Quest Pro, Vision Pro). Enabled through OpenXR features.
• Polygon budget on Quest 2 — ~350k triangles per frame. On Quest 3 it’s more, but not ×10.

Testing

• Editor Play Mode + XR Device Simulator — without a headset, emulated with the mouse. In the com.unity.xr.interaction.toolkit.samples.starter-assets package.
• PCVR (Link/AirLink) — Quest connected to a PC, Play Mode runs directly on the headset.
• Standalone Quest build — .apk, installed via ADB or Meta Quest Developer Hub.

Pitfalls

1. Comfort and motion sickness — test on your target audience. What is comfortable for you may make others nauseous.
2. UI in World Space — the main mode for VR. Screen Space Overlay does not work in VR.
3. Audio Listener — must be on the VR Camera (not on the XR Origin).
4. Spatial Audio is mandatory — VR without 3D sound loses half its immersion.