OpenXR: Add documentation page about the new composition layers

tutorials/xr/index.rst

@@ -30,4 +30,5 @@ Advanced topics
    openxr_settings
    xr_action_map
    xr_room_scale
+   openxr_composition_layers
    openxr_hand_tracking

tutorials/xr/openxr_composition_layers.rst

@@ -0,0 +1,296 @@
+.. _doc_openxr_composition_layers:
+
+OpenXR composition layers
+=========================
+
+Introduction
+------------
+
+In XR games you generally want to create user interactions that happen in 3D space 
+and involve users touching objects as if they are touching them in real life.
+
+Sometimes however creating a more traditional 2D interface is unavoidable.
+In XR however you can't just add 2D components to your scene.
+Godot needs depth information to properly position these elements so they appear at
+a comfortable place for the user.
+Even with depth information there are headsets with slanted displays that make it impossible 
+for the standard 2D pipeline to correctly render the 2D elements.
+
+The solution then is to render the UI to a :ref:`SubViewport <class_subviewport>`
+and display the result of this using a :ref:`ViewportTexture <class_viewporttexture>` on a 3D mesh.
+The :ref:`QuadMesh <class_quadmesh>` is a suitable option for this.
+
+.. note::
+    See the `GUI in 3D <https://github.com/godotengine/godot-demo-projects/tree/master/viewport/gui_in_3d>`_
+    example project for an example of this approach.
+
+The problem with displaying the viewport in this way is that the rendered result
+is sampled for lens distortion by the XR runtime and the resulting quality loss 
+can make UI text hard to read.
+
+OpenXR offers a solution to this problem through composition layers.
+With composition layers it is possible for the contents of a viewport to be projected
+on a surface after lens distortion resulting in a much higher quality end result.
+
+.. note::
+    As not all XR runtimes support all composition layer types,
+    Godot implements a fallback solution where we render the viewport
+    as part of the normal scene but with the aforementioned quality
+    limitations.
+
+.. warning::
+    When the composition layer is supported,
+    it is the XR runtime that presents the subviewport.
+    This means the UI is only visible in the headset,
+    it will not be accessible by Godot and will thus
+    not be shown when you have a spectator view on the desktop.
+
+There are currently 3 nodes that expose this functionality:
+
+- :ref:`OpenXRCompositionLayerCylinder <class_OpenXRCompositionLayerCylinder>` shows the contents of the SubViewport on the inside of a cylinder (or "slice" of a cylinder).
+- :ref:`OpenXRCompositionLayerEquirect <class_OpenXRCompositionLayerEquirect>` shows the contents of the SubViewport on the interior of a sphere (or "slice" of a sphere).
+- :ref:`OpenXRCompositionLayerQuad <class_OpenXRCompositionLayerQuad>` shows the contents of the SubViewport on a flat rectangle. 
+
+Setting up the SubViewport
+--------------------------
+
+The first step is adding a SubViewport for our 2D UI,
+this doesn't require any specific steps.
+For our example we do mark the viewport as transparent.
+
+You can now create the 2D UI by adding child nodes to the SubViewport as you normally would.
+It is advisable to save the 2D UI in a subscene, this makes it easier to do your layout.
+
+.. image:: img/openxr_composition_layer_subviewport.webp
+
+.. warning::
+    The update mode "When Visible" will not work as Godot can't determine whether
+    the viewport is visible to the user.
+    When assigning our viewport to a composition layer Godot will automatically adjust this.
+
+Adding a composition layer
+--------------------------
+
+The second step is adding our composition layer.
+We simply add the correct composition layer node as a child node of
+our :ref:`XROrigin3D <class_xrorigin3d>` node.
+This is very important as the XR runtime positions everything in relation to our origin.
+
+We want to position the composition layer so it is at eye height and roughly 1 to 1.5 meters
+away from the player.
+
+We now assign the SubViewport to the ``Layer Viewport`` property and enable Alpha Blend.
+
+.. image:: img/openxr_composition_layer_quad.webp
+
+.. note::
+    As the player can walk away from the origin point,
+    you will want to reposition the composition layer when the player recenters the view.
+    Using the reference space ``Local Floor`` will apply this logic automatically.
+
+Making the interface work
+-------------------------
+
+So far we're only displaying our UI, to make it work we need to add some code.
+For this example we're going to keep things simple and
+make one of the controllers work as a pointer.
+We'll then simulate mouse actions with this pointer.
+
+This code also requires a ``MeshInstance3D`` node called ``Pointer`` to be added
+as a child to our ``OpenXRCompositionLayerQuad`` node.
+We configure a ``SphereMesh`` with a radius ``0.01`` meters.
+We'll be using this as a helper to visualize where the user is pointing.
+
+The main function that drives this functionality is the ``intersects_ray``
+function on our composition layer node.
+This function takes the global position and orientation of our pointer and returns
+the UV where our ray intersects our viewport.
+It returns ``Vector2(-1.0, -1.0)`` if we're not pointing at our viewport.
+
+We start with setting up some variables, important here are the export variables
+which identify our controller node with which we point to our screen.
+
+.. code:: gdscript
+
+    extends OpenXRCompositionLayerQuad
+
+    const NO_INTERSECTION = Vector2(-1.0, -1.0)
+
+    @export var controller : XRController3D
+    @export var button_action : String = "trigger_click"
+
+    var was_pressed : bool = false
+    var was_intersect : Vector2 = NO_INTERSECTION
+
+    ...
+
+Next we define a helper function that takes the value returned from ``intersects_ray``
+and gives us the global position for that intersection point.
+This implementation only works for our ``OpenXRCompositionLayerQuad`` node.
+
+.. code:: gdscript
+
+    ...
+
+    func _intersect_to_global_pos(intersect : Vector2) -> Vector3:
+        if intersect != NO_INTERSECTION:
+            var local_pos : Vector2 = (intersect - Vector2(0.5, 0.5)) * quad_size
+            return global_transform * Vector3(local_pos.x, -local_pos.y, 0.0)
+        else:
+            return Vector3()
+
+    ...
+
+We also define a helper function that takes our ``intersect`` value and
+returns our location in the viewports local coordinate system:
+
+.. code:: gdscript
+
+    ...
+
+    func _intersect_to_viewport_pos(intersect : Vector2) -> Vector2i:
+        if layer_viewport and intersect != NO_INTERSECTION:
+            var pos : Vector2 = intersect * Vector2(layer_viewport.size)
+            return Vector2i(pos)
+        else:
+            return Vector2i(-1, -1)
+
+    ...
+
+The main logic happens in our ``_process`` function.
+Here we start by hiding our pointer,
+we then check if we have a valid controller and viewport,
+and we call ``intersects_ray`` with the position and orientation of our controller:
+
+.. code:: gdscript
+
+    ...
+
+    # Called every frame. 'delta' is the elapsed time since the previous frame.
+    func _process(_delta):
+        # Hide our pointer, we'll make it visible if we're interacting with the viewport.
+        $Pointer.visible = false
+
+        if controller and layer_viewport:
+            var controller_t : Transform3D = controller.global_transform
+            var intersect : Vector2 = intersects_ray(controller_t.origin, -controller_t.basis.z)
+
+    ...
+
+Next we check if we're intersecting with our viewport.
+If so, we check if our button is pressed and place our pointer at our intersection point.
+
+.. code:: gdscript
+
+    ...
+
+            if intersect != NO_INTERSECTION:
+                var is_pressed : bool = controller.is_button_pressed(button_action)
+
+                # Place our pointer where we're pointing
+                var pos : Vector3 = _intersect_to_global_pos(intersect)
+                $Pointer.visible = true
+                $Pointer.global_position = pos
+
+    ...
+
+If we were intersecting in our previous process call and our pointer has moved,
+we prepare a :ref:`InputEventMouseMotion <class_InputEventMouseMotion>` object
+to simulate our mouse moving and send that to our viewport for further processing.
+
+.. code:: gdscript
+
+    ...
+
+                if was_intersect != NO_INTERSECTION and intersect != was_intersect:
+                    # Pointer moved
+                    var event : InputEventMouseMotion = InputEventMouseMotion.new()
+                    var from : Vector2 = _intersect_to_viewport_pos(was_intersect)
+                    var to : Vector2 = _intersect_to_viewport_pos(intersect)
+                    if was_pressed:
+                        event.button_mask = MOUSE_BUTTON_MASK_LEFT 
+                    event.relative = to - from
+                    event.position = to
+                    layer_viewport.push_input(event)
+
+    ...
+
+If we've just released our button we also prepare
+a :ref:`InputEventMouseButton <class_InputEventMouseButton>` object
+to simulate a button release and send that to our viewport for further processing.
+
+.. code:: gdscript
+
+    ...
+
+                if not is_pressed and was_pressed:
+                    # Button was let go?
+                    var event : InputEventMouseButton = InputEventMouseButton.new()
+                    event.button_index = 1
+                    event.pressed = false
+                    event.position = _intersect_to_viewport_pos(intersect)
+                    layer_viewport.push_input(event)
+
+    ...
+
+Or if we've just pressed our button we prepare
+a :ref:`InputEventMouseButton <class_InputEventMouseButton>` object
+to simulate a button press and send that to our viewport for further processing.
+
+.. code:: gdscript
+
+    ...
+
+                elif is_pressed and not was_pressed:
+                    # Button was pressed?
+                    var event : InputEventMouseButton = InputEventMouseButton.new()
+                    event.button_index = 1
+                    event.button_mask = MOUSE_BUTTON_MASK_LEFT
+                    event.pressed = true
+                    event.position = _intersect_to_viewport_pos(intersect)
+                    layer_viewport.push_input(event)
+
+    ...
+
+Next we remember our state for next frame.
+
+.. code:: gdscript
+
+    ...
+
+                was_pressed = is_pressed
+                was_intersect = intersect
+
+    ...
+
+Finally, if we aren't intersecting, we simply clear our state.
+
+.. code:: gdscript
+
+    ...
+
+            else:
+                was_pressed = false
+                was_intersect = NO_INTERSECTION
+
+
+Hole punching
+-------------
+
+As the composition layer is composited on top of the render result,
+it can be rendered in front of objects that are actually forward of the viewport.
+
+By enabling hole punch you instruct Godot to render a transparent object
+where our viewport is displayed.
+It does this in a way that fills the depth buffer and clears the current rendering result.
+Anything behind our viewport will now be cleared,
+while anything in front of our viewport will be rendered as usual.
+
+You also need to set ``Sort Order`` to a negative value,
+the XR compositor will now draw the viewport first, and then overlay our rendering result.
+
+.. figure:: img/openxr_composition_layer_hole_punch.webp
+   :align: center
+
+   Use case showing how the users hand is incorrectly obscured
+   by a composition layer when hole punching is not used.