Hello XR Multiview

Overview

Virtual reality applications must render two separate views (left and right eye) for stereoscopic 3D. The naive approach duplicates all rendering commands—issuing every draw call twice with different camera matrices. This doubles CPU overhead and wastes time on redundant state changes.

The Multiview extension (Vulkan 1.1) solves this by broadcasting a single set of draw calls to multiple layers of an image array. Each layer represents one eye's view. The GPU automatically replicates geometry processing, while shaders use gl_ViewIndex to access per-view data (like different projection matrices). This cuts CPU overhead nearly in half while maintaining full stereo rendering quality.

This example extends Hello XR (OpenXR VR Application) by adding multiview optimization. Instead of rendering each eye separately, the projection layer uses multiview to render both eyes efficiently. For VR applications with complex scenes and many draw calls, multiview provides substantial performance improvements—especially important for maintaining the 90+ Hz refresh rates required for comfortable VR.

Vulkan Requirements

• Vulkan Version: 1.1 or later (for multiview core support)
• Required Extensions:
  • VK_KHR_multiview (core in Vulkan 1.1)
• Required Features:
  • multiview: Enables rendering to multiple views with one draw call
  • multiviewGeometryShader (optional): Allows geometry shaders with multiview
  • multiviewTessellationShader (optional): Allows tessellation with multiview
• OpenXR: Requires OpenXR runtime with multiview composition layer support

Key Concepts

Multiview Rendering: Multiview broadcasts each draw command to multiple layers of an image array attachment. Instead of:

for (int eye = 0; eye < 2; eye++) {
    bindCamera(eye);
    drawScene(); // 100+ draw calls
}

You do:

bindBothCameras();
drawScene(); // 100+ draw calls, automatically rendered to both layers

View Index:

Shaders receive gl_ViewIndex (GLSL) or SV_ViewID (HLSL), an integer [0, viewCount) identifying which layer is being rendered. Use this to index into arrays of view/projection matrices: ```glsl

gl_Position = viewProjection[gl_ViewIndex] * vec4(inPosition, 1.0);

**Image Arrays**: Multiview requires array textures (2D array images). Each layer is a separate image accessed by index. OpenXR swapchains provide these automatically for multiview projection layers.

**Multiview vs Instanced Rendering**: Both can render multiple views, but multiview is more efficient:

* **Multiview**: Single vertex shader invocation per vertex, geometry broadcast to layers by hardware
* **Instanced Rendering**: Multiple vertex shader invocations (one per view), manual layering in geometry shader


Multiview has lower overhead and better hardware support for VR use cases.


# Implementation Details

**OpenXR Multiview Configuration**: The projection layer must request multiview support from the OpenXR runtime. This enables the runtime to provide multiview swapchains:

```cpp

    // Create a projection layer to render the 3D scene
    const XrProjectionLayerOptions projectionLayerOptions = {
        .device = &m_device,
        .queue = &m_queue,
        .session = &m_session,
        .colorSwapchainFormat = m_colorSwapchainFormat,
        .depthSwapchainFormat = m_depthSwapchainFormat,
        .samples = m_samples.get()
    };
    m_projectionLayer = createCompositorLayer<ProjectionLayer>(projectionLayerOptions);
    m_projectionLayer->setReferenceSpace(m_referenceSpace);

    // Create a quad layer to render the ImGui overlay
    const XrQuadLayerOptions quadLayerOptions = {
        .device = &m_device,
        .queue = &m_queue,
        .session = &m_session,
        .colorSwapchainFormat = m_colorSwapchainFormat,
        .depthSwapchainFormat = m_depthSwapchainFormat,
        .samples = m_samples.get()
    };
    m_quadImguiLayer = createCompositorLayer<XrQuadImGuiLayer>(quadLayerOptions);
    m_quadImguiLayer->setReferenceSpace(m_referenceSpace);
    m_quadImguiLayer->position = { -1.0f, 0.2f, -1.5f };

    // Create a cylinder layer to render the ImGui overlay
    const XrCylinderLayerOptions cylinderLayerOptions = {
        .device = &m_device,
        .queue = &m_queue,
        .session = &m_session,
        .colorSwapchainFormat = m_colorSwapchainFormat,
        .depthSwapchainFormat = m_depthSwapchainFormat,
        .samples = m_samples.get()
    };
    m_cylinderImguiLayer = createCompositorLayer<XrCylinderImGuiLayer>(cylinderLayerOptions);
    m_cylinderImguiLayer->setReferenceSpace(m_referenceSpace);
    m_cylinderImguiLayer->position = { 1.0f, 0.2f, 0.0f };
    m_cylinderImguiLayer->radius = 2.0f;
    m_cylinderImguiLayer->centralAngle = 1.0f; // 1 radians = 57.3 degrees

Filename: hello_xr_multiview.cpp

Multiview Graphics Pipeline: The pipeline options specify viewCount to inform Vulkan how many views will be rendered. This metadata enables driver optimizations:

Filename: projection_layer.cpp

Multiview Render Pass: Legacy render passes also need the view count. With KDGpu's render pass options:

    auto opaquePass = commandRecorder.beginRenderPass(KDGpu::RenderPassCommandRecorderOptions{
            .colorAttachments = {
                    {
                            .view = m_colorSwapchains[0].textureViews[m_currentColorImageIndex],
                            .clearValue = { 0.3f, 0.3f, 0.3f, 1.0f },
                            .finalLayout = TextureLayout::ColorAttachmentOptimal,
                    },
            },
            .depthStencilAttachment = {
                    .view = m_depthSwapchains[0].textureViews[m_currentDepthImageIndex],
            },
            .viewCount = viewCount(),
    });

Filename: projection_layer.cpp

Unified Rendering: All draw calls are issued once. The shader uses gl_ViewIndex to select the appropriate matrices and render to the correct layer:

    m_fence.wait();
    m_fence.reset();

    // Update the scene data once per frame
    updateTransformUbo();

    // Update the per-view camera matrices
    updateViewUbo();

    auto commandRecorder = m_device->createCommandRecorder();

    // Set up the render pass using the current color and depth texture views
    auto opaquePass = commandRecorder.beginRenderPass(KDGpu::RenderPassCommandRecorderOptions{
            .colorAttachments = {
                    {
                            .view = m_colorSwapchains[0].textureViews[m_currentColorImageIndex],
                            .clearValue = { 0.3f, 0.3f, 0.3f, 1.0f },
                            .finalLayout = TextureLayout::ColorAttachmentOptimal,
                    },
            },
            .depthStencilAttachment = {
                    .view = m_depthSwapchains[0].textureViews[m_currentDepthImageIndex],
            },
            .viewCount = viewCount(),
    });

    // Draw the main triangle
    opaquePass.setPipeline(m_pipeline);
    opaquePass.setVertexBuffer(0, m_buffer);
    opaquePass.setIndexBuffer(m_indexBuffer);
    opaquePass.setBindGroup(0, m_cameraBindGroup);
    opaquePass.setBindGroup(1, m_entityTransformBindGroup);
    const DrawIndexedCommand drawCmd = { .indexCount = 3 };
    opaquePass.drawIndexed(drawCmd);

    // Draw the left hand triangle
    opaquePass.setVertexBuffer(0, m_leftHandBuffer);
    opaquePass.setBindGroup(1, m_leftHandTransformBindGroup);
    opaquePass.drawIndexed(drawCmd);

    // Draw the right hand triangle
    opaquePass.setVertexBuffer(0, m_rightHandBuffer);
    opaquePass.setBindGroup(1, m_rightHandTransformBindGroup);
    opaquePass.drawIndexed(drawCmd);

    opaquePass.end();
    m_commandBuffer = commandRecorder.finish();

    const SubmitOptions submitOptions = {
        .commandBuffers = { m_commandBuffer },
        .signalFence = m_fence
    };
    m_queue->submit(submitOptions);

Filename: projection_layer.cpp

The shader binds an array of ViewProjection structures (one per eye) and indexes by gl_ViewIndex. Each draw call automatically renders to all views without CPU-side loops.

Performance Notes

CPU Reduction: Multiview nearly halves CPU rendering overhead by eliminating redundant:

• Command buffer recording (one recording for both eyes)
• State changes (bind once instead of twice per draw)
• Draw call submission overhead

For scenes with 1000+ draw calls, this is a massive win.

GPU Impact: GPU work remains the same—vertex shading, rasterization, and fragment shading still happen twice (once per eye). However, improved cache locality and reduced driver overhead often yield measurable GPU gains too.

Bandwidth: Writes to both eye textures simultaneously may increase memory bandwidth. On tile-based GPUs (mobile), this is mitigated by on-chip tile memory.

Best Practices:

• Use uniform buffer arrays for per-view data (matrices, frustum planes)
• Ensure shaders index gl_ViewIndex efficiently (avoid divergent branches)
• Consider late vertex shader output for shared geometry processing
• Profile with VR-specific tools (RenderDoc, NSight, vendor profilers)

Hello XR (OpenXR VR Application) for basic OpenXR integration without multiview

Multiview Rendering for multiview with standard (non-XR) rendering

Multiview Stereo Swapchain for multiview stereo rendering to window

Further Reading

• Vulkan Multiview Spec
• OpenXR Multiview Composition Layers
• Multiview Performance Best Practices
• NVIDIA VR Best Practices

---

Updated on 2026-03-31 at 00:02:07 +0000