Custom Mirror for Fisherman's Tale on Oculus Quest
Reaching 72 FPS on Android platform was quite a challenge, especially with the mirror. Rendering it the "classic way' with another camera was not an option, due to the low bandwidth of the Oculus Quest, the fix cost for an additionnal rendering pass was way too high.
I had to write custom shaders to solve this issue :
- Reflected objects are rendered in the main pass.
- A vertex shader projects these objets on the mirror surface and creates an illusion of perspective. Objects are in fact in front of the mirror, flattened, and animated depending on the view so they seem further away than they actually are.
- It works in stereo (VR).
- These flattened objects have in fact a small thickness : vertices are sorted to avoid zFighting.
- A texture (projected in mirror space) is used to clip reflected objects when they get outside of the mirror. It also adds detail (dirt, mirror stain...) for correct integration.
- A blurred cubemap, projected on a proxy (cube) is used for the background.
Custom Mirror with no additionnal rendering pass. Reflected objects are manually projected on mirror surface to create an illution of perspective.
Effect Breakdown with debug mode activated : Player point of view (used by mirror) and editor point of view are decorrelated to see the effect from another perspective.
Material function and parameters
It seems easier to work in mirrorspace (to project mirror texture on objects for example). That's why I send mirror transform matrix to the GPU (using material parameter collection here).
We can transform objects world coordinates in mirrorspace (and the inverse) using matrix informations. We simply use HLSL for matrix and vector multiplication.
This material function projects vertices on mirror surface. It remaps depth and add a small thickness to projected objets to avoid zFighting.
A material function to find the intersection between a line and a plane.
A fake perspective for the animated paintings. For the layers in the backgroung we use some parallax effect in the fragment shader. For the rigged character it is the same trick as the mirror, moving vertices horizontally and vertically to simulate depth.
Here is a demo for our custom made parallax effect. The fragment shader can simulate multiple plans at multiple depths, which are moveable, scaleable et rotateable. There is also a customizable fog effect. This is what is used for the painting layers.
The material function used to simulate parallax. We use tangent space to find the correct offset in the texture's UVs.