TV-noise VR-only rendering artifacts

Enscape Requests, Issues, Discussions Enscape :: Issues
1

I am seeing some rendering artifacts when using quality high & ultra in VR, which so not exist in desktop-mode (at any quality setting) and do not exist in VR at quality medium.

Generally in can only use VR in medium because of these artifcats. With a 4090 I could sometime use High quality performance wise, but the rendering artifacts are too distracting.

What’s the cause for these? They do seem to be worse when the lighting isn’t 100% sunlight, but some artificial. Maybe more pronounced the dimmer a scene gets. However: On desktop the exact same scene with the same lighting renders crystal-clear, and when switching to VR mode , the rendering is very distracting. While wall, get a 90’s TV noise artifcat animation. Pics (the color difference might be due to the fact that I have to adjust exposure and saturation to counter the current bug where VR is way too bright):

vr-enscape.jpg
vr-enscape.jpg1500×683 328 KB

2

Here’s another image, showing the TV-noise effect in a situation where it is much less pronounced. However in real-time VR it is still very distracting because the TV-noise is constantly animated, making the scene very busy.

Other things to notice: The lighting at VR Ultra seem very different from Desktop ultra.

VR might actually be a better lighting than desktop if it werent for the TV noise.

  • On desktop the Enscape Shelf 010 acts as a light source for some reason on quality Ultra (not on Medium).

  • In VR the Shelf 010 is not a light source.

  • Light scattered across the floor from the door window actually looks better in VR than on desktop, but that might be because of the self-illumating shelf on desktop.

  • Shadows on wall in VR Ultra would actually look better than Desktop Ultra.

  • The chairs seem to cast a shadow in VR from the light from the door windows, but they don’t on desktop.

  • VR Ultra seem to use way more resources than Desktop Ultra, even after accounting for the fact that VR has to render two images, one for each eye. That scene would use around 20% of a 4090 to render on desktop, but maxes out the GPU in VR and would probably need twice the performance a 4090 offers to render smoothly.

Are the rendering engines fundamentally different in VR vs desktop? That scene looks very different in VR than on desktop, VR and desktop seem to have different bugs, and different performance characteristics.

enscape_quality_comparison.jpg
enscape_quality_comparison.jpg1349×1206 439 KB

3

I would hazard a guess that some of the temporal rendering is not there in VR and that perhaps that settings are changed so that the performance is kept higher - as you need it really really high framerate to not feel awful in VR.

4

After some more testing: It is related to ray tracing. If I turn off ray-traced sun shadows, then the Shelf 010 stop beeing brighly lit and in VR the noise is noticaly reduced. When ray tracing is switched off entirely it is reduced again.

So in essence ray-tracing doen’t work in VR.

Same thing with DLSS which works fine on desktop but not in VR. I wonder what makes VR so different. I figured Enscape would just use the exact same render pipeline, but render it from two different perspectives, once for each eye, but this doesn’t seem to the case?

5

From my experience, the opposite seems to be case. Performance in VR is worse than you would expect.

When rendering a model at Ultra Quality on a 4k desktop @ 60Hz I typically have less than 25% GPU ultilization on a 4090 to render 60 frames / second.

When in VR I need to render two 4k screens @ 90Hz, so 180 frames / second, which would mean that I should be able to render Ultra Quality with 75% ultilization or something like that. But typicall I can’t and get 99% GPU utilization with 50 frames/second and need to drop down to Quality Medium to get my 90 frame / second with 75% GPU.

I understand that these are only paper estimates and in reality it probably doesn’t work that way, but still – performance and rendering quality in VR seem to be worse than it should be, based on the good desktop rendering quality and performance.

6

Hello @burggraben and thank you very much for the detailed explanations and comparisons! We have some things to cover so lets go:

  • VR is so much more demanding that desktop. Having 2 4k screens in VR doesn’t mean that it will just take double time than rendering 1 4k screen in desktop. There are a lot of things that don’t scale very well with resolution, so the GPU utilization doesn’t scale as expected every time. For this reason, we reduce the VR quality by one level compared to the desktop quality; VR ultra quality is actually desktop high quality.

  • In addition, it is not just GPU work that is doubled, there is a lot of processing on the CPU for the preparation of the GPU commands for each eye. We don’t use any Single Pass Stereo technique currently to reduce the CPU overhead of the draw calls.

  • The 90s TV effect that you see is actually noise from the diffuse and specular reflections in the scene. For VR we perform a lot less ray tracing calls to try to stay on a specific framerate and that shows in these areas. Of course we apply a denoising step on the ray tracing results but this can not be enough in some cases. That’s why these are mostly resolved when you disable hardware accelerated ray tracing.

  • DLSS is also a bad candidate for VR. DLSS produces weird artifacts that can be almost invisible on desktop but become very obvious in VR. I would suggest to keep it off because it can also mess with stereopsis of people with more sensitive eye sight. There are also people who don’t notice anything at all.

  • Night time or areas with very low light are always hard to show noise-free; and in VR it becomes an even bigger problem.

Since we will be resolving the issue of the washed out image in VR in the next version, I would like to ask you to try that version and come back to us with what problems still remain.

7

Ahhh, got it. So that’s why I don’t see the bug where the Shelf 010 looks like a light source in the highest VR quality setting.

Correction: Just tested it and Shelf 010 appears like a bright light source (but is probably technically hyper-reflective?) on Deskop high and ultra, but not is VR Ultra. So there is some difference between VR Ultra and Desktop High, which makes Sheld 010 behave this way in VR only.

Just googled and read your blog post on SPS and it sounds interesting. Any plans to use to to improve VR performance?

I am usually not CPU bound in VR high/ultra. The CPU is typically bored, and a 4090 GPU sits a 99% on a high resolution Varjo Aero headset in high and ultra. (Medium is fine). There is only one CPU problem which I identified and reported, but it looks nothing happened on that front.

Extremly high CPU usage in certain spots

I just tested again, the issue still is present. Maybe you could try reproducing it again? I do have the simplified testing file still available which I sent to Demian last year.

Shame, it looks so pretty on desktop, but not pretty and very distracting in VR.

I would actually still prefer if you have a “real” Ultra in VR instead of what we have now.

Even with a 4090 currently I have to use Medium Quality to get 90FPS.

But I’d like to something switch to Ultra to get the prettiest picture in VR. Currently that gives me 50FPS or so, but I don’t use it because it looks worse than Medium. Ultra and High both looks worse than Medium in VR.

I want a “real desktop quality” version in VR, even if I only get 3 FPS. I do architecture, so mostly I just navigate to a spot and looks at it with my head still.

Shame, it seems to be a coin toss right now. Some VR apps / games work with DLSS, others don’t.

Happy to.

8

We generally don’t have that such high geometry complexity to justify the time required to implement such technique, so I can’t promise anything for now. We know the technology, that’s for certain.

With 3.5 we did a lot of changes in lighting even in VR with the new Global Illumination system. Please give that a try too.

That is a controversial topic and I understand where you are coming from. In my previous role I was working on VR for mechanical engineers and they, too, wanted the best possible image and they didn’t care about FPS; they just wanted to stay still and look at the point of interest. The point of controversy is that Enscape is designed to be real time and at least try to stay at specific framerates. If we provided that “real” ultra quality, then customers would think that the program is just bad because it can only produce 5-10 fps at the quality. Quality image in VR is just so hard to produce both in high fps and consistently.

This is why there are only a handful games that have great graphics in VR. The effort it requires to get that level of detail cannot be afforded by small studios; and they know what geometry the use and optimize/bake it before shipping. Enscape has to deal with anything on the fly.

Is the issue still present in 3.5.2? I could retry reproducing it then because I couldn’t see anything close to that behavior before. What is the resolution that you are using?

9

I would agree that real-time, high-FPS VR is the more important of the two for Enscape VR. Currently I run Enscape Desktop as pretty as possible and Enscape VR on whatever gets me to 90 FPS. VR is great to get a sense for how big a space really is, or to let someone stand in their new kitchen and see if the space between island and table is big enough.

Having this as-pretty-as-possible VR mode would just be the cherry on the cake, but it would see much less use than 90-FPS-VR.

On more question, since you seem to be very knowledegable about this (great to see that Enscape has some new devs with VR expertise):

On VR Medium I get crystal clear 90 FPS, on VR High I get lower FPS around 50 or so, which isn’t great but acceptable.

There is another reason I don’t use VR High and Ultra and stick with Medium. It’s a little hard to describe, but I’ll try. Whenever I turn my head to a new spot, the image is more grainy and the it take a couple second before it “loads” additional pixels and gets rid of some of that grain. Like a classic TV-show picture loading animation. It is distinct from the noise from the diffuse and specular reflections we discussed above, because that is permanent. This goes away after a few seconds or looking in the same direction. Any idea?

Yes it is present in 3.5.2. My resolution is 2880 x 2720 px per eye @ stable 90 FPS on industry-grade Varjo Aero headset. But this also used to occur on my much lower resolution consumer-grade Valve index before I switched. If you can’t manage to reproduce I can test again on the Index. It seems to be GPU load independant however. There are certain geometries which just cause crazy CPU loads, but only sometimes, depending on how far you are from them, how long you have been looking at them, etc. With the table Revit it is pretty reproducible if you hover over it and move around.

10

Just an idea: Maybe Enscape doesn’t always have to do everything on the fly. On-the-fly mode is great while developing a model and making changes.

Once it is semi-finished, VR performance become more important than making real-time changes. It would be nice having a packaging-export-feature which would package everything for optimal performance. It could produce an unchangable exe or something. It would be fine if this process took several hours because it optimizes things, bakes lighting etc just like it takes a long time to compile a game.

Just an idea. I know you probably don’t have time for it, since it is an epic task.

11

This is the global illumination algorithm warming up when new geometry enters the view or the lighting of the space changes drastically. In simple terms, to be able to emulate a lot of bounces for global illumination, we store the calculated lighting for the surfaces that have entered the view and reuse that for other bounces. The stored value is updated as more rays enter the scene and light gets scattered.
This “loading” that you see is just every new surface converging to the correct lighting after a lot of bounces. Since in VR we perform less ray traces, this convergence effect is more pronounced in VR but you can see it in desktop too if you rotate the camera very fast or move very fast in the scene and new objects come into view.

I managed to reproduce it on an Index and filed a bug for it, thanks for forwarding your test scene again!

This is actually a great idea that has been getting in discussions by the devs on and off but since it is a very big feature, it hasn’t been scheduled yet. What I can do is just to link to the Enscape Roadmap Portal so that you can raise your voice about this feature there:

Enscape Roadmap Portal (How to forward/upvote Feature/Asset Requests directly)

12

I saw in the release notes of the latest alpha that you split up the previous denoiser into a GI Denoiser and a Shadow Denoiser. Is that supposed to help with the issue from the thread or should I expect to see improvement in some other area?

13

Now that we have separate denoisers we can optimize their settings for each use case. So you can expect better (but not wildly different) results in walkthrough and captures but not much difference in VR yet.