Nvidia denoiser has made a huge impact in our workflow, allowing us to vizualize a close aproximation of the final render much faster.
With the latest set of supersampling algorithms, I think there is another oportunity to bring even more speed into the process.
My idea is to have a checkbox called SuperSampling that would essentially devide the render size by half and use AI to upscale to the expected resolution.
I’m not sure, but I believe supersampling is much faster than actually rendering 4x the pixels.
This won’t be intended for final renders. But it could be awesome to speed up work.
Question is, would AI upscaler benefit from 3D data that renderer can provide? If not, then i think it’s better to leave it for post. Personally, i don’t think that i would use such feature much, if at all, but then again, i thought the same about denoiser, before Corona had any, but now i barely can do a render without it.
I believe the AI would benefit since 3D data can provide information where the edges are. So everything would be bit less smooth than post-production tool provides.
Although, judging the results from nVidia DLSS which sort of does this (although primarily as means for smooth animation AA) the results is kinda smooth.
I believe it will be necessary eventually anyway, rendering 2k animation is quite expensive and timey hassle in Corona (or offline renderer of any sort for the matter) yet we already see amazingly sharp animation in 4k from Unreal rendered basically instantly with the new cards. Offline rendering is still easier and much better quality but at some point, if you can get 8k Unreal animation within single day for nothing with single GPU, or wait two weeks for 2k Corona animation, the decision will be a lot different.
I would never personally use these hacks for still images where every loss of quality is noticeable (I barely use denoising on finals these days), but animation is concerning me.
I didn’t imagine to be meant for final images, but for drafting… however, it would be great for final animations. In my tests, upscaling 1920 to 4k is kind of the best scenario, it’s hard to see the difference.
Upscaling 960 to 1920 does produce some artifacts, but I still think it would be useful in the same way Nvidia Denoiser is.
why you want this in 3ds max as it’s post processing after all, and there is many upscaling algorytms in presents what you may choose without implementing it in 3ds max. I can’t imagine why you need to save images with upscaling artifacts right after rendering, and how you will fix them if you rendering and save animation sequence? Again this should be postprocessing work from raw renders images
This would suppose to be magical “AI” upscaling, and it will result in a much faster rendering while looking almost the same, so it will help you in lookdev and also in time consuming animation rendering.
Kind of like Nvidia denoising, which already does something similar, instantly showing you the results. You could have the same argument there too: Why not denoise in post-processing ? instead of instant, during rendering.
AI upscaling is much better than pretty much any other upscaling algorithm out there. Opening the possibility of a feature where you would actually render at a much lower resolution and 4x faster (or even more), while looking pretty much the same as a higher resolution render.
Again, I’m not saying this feature would be useful for final renders. As even the animation scenario described above would be covered by current plugins, as you only need to set up and run once.
Maybe you work in a different way, but when I’m working on a new scene, I don’t need to see 100% of the details all the time. If I can get a 95% approximation that allows me to place and adjust lights, adjust materials etc… it can speed up my workflow significantly, the same way Nvidia Denoiser did.
Please do. Unless i’ll see the magic with my own eyes, i have a hard time to believe those claims “nearly indistinguishable from originally high-rez”. I have very little experience with AI upscalling - only used some online service demo and ocassionally waifu2x - but never managed to see astonishing results even at basic 2x upscalling.
If you’re refering to IR, then there are few questions to be answered. Would IR still be interactive with AI denoiser + AI upscaler? Also you mentioned, that upscaler works best at already pretty high resolution input. Would IR benefit much from upscaler if it does not give optimal results at lower resolution?
I’m not against this request, but i hope that the team won’t jump on that AI bandwagon in expense to more conventional features, which are waited by community with great anticipation for a long time.
I wouldn’t worry about devs jumping on bandwagon ;- ). It took half a year (or year?) to get nVidia denoiser which is amazing and less than day to get Intel one which is pure shit and no one asked for it. It’s not done on basis of request intensity.
This tech doesn’t look near close to ready for real-time implementation anyway, but let’s watch how fast and good something like DLSS will get and it will be an absolute must eventually. 8k TVs are here, 8k monitors around the corner.
Usually the results could have big differences in quality depending on the source image. Some said it can depend on what kind of images the resize algorithm was trained with.
Also usually works best not from high resolution images, low resolution could work just as well, but for good results that low resolution image should be at a “final” quality and noise free, which would maybe conflict with the idea of drafting or lookdev usage.
Been using Enhanced AI in my (RnD) pipeline for some time now… quite a good time saver.
BTW, bad quality comes from not having enough of proper training, lack of “learning” material to cover all cases. Which is most obvious with OIDN & was also observed with NVidia Ai. But better times are coming, machines are learning & we’re just at the beginning… soon even standard scenes will be designed, visualized, produced, delivered & build by machines… still under human control
Tho let’s not get ahead of ourselves… we’ve already seen idiots on speed doing more damage than good. Speed kills!
Ever since a thread linked a product called Evotis (now their website is not public facing anymore, but there are press reports and the wayback machine), which saved samples instead of pixels to create subpixel perfect masks, I am very hopeful for a resolution independent renderer.
Instead of losing a sample’s information in a pixel through the reconstruction filter, it would be interesting to save all the samples to disk. Then after rendering you could set the resolution after the fact.
Kinda like RAW allows you to set white balance, because debayering has not been done yet - saving samples would allow you to set a resolution after rendering finished, because the samples were not collapsed into pixels yet. Be it 480p or 4k. Ignoring pixel-grid alignment, 16 passes at 1080p would equate to 4 passes at 4k with no loss in sharpness, allowing you to switch back and forth.
In the world of offline rendering this would be way more useful than upscaling.
Hope to code up a prototype of this sometime this year.
Hmmm… quite doubtful about it. From “Performance Evaluation of Evotis within a VFX Environment” study by Tim Klink (August 2018) it seemed as there’s way too much overhead.
As Ondra put it years ago: “not in the foreseeable future ;)”
I mean - Yeah. With 16 Passes you will have effectively wasted space of 16 Full-res EXRs. (Although the math works out differently)
That storage hit is no joke.
But Evotis had different goals in mid - subpixel perfect compositing. Deep Compositing mostly solved this in the high end VFX world. I would just use the samples to rebuild a dynamic resolution image. There is basically no overhead in terms of calculation. Although running 100 passes worth of samples through a reconstruction filter like the default “Tent” seems much, the averaging is very quick. (Considering every Progressive renderer like corona does this every time the VFB updates for one pass anyways…)
Compositing every frame of an animation that way is deadly to performance. Updating the VFB for a different res on a single image? Quite easy on resources.
Also this would be a checkbox type of thing, if such an idea would ever be implemented. Saving samples to disk is not really difficult to do. Which is why I wanna tackle it as a side project :]
Just to get an idea, we could very easily do this. Do some render, take your “samples/s” value, multiply it with 12 * number of your render elements. That is the bandwidth produces. If you can store it somewhere, then we can talk about coding this ;).
not sure how serious this jab was, but you need to consider that for nvidia denoiser we had to solve CUDA deployment in Corona and add concept of realtime denoising, which was not previously present. For intel denoiser we did not have to do anything but compile and link new library. Also we got requests for intel denoiser. As always, I have no problem with people asking me daily to implement something, but what I really hate is some people dissing feature requests of others as “nobody asked for it” or “that is useless” or “people want this only because they are noobs” etc.
