Generating gaussian splat from a vray scene?

[Ihno]

Rendering gaussian splats is cool and all but you know what would get me really exited?
The ability to generate a gaussian splat from a 3d Scene.
These days 80% of my income comes from realtime stuff.
I'd love to setup a scene in vray, setup some sort of region, press a button wait for, (probably ages) and get a ready made gaussian splat out of it.
That would turn my workflow for everything upside down.

I guess I could render a few thousand frames from a scene and generate the splat outside of vray. Which would be cool too. I kinda shy away from the setup time for that though. Is there reserch in that direction from you guys? Is it even possible?

[kosso_olli]

This will be available in the future.
Vlado already showed a working implementation somewhere

[Ihno]

Thats very exiting

[dimitar_panayotov]

If you have any recommendations about workflows related to such a feature, please share them here.

[Ihno]

Im still wrapping my head around this tech.
So I dont know how much sense stuff that I say might make.
If capured from the real world you'd obviously have to walk around objects in order to get them represended from all sides. Idealy you'd do that from different angles too.
So, by that you define the "explorable" area. Still, because you'll also capture the background while doing that, it will also be included but kind of "as seen" from the explorable area. So you can't go there without it falling apart.

Thats what I have in mind when I say:

setup some sort of region, press a button wait for, (probably ages) and get a ready made gaussian splat out of it.

So the way I imagine this is as following:

We can put a gizmo or even better a custom object in the scene which defines the exporable area. Objetcs inside of that area are captured from mutlible different Angles*. But the rest of the scene outside of that area would be also included, but only to be viewn from the exploreable area if you know what I mean.


*As this will be very important in terms of rendertimes in vray and complexity/size of the gaussian spalt scene in the end (i.e realtimyness) I guess it would be good to have some sort of parameter to influrence that.

[Ihno]

What I hope to get is the ablility to get a scene or the static part of it into a realtime engine and be able to use all the tools Max/Forestpack/Vray have to offer.
So I can put background trees with millions of poligons and 3d leafes in. I can use displacement, and where I can, avoid having to have Uvs. All be lit with GI and have raytraced reflections (that obviosliy won't reflect any additional object I'd put in, in the realtime engine.)
I would have used that for many of my projects in the last 5 years.
But of course for it to be usefull, the rendertimes would have to be managable. And the quality would have to be good enough while keeping the file size (because I guess VRAM is the limiting factor here) in reasonable I.e realtime ready bounds.

By "rendertimes would have to be managable"
I mean
1: I'd have to have parameters to balance between quality and rendertimes.
And
2: while it renders be able to view or output the scene so I can judge is everything is going in the right direction.
Much like the resumeable rendering feature which I can set to save its state every n amount of time.
Thats how I manage to do Matterport style 360° tours with 12x2k 6:1 Cubemaps

If I've got those two things in place I'd be able to use it in production even if I'd have to wait for a week or two for it to be done. I guess distributed rendering wouldn't harm though

[cheerioboy]

If I could create a gaussian splat from a 3d model in vray and load it into Unity3d to view in a Quest, that could potentially be huge. Currently I have a very complex process of baking lighting onto the geometry, and importing the geometry into Unity3d. Which loses some of the reflections/materiality.

[Vizioen]

If I could create a gaussian splat from a 3d model in vray and load it into Unity3d to view in a Quest, that could potentially be huge. Currently I have a very complex process of baking lighting onto the geometry, and importing the geometry into Unity3d. Which loses some of the reflections/materiality.

Well you can you just need to render the whole scene from a lot of different angles: https://www.youtube.com/watch?v=KriGDLvGDZI

[cheerioboy]

Well you can you just need to render the whole scene from a lot of different angles: https://www.youtube.com/watch?v=KriGDLvGDZI

I need to experiment with this, could be amazing. Thanks