Mining Rig turned to a V-Ray Swarm - Expected Rendering Performance?

[dainon_woudstra]

I have a rig that I am using as a rendering node. I own 1 license of V-Ray for Sketchup, 1 free trial license not yet expired, and 1 Render Node license. I'm having issues believing all this is working properly...

** Setup:
Desktop PC: Sketchup Pro 2020 (V-Ray license, 2020 before the Hotfix 2 for 2021)
Windows 10 x64
1 1080Ti
RAM 16BG
Intel Core i7 6700K 4.0GHz
Network - Hardwired to router

Rig: Render Node running V-Ray Swarm (Render Node License), Tested V-Ray for Sketchup 2020, 2021, and stand-alone V-Ray Swarm install
Windows 10 x64
5 1080Ti
RAM 8GB
Pentium G4560 3.5GHz
Network - Wifi Connection to router

Network:
Internal Home Network 1Gbit

Render
Engine CUDA
Quality Medium

Rendering Output
Image Width/Height - 1201 x 513

Scene
8' x 12' rendering of my bathroom remodel
2 mirrors
2 panes of glass
1 window (4 panes of glass
reflective marble shower tiles
reflective glass shower tiles
6 omni lights
2 spot lights
1 sun
*image included

Experience
I'm a software engineer
Been 3D modelling personal home projects for 5 years
2 weeks of V-Ray (with a bunch of help from a professional/friend)

When I run my Sketchup render project from Desktop PC, I see 6 boxes in the render image with my Desktop PC's name... then a little bit later the Swarm section of the V-Ray Asset Editor goes to "100%" and a bit later a bunch of boxes show up with my Rig's name, which quickly disappear. They don't return until the next frame render (Running an animation of scenes, BTW). I watch the the GPU's usage in the Rig's Windows 10's Task Manager and the GPUs look mostly unused (0%). On my Desktop PC it looks consistent around 11%. I'm new to rendering and the V-Ray Swarm, yet assume it should provide quicker rendering by using 5 extra GPU's through distributed rendering. While writing this... it output 8 PNG files in 18 minutes. I understand there can be many contributing factors as I've been working with a co-worker educated in 3D-Modelling, with experience in 3DS Max, but not extensively in rendering. He has worked with a Swarm before at a previous job, but not set one up and suggested I post here.

Any input you can provide is appreciated.

Thank you,
Dainon

[dainon_woudstra]

Update...

I ran the Swarm on another computer and it worked just fine.

Going back to the rig, I disabled all but 1 GPU and it rendered. This was the case until I had 4 cards enabled. For anyone reading this, I believe the issue on my rig, which wasn't designed for rendering, is RAM/CPU based.

Now my render is running on 3 computers: 1 host render machine with a 1080TI, 1 Swarm with 3 1080Ti, and 1 Swarm with a 580.

Thank you,
Dainon

[Peter.Chaushev]

Hello, Dainon,

Thank you for contacting us about this problem!
In order to learn why your GPU rig fails shortly after initializing the distributed render process, it would be best to take a look at its render log.
The log can be accessed by opening a browser and typing in the IP address of your rig followed by":24267/log", e.g. http://10.01.23.45:24267/log .
When you open this page, first, please click the ⋮ button at the top-right corner and choose "Clear browser log". Then, do a render attempt and wait until the issue occurs.
Once done, please share the log with us for further investigation either here or via https://www.chaosgroup.com/help/cont...hnical#options (make sure to mention this forum thread).

A couple of side notes:
1. The "100%" swarm utilization you see in the Asset Editor mean that 100% of the machines set to participate in the render job are engaged. If you had 3 additional machines and 2 of them are participating, the percentage would be %66. In your case it is 1 out of 1, meaning 100%.
2. If your scene uses heavy asset files (large textures or proxy files) it may take a while to transfer them to your rig. In this case, the wi-fi connection may affect the time it takes your rig to join the render job.

Kind regards,
Peter

[dainon_woudstra]

Peter,

Sure. Here's the log from the error. The error are shown here, but the attachment includes the entire log. If this gives you clues to test, I will happily try any tests or potential fixes.

Thank you,
Dainon

... See attached error.txt for full output
[2021/Jan/15|13:08:40] [2530 MB] error: Failed to allocate 262140 bytes, device 1, buffer default bufname
[2021/Jan/15|13:08:40] [2530 MB] error: Failed to allocate 4 bytes, device 1, buffer default bufname
[2021/Jan/15|13:08:40] [2530 MB] error: Failed to allocate 262140 bytes, device 0, buffer default bufname
Could not allocate 53687040 bytes of GPU memory
Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
[2021/Jan/15|13:08:40] [2530 MB] error: Failed to allocate 4 bytes, device 0, buffer default bufname
[2021/Jan/15|13:08:40] [2530 MB] error: Failed to allocate 57042480 bytes, device 2, buffer default bufname
[2021/Jan/15|13:08:40] [2530 MB] error: Failed to allocate 139810 bytes, device 2, buffer default bufname
Could not allocate 53687040 bytes of GPU memory
Could not allocate 53687040 bytes of GPU memory
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 57042480 bytes, device 4, buffer default bufname
Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
Could not allocate 53687040 bytes of GPU memory
Could not allocate 53687040 bytes of GPU memory
Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 57042480 bytes, device 4, buffer default bufname
Could not allocate 53687040 bytes of GPU memory
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 139810 bytes, device 4, buffer default bufname
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 139810 bytes, device 4, buffer default bufname
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 139810 bytes, device 3, buffer default bufname
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 57042480 bytes, device 4, buffer default bufname
Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
Could not allocate 53687040 bytes of GPU memory
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 139810 bytes, device 3, buffer default bufname
[2021/Jan/15|13:08:40] [2711 MB] error: Failed to allocate 139810 bytes, device 4, buffer default bufname
Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
[2021/Jan/15|13:08:40] [2932 MB] Could allocate 53687040 bytes of non-paged host memory; will use regular buffer
Could not allocate 53687040 bytes of GPU memory
V-Ray GPU kernel render time: 1.51s
... See attached error.txt for full output
[2021/Jan/15|13:08:40] [1948 MB] error: [MemoryManagerGpu::releaseUnmappedMemoryFromInstanc e] 2: Could not release device buffer
[2021/Jan/15|13:08:40] [1948 MB] error: [MemoryManagerGpu::releaseUnmappedMemoryFromInstanc e] 2: Could not release device buffer
[2021/Jan/15|13:08:40] [1948 MB] error: [MemoryManagerGpu::releaseUnmappedMemoryFromInstanc e] 2: Could not release device buffer
[2021/Jan/15|13:08:40] [1948 MB] error: [MemoryManagerGpu::releaseUnmappedMemoryFromInstanc e] 2: Could not release device buffer
[2021/Jan/15|13:08:40] [1948 MB] error: [MemoryManagerGpu::releaseUnmappedMemoryFromInstanc e] 2: Could not release device buffer

[Peter.Chaushev]

Hello again, Damion,

Apologies for the severe delay!
After conducting some investigation I was not able to find out the cause of your problem based on the current info although some similarities with another case were established.
To get to the bottom of this could you please provide the scene in question (or any other that exhibits the issue) via a file-sharing service of your choice?
Also, please let us know if you are willing to accept assistance in a more direct manner from our support team.

Kind regards,
Peter

[akos_seres]

Could you go into a brief detail of how you use your rig to mine with? We have a render farm and would like to start mining with it when we are not rendering. Any information is good information!

[can_soyaslan]

Hello Dainon,

I am looking for a solution to upgrade our render quality and efficiency for a while. Using a render farm was one option but I couldn't find any reference yet. Thank you for sharing your experience.

I wonder if you can share some more information about the configuration with V-ray. We are using V-ray for rhino.

Best Regards,
Can