RenderFarm File Loading issue

[freed-kenneth]

HI All,

We are switching from 3dsmax to Houdini recently.
There is an issue that our render farm machines start rendering even some files are not loaded, leading to randomly missing models/textures.

A little background of our setup:
Workstations and render farm are using Houdini 19.0 and Vray 5
The rendering management software is Deadline.
We store models (mostly Alembic file) and textures (exr udim) on two separate local-network machines, and the network speed is not very fast.
Vray for 3dsmax has no such problem at all.

I would like to know if there is some kind of settings we can set to prevent the problem.

Thank you for your time.


edit 1:
I take a look of Deadline logging. When the files are missing, there will be a warning message:
V-Ray: Bitmap file "\\xxx/xxx/xxx.exr" failed to load: cannot open file:[LoadImageFile] 1: Failed to open file: no such file

Is there anyway to stop the renderer if missing files, or to reload the file?

[bdancer]

> V-Ray: Bitmap file "\\xxx/xxx/xxx.exr" failed to load: cannot open file:[LoadImageFile] 1: Failed to open file: no such file

So, is this location reachable from the render node?

> Is there anyway to stop the renderer if missing files

ROP -> Renderer -> Options -> Rendering -> Abort Rendering On Missing Asset

> or to reload the file?

No.

[freed-kenneth]

So, is this location reachable from the render node?

Yes, all the path are reachable. And I noticed that It's very likely to happen if too many machines are trying to render at the same time.

ROP -> Renderer -> Options -> Rendering -> Abort Rendering On Missing Asset

Thank you, this might be helpful.

[freed-kenneth]

ROP -> Renderer -> Options -> Rendering -> Abort Rendering On Missing Asset

After few tests, I can confirm texture missing doesn't count as missing asset.

[bdancer]

> After few tests, I can confirm texture missing doesn't count as missing asset.

It should count. Are we talking UDIM or file names with tags/frame number?
You could also try setting VRAY_UNC_TIMEOUT env. var. on your render nodes to some higher value then 5000 (timeout in milliseconds).

[freed-kenneth]

It should count. Are we talking UDIM or file names with tags/frame number?

Yes you're right, it's UDIM textures with tags. And only one or few frames of UDIM missing when happening.

You could also try setting VRAY_UNC_TIMEOUT env. var. on your render nodes to some higher value then 5000 (timeout in milliseconds).

OK, I will try it.

[Dubbie99]

We are just experiencing this issue now.
Exact same symptoms as OP.

[freed-kenneth]

Unfortunately we're not able to solve the problem, I hope this thread can raise the awareness for the developers.

For those who might not understand, I attached this video to illustrate the issue better.
18 frames of UDIM projected on a plane.

[Dubbie99]

For us the environment variable seems to have fixed the issue. I set it to 10 sec instead of 5.
Since doing that we haven't seen the issue again.
I don't know why it would be timing out - our NAS is usually extremely fast. Especially cached files are served up in milliseconds.

[freed-kenneth]

For us the environment variable seems to have fixed the issue. I set it to 10 sec instead of 5.
Since doing that we haven't seen the issue again.
I don't know why it would be timing out - our NAS is usually extremely fast. Especially cached files are served up in milliseconds.

Thanks for posting that!
I'll definitely check again.

[Dubbie99]

Just another data point on this issue.
Checking on "Low priority thread" also seems to solve this problem.

[GoodbyeKansas]

bdancer could this help on linux as well? We got the sporadic error where textures randomly doesnt load for some objects in the scenes on random frames

[bdancer]

> could this help on linux as well?

No, timeout is Windows only.

[juleslin]

May I know if there's any update about this issues? This really affect the the production speed and i am thinking if Vray is not ready for proper VFX production....

[Gosho.Genchev]

Hey juleslin ,

Could you please take a look at the info in the Red warning box here: https://docs.chaos.com/display/VRAYHOUDINI/IPR+Tab
Hopefully it's the same issue.

Best regards!