The dreaded black buckets

[Bertjenkins]

Hi

Could someone please clarify the best way to check the assets being transferred with DR.
I'm getting two different renders when i click 'use local machine' on and off. So some assets are not beong transfrred properly. Hence i'm getting black squares in my GI channel. They are not hard squares but soft ones.
Is there a way to 'flush out the cache' so to speak and make sure all machines are using the correct maps?
Also I was under the impression that DR did not have an effect on Light Cache passes. Yet when i use a light cache map from file, this seems to increase the black square problem. It goes when i return to single frame. Why should this affect anything?

I can get rid of them eventually with trail and error but i really want to understand what causes them so that i can eliminate them early on in future. They have wasted a hell of a lot of my life. I thought these black bucket nightmares were a thing of the past!!

I'm using build 23208 osx


thanks in advance

[t.petrov]

Can you make a simple scene that reproduces the problem?
Have you inspected the logs from all renderslaves to see if some of the files has not been read correctly or if there aren't other errors in the logs?

[Bertjenkins]

Hard to recreate the scene as it seems to happen pretty randomnly.

All the bitmap files have loaded fine. Sometimes one frame works again when i turn DR off, then render, then turn it back on again. But as soon as i render a sequence, the gi frames cause a horrible noise from the different tones in certain buckets.

The only warning is: environment variable vray assets cache path is not specified... but i've had it say this too when it's working fine? I can never get the assets cache path env variable to work on OSX so i usually give up. Coudl this be the problem?

[Adbk]

We have experienced similar issues; one of our network drives was *probably* not fast enough to pull the HDRI's from the drive on and they would not load. We simply put them on a different drive. As stated, if you haven't checked the log already maybe start there and see if it is specific textures in specific locations not loading.
good luck

[Bertjenkins]

Just checked and the main dome light hdr is not listed. However there is no warning.
Did you do this via the assets cache path? Or just copy the sourceimages folder to a server?

[Bertjenkins]

I just increased the light cache subdivisions and now it works fine. As i said earlier, i don't understand why light cache should affect anything? Unless it's not a DR issue at all? But the noise looks extremely square and bucket sized. Have i missed something?

[hsz]

Where Can I find the log of Vray on OSX ??? Thank You

[tashko.zashev]

Where Can I find the log of Vray on OSX ??? Thank You

Hi,

vray4maya_log.txt file could be accessed from the OSX terminal :
cd /tmp

Also you can check the Maya render output file here:

/users/account_name/library/logs/maya - mayaRender.log

[Bertjenkins]

Hi

Is there an issue with build 23208 and Irradiance map? It's giving me some very erratic results. I'm using 2013 and snow leapoard.
Any tips on a good stable build from the recent nightlies?

Also the problems below have not really been resolved... ok i can use huge render settings and it works, but i cant see why the problem happens in the first place? Why should there be dark bucket sized areas in the GI pass which go when i turn off DR?

[Bertjenkins]

Just to outline Irradiance Cache the problem: I have meshes built with smooth mesh preview. In some scenes they render fine, in other ones the same mesh renders with hideous spots in the GI pass. When you smooth the meshes the spots go. But i don't want to smooth the meshes.

I believe this problem has only started since installing build 23208 osx

[Svetlozar Draganov]

I would like to summarize that all those problems are very unusual and it is very difficult for us to guess what is going on and how could we help you without having those issues reproduced.

Starting back to front of this thread you have mentioned that the issue have occurred with the last Nightly Build installation, so the easiest way to find out if this is the root of the issue is to switch back to the previously used Vray version and run the same tests looking for same result. If the issues do not persist with the previous build then we can conclude that there is something wrong with the Nightly build.
Please have in mind that all Nightly Builds are not stable versions - so such unexpected results could happen.

Just to outline Irradiance Cache the problem: I have meshes built with smooth mesh preview. In some scenes they render fine, in other ones the same mesh renders with hideous spots in the GI pass. When you smooth the meshes the spots go. But i don't want to smooth the meshes.

What these meshes represents ? Are they created with Maya or any other software ? Can you show us example of those meshes or send one of them ?
Is the issue focused only on them or it appears on other places in the end result as well ? If the bug is not only on them can you spot if it is related to some materials , textures or any other geometry ?
Have you tried to subdivide it by adding Vray Subdivision and Displacement attribute and to avoid Render Viewport Subdivisions ?

Hi
Is there an issue with build 23208 and Irradiance map? It's giving me some very erratic results. I'm using 2013 and snow leapoard.
Any tips on a good stable build from the recent nightlies?
Also the problems below have not really been resolved... ok i can use huge render settings and it works, but i cant see why the problem happens in the first place? Why should there be dark bucket sized areas in the GI pass which go when i turn off DR?

Have you tried any other build ? Newer , older or stable one ? Unfortunately there aren't any tips about those builds , they are work in progress versions and they aren't tested extensively in order to determine which one is stable and which don't.
Can you show us with what settings the bugs appear and what are the high-settings that resolves it ?
I am a bit confused about you last sentence in this post - the buckets appears when DR is OFF or ON ?
What GI engines are you using ? Is the GI pre-calculated or it is calculated at each frame ? Are you rendering animation or still frames ?
Do you machines have enough RAM in order to render that scene ?
This is very important one - what type of Default Geometry are you using in System Rollout - it is a very good idea to run a separate tests with Dynamic and Static geometry - recently we have a very similar issue which was solved by using Static geometry instead of Auto.

How many slave machines are you using ? Only some of those machines have this bug or it happens on all of them ? Have you got the same Vray version installed on all machines ?
What about slave specification - are the slave with the same power or some of them are weaker ? Are all of the machines have the same operating system , Maya version and Vray version ?
Have you tried to run the test only on a single slave machine ?

How heavy is your scene ? Are you using a lot of textures and etc ? How many GB is the entire project , including all assets and scenes ?
From where all the slaves load your project ? Is it copied locally on each machine or it is loaded though server ? Are you sure that everything is accessible for all of the machines ?
Does the issue persist without GI ?

I just increased the light cache subdivisions and now it works fine. As i said earlier, i don't understand why light cache should affect anything? Unless it's not a DR issue at all? But the noise looks extremely square and bucket sized. Have i missed something?

Again - what are the bad-settings and what are the good ones ? Can you show is the before and after ? What happens if you are using pre-calculated LC ?
What value do you have in Number of Passes option and does this number match the cores of all workstation and slave machines ?
Does the issue persist if LC is OFF ?

Hard to recreate the scene as it seems to happen pretty randomnly.
All the bitmap files have loaded fine. Sometimes one frame works again when i turn DR off, then render, then turn it back on again. But as soon as i render a sequence, the gi frames cause a horrible noise from the different tones in certain buckets.
The only warning is: environment variable vray assets cache path is not specified... but i've had it say this too when it's working fine? I can never get the assets cache path env variable to work on OSX so i usually give up. Coudl this be the problem?

Is the issue persist only when you are rendering sequence ? Are you using batch-mode ?

Hi
Could someone please clarify the best way to check the assets being transferred with DR.
I'm getting two different renders when i click 'use local machine' on and off. So some assets are not beong transfrred properly. Hence i'm getting black squares in my GI channel. They are not hard squares but soft ones.
Is there a way to 'flush out the cache' so to speak and make sure all machines are using the correct maps?
Also I was under the impression that DR did not have an effect on Light Cache passes. Yet when i use a light cache map from file, this seems to increase the black square problem. It goes when i return to single frame. Why should this affect anything?

I can get rid of them eventually with trail and error but i really want to understand what causes them so that i can eliminate them early on in future. They have wasted a hell of a lot of my life. I thought these black bucket nightmares were a thing of the past!!

I am not sure if such way exist but what I do in such cases is to delete everything from DRTemp folder and run the render again - once everything is removed then it will be transfered again and Vray will transfer all missing assets. I use the following method when I want to make sure that everything works correctly - from the workstation machine I copy the entire project into a single directory which is not accessible for the slaves, then I run a DR with transfer missing maps ON and after that I compare the last directory with DRTemp directory at each slave - if all directories contains all the files then everything works fine.
Make sure that all files are updated to their last version - is it possible to have some of them copied to the slave machines but like earlier version, so this could cause different renders between Use Local Machine On and Off.

There is no way to flush out the cache but you can manually delete everything from DRTemp dir and copy them like I mention earlier.
LC is not distributable - it is calculated on each machine and is some cases it is better to pre-calculate it and load it from file. In general loading LC from file should be more stable than calculating it at each frame but in your case for some reason the opposite is true - so there must be something wrong with loading information for disks.

I am sorry about a hundred of questions that I asked but since this case is very odd and random we are not able to give you a straight forward receipt to solve the issue - however it will be a good idea to update us about the results you have by trying some of the tips we share here.

[Bertjenkins]

Svetlozar

Thank you so much for taking the time to send that answer- it's exactly what i needed!

As for the nightly build, yes i'm aware that they are not necessarily stable, but was trying to 'shortcut' my problem by seeing if anyone else had experienced anything similar, before re-installing on all machines.

I tried using the vray subdivision and displacement attribute and manually setting the subdivision quality for specific objects and this was a huge improvement. Reducing the max subdivisions from 256 to 16 has speeded up my renders also. I will do it this way every time in future as it's much more predictable.

I checked the builds and yes, one machine had a different build to the others, and so far this seems to have helped. So far i believe this to be the cause of the black buckets- but i'll wait and see if they are gone for good.

The black buckets were not appearing on a local machine render but were appearing when DR was on. I can't seem to re-enact the initial problem, so i'm hoping that it was the machine with a previous build that was somehow causing the issue.

I'll try the static geometry setting, that sounds just what i need- can you set it to dynamic on a per object basis if there is only one object moving in the scene? Anyway it's one more thing to troubleshoot so very useful.

Also i've now ticked 'use camera path' and that has made a big difference for me in reducing inter frame flicker. I had thought this was only for flythroughs previously. Works great with animated cameras etc.

The scene is 250 mb and the assets are under 1 GB. The issue is only a GI thing.

The number of passes is based on the most powerful machine (24) - can i add the other cores too? Will it use all the cores in the LC? If so there are 32 more virtual cores. So i could up this to 56?

The black bucket thing happened both in batch mode and in the VFB.

The DRTemp trick is just what i was looking for- just something i can do to double check that the assets are all transferring accross properly, so i can be sure this is not the problem.

Thanks for the detailed reply- Hopefully those darn black buckets will never return again!!

[Svetlozar Draganov]

You could control viewport subdivisions also by Default Displacement and Subdivision Tab into Settings rollout.
By default they are too much so it is a good idea to change Max Subdivs to something like 4 , 8 , 16 .....

The slave machine with different Vray build could be the root of the issue - please have in mind that it is necessary to use the same Vray build across all machines with DR.

Use camera path option is designed to produce smoother and less flicker animations - so it is a good idea to get used with it

LC passes should be equal to the number of the cores installed on the machine - if this number is lower Vray will use less cores to compute LC and the calculation will be slower, this is even true if the number is greater than the cores but the different won't be as if less cores are used. The best option is to set it to 0 and Vray will automatically get the number of the cores and it will use that number to calculate LC.
Please have in mind that 0-value works only in Nightly Builds - I am not sure if this is possible with the last stable build.

[bigbossfr]

LC passes should be equal to the number of the cores installed on the machine - if this number is lower Vray will use less cores to compute LC and the calculation will be slower, this is even true if the number is greater than the cores but the different won't be as if less cores are used. The best option is to set it to 0 and Vray will automatically get the number of the cores and it will use that number to calculate LC.
Please have in mind that 0-value works only in Nightly Builds - I am not sure if this is possible with the last stable build.

Hi,
If we set a value of 0 for the LC, and if we have some computers which haven't the same number of core : we can have some flickering ? I think yes.

[Bertjenkins]

Thanks.

My main machine is 24 (hyperthreaded) cores. The other two machines are 16 cores. Should my light cache passes be 24 , 56, or 0? Currently 24 is causing no flickering, but is quite slow.