Hqueue issues

[Jim Meston]

Thank you Gosho. One step closer !!

Firstly I get the following errors on scene load.

Load warnings for C:/Users/jmeston/Desktop/forum_1070445_HQSubmit_envVars.hip

Warning:

/out/vray:

Skipping unrecognized parameter "UVWGenChannel_uvw_transform".
Skipping unrecognized parameter "UVWGenChannel_uvw_transform_tex".
Skipping unrecognized parameter "UVWGenChannel_tex_transform".
Skipping unrecognized parameter "reset_current_vfb".
Skipping unrecognized parameter "geom_export_proxy_mesh".

Secondly the quadruple \\\\ seems to make all the difference, as with just 2 it goes back to previous behaviour.
It is trying to render all the frames, and it succeeds on 1 workstation. It fails on the workstation I am launching the job from, the one with GUI houdini open
See attached image


Checking the successful renders all seems well. They look how they should.

I attach also the diagnostics for the fail.
Highlights here...

Warning: Multiple primitive definitions for: VRaySceneRef
14904: Fatal error: Segmentation fault

I am running Houdini 18.0.460 and "vray_adv_43003_houdini18.0.416_18b5bdd_7012.e xe".
It works fine rendering locally (workstation39) and as can be seen it also renders on workstation36. It just isn't rendering on workstation39 through HQueue with vray. (All is well in Mantra world).

I can't imagine trying to install HQ remotely

Thanks for your help on this, I really need to get some renders going. Feeling the pressure from upstairs!

Best,
Jim

[Jim Meston]

Failed render diags

[Jim Meston]

I wonder if it is a network issue, as it seems after double checking that Mantra is very slow to send jobs from HQserver to workstation39.
You can see the behaviour in the attached screen caps.

[Gosho.Genchev]

Hey, Jim,

1) The "Skipping unrecognized parameter..." warnings are fine if they only happen for a few nodes in your scene - it means we either added or removed a parameter to some V-Ray node.

2) Quadruple "" is because you need to escape each slash - the '\' is usually an escape symbol. Thus, you need the path to start with '\\', so you use '\\\\'

3)

It is trying to render all the frames, and it succeeds on 1 workstation. It fails on the workstation I am launching the job from, the one with GUI houdini open
...
Warning: Multiple primitive definitions for: VRaySceneRef
14904: Fatal error: Segmentation fault
...
It just isn't rendering on workstation39 through HQueue with vray.

This means that for whatever reason, V-Ray is loaded twice on workstation39. HQ uses Hython in the background - the environment variables you set up on the HQueue ROP are passed to the script and Hython is started. Is there a houdini.env file in User/Documents/houdini18.0 , or a vray_for_houdini.json file in (Houdini_install_dir)/packages on workstation39 ?

Best regards!

[Jim Meston]

Yes. This workstation39 is my workstation where I am firing the jobs from an open instance of GUI Houdini (this machine is also setup as a HQueue Client). In the houdini install there is a json package file file for vray. obviously as I use Houdini GUI on this machine.

[Gosho.Genchev]

Hey, Jim Meston ,

surprisingly, it seems that the variables used to load V-Ray are preserved when using the package json file but discarded when using an environment file in documents/houdini. Workstation39 fails because V-Ray is loaded twice.

Therefore, you don't even need the environment variables setup on the HQueue ROP - can you try submitting without them ? That's assuming that the other machine also has V-Ray installed through the installer, with a vray_for_houdini.json file in $HFS/packages.

Best regards!

[Jim Meston]

Will do ASAP. Thanks!

[Jim Meston]

Ok so I am getting some odd behaviour. I just want to go over my setup a little...

I ensured Vray for Houdini was installed (json package file etc) on both client machines.
So I both machines have Houdini and Vray installed with typical settings.

Our Vray licence server is on workstation9
I currently have 2 x Vray for Houdini clients, worstation36 and workstation39.
My gui Houdini is also workstation39.

This is the working licence configuration (working on workstation39)




These are the Vray licences we have for Max (x19) - (see better res image in attachment)


These are the licences we have for Houdini (x1) - (see better res image in attachment)



I am concerned that there is a licence error, though we were told that as we have enough licences.
As I just rendered a job and it rendered frames on workstation39 correctly and pretended to render on workstation36 but the output was black frames. (see image below)



So before with your test scene it was rendering on workstation36 and failing on workstation39.
Now since installing Vray for houdini on workstation36 it is failing on that machine with my own scene.
This might be scene specific. So I will do some tests directly after I have posted this.

AND!!!!

Your scene that you uploaded just fails and won't render anything, unless I remove the env vars from the HQ rop.
Then it renders all frames on all machines!!
Again I am going to do a couple of tests to see if something breaks with my own scene.

[Jim Meston]

Ok to add.

It turns out that it is rendering on all workstations, without the env vars.

BUT !

When I put an HDRI on a dome light, workstation36 can't find the HDRI even though it has been copied to the shared folder. That's why it looked like it was rendering black on that machine.
Shame there is no failure on that job with a warning about assets...

I suspect it is the classic $HIP $JOB stuff.

A little further exploration on this. I will report back.

[Gosho.Genchev]

Your scene that you uploaded just fails and won't render anything, unless I remove the env vars from the HQ rop.
Then it renders all frames on all machines!!

That is expected. I'll try to explain but it will be a long read so I apologize in advance

Plugins in Houdini are loaded when you start Houdini. If you want to load a plugin for your Houdini session, you need to tell it where to find that plugin.

This can be done in one of 3 (major) ways:

1. You can start the Houdini executable through some sort of shell script or bat file
- if you do this, you need to set the environment variables inside the bat file/shell script
- when those variables are inside the file, any process launched with file will inherit them

SET MY_VAR=C:\
start /d "C:\Program Files\Side Effects Software\Houdini 18.0.460\bin" houdinifx.exe
PAUSE

- if you
1) add a new text file,
2) place the text above and
3) rename the file to test.bat
4) double-click it
this will launch Houdini and the Houdini process will inherit the MY_VAR variable

- you can double check that this is indeed the case by opening a Python Shell tab in the Houdini UI and typing:

import os
print os.environ['MY_VAR']

This is essentially how everything works - you need those variables telling Houdini where to find V-Ray to be present in your Houdini session


2. The second way to do this is by using the Users/Documents/houdini18.0/houdini.env file:
- all variables placed in that file are inherited exactly as described above when you start Houdini
- if you start Houdini from a bat file that lists the same variables as the environment file, Houdini will attempt to load the V-Ray libraries twice - this will cause a crash
- variables in the environment file are also inherited by Hython

3. The third way to do this is by using the json file in $HFS/packages
- exactly as above, everything set in that file is inherited when you start Houdini
- exactly as above, setting the same stuff in multiple places will cause a crash
- exactly as above, variables in the json file are also inherited by Hython


Now, what is Hython ?
- this is an executable found in "C:\Program Files\Side Effects Software\Houdini 18.0.460\bin\hython.exe"
- it's essentially regular Python + all the Houdini python libraries: https://www.sidefx.com/docs/houdini/hom/hou/index.html
- it allows you to write a python script and launch it without starting a Houdini UI session
- you might want to use it because it consumes an Engine licenses rather than an FX license
- for example:
1) add a new text file
2) rename it to testHython.py
3) open it in a text editor and add the following

# MAKE SURE to use forward slashes for the path
pathToFile="C:/Users/Gosho/Desktop/myTestFile.hip"

# load the file
hou.hipFile.load(pathToFile)

# get the nodes in /out
rops = hou.node('/out').children()

# print the ROP nodes' names
for rop in rops:
print rop.type().name()

# if a V-Ray Renderer node, render it
for rop in rops:
if rop.type().name() == 'vray_renderer':
rop.render()

4) Open a Command Prompt and call hython.exe with the file you created:

"C:\Program Files\Side Effects Software\Houdini 18.0.460\bin\hython.exe" "C:\Users\Gosho\Desktop\testHython.py"

This is what both the Deadline submitter and the HQueue submitter do - they call Hython with your file in the background.

What's the issue with HQueue:
1) when you submit a hip file, it calls Hython but it also wipes away the environment variables from your session
2) therefore, you need a way to tell it where to load V-Ray from - that's what all those variables on the HQueue ROP were present for - to load V-Ray for the Hython call generated by HQueue
3) Now, it appears that variables placed in the JSON file are not wiped away so you don't need to set them on the HQueue ROP - that's great news - you don't need to worry about setting them up on the ROP



I think that covers everything that's going on with render farms and evrionment variables. I feel that it's useful info you should be aware of.

So before with your test scene it was rendering on workstation36 and failing on workstation39.
Now since installing Vray for houdini on workstation36 it is failing on that machine with my own scene.
This might be scene specific. So I will do some tests directly after I have posted this.

It would be great if you could share that scene.

Best regards!

[Jim Meston]

Catch 22

If I set $JOB I can send the project but it can't find the $JOB/images as they are set on my local machine where the HIP file is.
If I set the texture path to $HIP it shits it's pants and won't copy the files for the job.

[bdancer]

Catch 22 at workstation36 only? Or everywhere?

[Jim Meston]

The issue wasn't with workstation36. It just looked like it as it couldn't find the HDRI light. So everything was rendering black on that machine. But it rendered without an error.

The issue I am now having is that with my texture paths set to $HIP I can't get the submitter to copy the files over.
When it set to $JOB it will copy the files over but the $JOB is not set to the new job folder that it has copied the files to.

[bdancer]

> The issue I am now having is that with my texture paths set to $HIP I can't get the submitter to copy the files over.

I'm not sure how that could be related to VFH then...

[Jim Meston]

Ok so ... I can get everything working if I copy all the assets to the shared folder manually, save the hip to the shared folder and then save the project and then submit it. So that is licencing worries out of the picture!!
This is great news!!!!!

But can you recommend a way to get it working with "Copy Project Files to Shared Folder and Render" then this would be a success.
Having to go and find assets which are all over the place, with relative references, and some local references is a layer of labour I would like to avoid bigtime.