Phoenix FD elements not rendering on AWS / Deadline cloud farm

[jeffbrown]

I am somewhat new to PhoenixFD, and trying to get it working with an Amazon AWS renderfarm running on Deadline (vers 10.0.23.4). I have tried several combinations of server Images to render with; all seem to work except for PhoenixFD elements. The renders do not fail, they just do not render any Phoenix elements (fire, liquid). Here's what I know so far:
I am using UNC paths for all assets, including the Output and Input of *.AUR files.
The logs indicate that the *.AUR cache files are being transferred via Deadline Asset Server.
My most recent AMI (Amazaon Machine Image) is a custom image built from Deadline (Windows) base, using Max 2019 sp2, VRay Next 4.1, and PhoenixFD 3.1.200
Max/VRay projects without Phoenix render fine.
Local (not AWS) Deadline renders with Phoenix work fine on my farmette of 5 machines.
Any suggestions or request welcomed, thanks.

[Svetlin.Nikolov]

Hey,

What if you load the Phoenix caches in a V-Ray volume grid and transfer the render settings by saving a Phoenix render preset and loading it into the volume grid? Also, what if you try to setup a Phoenix render that does not require a cache, such as this setup: https://www.youtube.com/watch?v=0lznYojEZok

This will help narrow down the issue.

Also, if you have access to the Phoenix log file, it would also help shed light on the issue.

Cheers!

[jeffbrown]

Thank you, Svetlin. I will try those approaches as soon as I can find time. I will also try to get a Phoenix log from the current setup.

[jeffbrown]

Hello again,
V-Ray volume grid does not seem to help; still no geometry rendered up in cloud-land. The non-cache render does work, but seems to render with inconsistent lighting. I have attached logs and screengrabs from the tests. Please let me know if there are other items that might be useful. Thanks for the support!

[Svetlin.Nikolov]

Hey,

Can you tell me more about the circumstances under which Phoenix logs 1, 2, and 3 were generated? Seems like the mesh cannot be built in 1 and 2 and builds okay in 3...

Cheers!

[Svetlin.Nikolov]

From the logs it seems that Phoenix cannot find the caches anyway though. Something that comes to mind when looking at the assert server log is for you to try and change the backslashes \ in your cache path to forward slashes / and see if this would help...

[jeffbrown]

If I labeled them correctly, Log-1 and log-2 were from rendering the liquid sim in a VR Volume. Log-3 is from rendering the 3D Texture-to-mesh.
I tried changing the backslash to forward slash, that seems to have no effect (still "blank" renders). I also tried explicitly adding the "copy external files (seq.) to Repository" in Deadline submitter, along with "pre-cache on AWS", and that did not help, either.

[jeffbrown]

and... a reality check: I added an element to the scene that used a file sequence texture map from the same drive as the AUR cache files. Those images got read and rendered OK on the remote network.

[Svetlin.Nikolov]

Hmmm, I'm afraid I am out of ideas. There seems to be something going wrong with the automation that transfers the data, but not sure what exactly - according to the logs Phoenix does not read the cache files at all. Have you already tried contacting the AWS guys?

Meanwhile, we are trying to get our license running so we can try and reproduce the issue here, but seems like it's gonna take some time...

[jeffbrown]

Ah, well. I will follow up with the Deadline group, and see if they have any ideas. Thanks for looking into it, and let me know if you are able to replicate the issue when you can. Thanks!

[jeffbrown]

Update: So far, still not working. Here is a link to the discussion and results on the Deadline forum, if you can take a look:
https://forums.thinkboxsoftware.com/...no-joy/24474/4

[Svetlin.Nikolov]

Ahhhh, I see what's up - between Phoenix 3.11 and Phoenix 3.12 the default cache path got changes from "$(scene_dir)" to "$(scene_path)". Seems like Deadline does not recognize this path.

However, this does not answer the question why even UNC paths are not working ...

[jeffbrown]

Yes, it is strange. I use the same share (//workstation/mediaDrive/...) for image sequences in and out, and Deadline works with those. And I believe the cache files get transferred to AWS, they just do not get read for the render.