Announcement

Collapse
No announcement yet.

Question on region rendering

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Question on region rendering

    Hi,

    For large stills, we always distribute the EXR render onto multiple machines by defining regions. We also output each render element separately. This results in a lot of files.

    Instead of having artists manually merge all regions for different render elements, I wrote a script which does this as a post-render task on our farm. However, I've noticed that some render elements have an alpha, some render elements do not have an alpha, some render elements come with color values around the region being rendered etc. So all render elements are not outputted on the same format. This means I have to perform operations on a per-render element level before doing the actual merging of regions. Or to be honest, I have a number of conditions and all other scenarios are treated in a "default" manner.

    So, now when we'll soon be able to output a Cryptomatte render element, it's likely that I have to amend my code to support this. At least I can't just assume it'll all just work.

    Question: I would love to avoid having to update code because a new render element is available due to a V-Ray update. What are my options here, would you say?
    Unfortunately, we cannot use DR as our render farm uses runs jobs with different V-Ray builds simultaneously as well as queues up non-V-Ray jobs which need to be prioritized together with V-Ray jobs.

    I wouldn't mind rendering to .vrimg and then have a Chaosgroup-developed tool to merge these .vrimg files. And then I'd leave the render element compatibility up to you guys Perhaps this is already possible, using e.g. vrimg2exr?

    Also, there's openimageio (OIIO) which has a built in merge tool. The problem here is that I've had issues with this in the past. The problem was that V-Ray didn't properly register the region coordinates vs the canvas coordinates of an EXR, so the merge tool wouldn't know where in the buffer the region exists and it would cause the merge tool to merge the entire canvas (just overwriting the entire buffer). I'm not sure if this behavior was ever changed with how V-Ray and V-Ray for Maya writes EXRs?

    In short; what would your suggestion be?
    Last edited by Fredrik Averpil; 22-09-2017, 09:46 AM.
    Best Regards,
    Fredrik

  • #2
    Try enabling "Auto data window" in the V-Ray Image Format Options for EXR. This might provide just enough information for the OIIO merge tool to work correctly.
    V-Ray for Maya dev team lead

    Comment


    • #3
      Hi, I just wanted to try this "Auto data window" out. I'm using V-Ray 3.5 build 27650 for Maya 2017 Update 4 and I'm getting some really strange results. See my screenshots.

      So I don't think I can use "auto data window" when region rendering. It's more when you render ONE single image... but I still can't explain why the last region has a lot of black around it...

      Is there no way I can define the data window size myself?
      These are my render commands:

      Code:
      Render.exe -r vray -x 4000 -y 2000 -reg 0 1414 0 707 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice1_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 1415 2829 0 707 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice2_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 2830 3999 0 707 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice3_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 0 1414 708 1415 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice4_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 1415 2829 708 1415 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice5_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 2830 3999 708 1415 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice6_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 0 1414 1416 1999 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice7_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 1415 2829 1416 1999 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice8_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb
      Render.exe -r vray -x 4000 -y 2000 -reg 2830 3999 1416 1999 -s 1 -e 1 -rl masterLayer -cam camera1 -proj //FILESERVER/prod/proj2/maya -rd //FILESERVER/prod/proj2/maya/images/scene63/slices -im <Scene>_slice9_<Layer>_<Camera>. //FILESERVER/prod/proj2/maya/scenes/scene63.mb

      Here's the Maya 2017 Update 4 binary scene file: https://drive.google.com/open?id=0B0b1IUnYqENgWk5OdTVVRHpDODQ
      Here's a zip of the region renders: https://drive.google.com/open?id=0B0...nBJWktlMUhnQTQ
      Last edited by Fredrik Averpil; 01-09-2017, 12:09 AM.
      Best Regards,
      Fredrik

      Comment


      • #4
        Unless I have misconfigured my V-Ray Sky, I really think that there's a bug here. The Nuke-merged image should not have those black holes in it. It seems like V-Ray doesn't recognize the V-Ray sky (and/or the V-Ray plane which I'm using as ground, perhaps?) as a proper object and excludes it from the data window...?
        Best Regards,
        Fredrik

        Comment


        • #5
          In the V-Ray EXR file format options, the data window is only supported for multichannel EXR images. Would it possible to add this functionality also for non-multichannel (regular) EXR images?

          The data window feature is crucial for post-render batch conversions which we do here. Currently, some render elements are inproperly merged when e.g. region rendering due to them not having a proper data window defined.
          Best Regards,
          Fredrik

          Comment


          • #6
            Indeed, the auto data window is only for multichannel exr files. This has been requested by other users and it's on our to-do list already. I've added your request too, so when we have this done I can update this thread.
            Alex Yolov
            Product Manager
            V-Ray for Maya, Chaos Player
            www.chaos.com

            Comment


            • #7
              Ok, many thanks, much appreciated!

              Do you think there could be a bug with the data window and V-Ray sky?
              Best Regards,
              Fredrik

              Comment


              • #8
                Hi,
                I've reproduced the issue with the sky and reported it in our system. It is caused by the "Auto data window" option. We'll get back to you as soon as we have a fix for it.
                Thank you for you feedback
                Ivan Shaykov
                chaos.com

                Comment


                • #9
                  Thanks so much Ivan for investigating.
                  Best Regards,
                  Fredrik

                  Comment


                  • #10
                    Hi again Ivan,

                    I'm very keen on having this bug fixed, as we're really anxious to start using region rendering of multichannel EXR images with data window enabled. Right now it seems this bug is still being hit with 3.60.02 when using a V-Ray sky.

                    Do you think we'll see a fix anytime soon?
                    Best Regards,
                    Fredrik

                    Comment


                    • #11
                      I just realized that this issue will be hit anytime an object is rendered against black alpha. I'll make a completely different thread on what I'm trying to do, as I don't think "auto" data window is the feature I can use.
                      Best Regards,
                      Fredrik

                      Comment


                      • #12
                        Hi Fredrik,

                        I am responsible for fixing the issues regarding EXR (auto) data window. After reading your post, I noticed that there was a general problem, which is - in region rendering, the data window was not saved with single and multi channel EXRs. So that is what I started fixing. The fix will be included in the next VRay for Maya 3.60.03 version.

                        The auto data window option served as a workaround or at least we hoped so. Apparently auto data window does not include any kind of environment, such as the Sky. This is actually the expected behavior, since the auto data window functionality uses the alpha channel and an environment does not affect the alpha. Is there any other reason for which you need the auto data window, besides for the workaround? If not, then once the fix is released, you can turn it off.

                        Regards.

                        Last edited by asparuh.krastev; 31-10-2017, 06:45 AM.

                        Comment


                        • #13
                          Hi!


                          I think my use case falls outside of what the "auto data window" was intended for as my use case is region rendering-centric, let me explain:

                          When we need to render very large still images, we prefer to region render these. Let's say 64 regions as an example. This way we can distribute the rendering onto 64 farm machines.

                          Side note: We don't use DR because we want to efficiently queue and prioritize this with other jobs (possibly running different V-Ray versions) on our farm, which makes this difficult with DR.

                          In order to merge the 64 EXRs together into the final image, we use Nuke. In fact, a script invokes Nuke on the commandline and performs this as a post-render operation. In our current pipeline, we do not render multichannel EXR out of V-Ray. Instead we render non-multichannel EXR out of V-Ray which means each render element is rendered into 64 pieces.

                          Side note: I'm currently adding support for multichannel EXR in our pipeline, but old projects may still use non-multichannel EXR.

                          Historically, some render elements come with an alpha. Some does not. Others have black outside of the rgb region while some others do not. This means you cannot just "merge" all render elements using one single approach. For this reason, we have -over time- built recipes on how to merge regions depending on which type of render element is being merged. So every time you release an update to V-Ray which enables a new render element, we might need to add a new recipe. This is very cumbersome and not ideal.

                          Therefore I've been looking into how to solve this the best way, moving forward. If a region render would have its data window set to the region's coordinates, you would be able to simply merge all regions together in Nuke (using "over"). No special recipes. This would be nice. And this is what I've been requesting and initially thought that the "auto data window" could do. But this feature does not guarantee that the data window of a region has the same coordinates as the region itself. The "auto" in "auto data window" can make the data window smaller than the region itself, which will cause the merge operation in Nuke to produce undesired results.


                          Conclusion: So, what I am looking for, in a nutshell, is to easily merge region renders. If multichannel EXR as well as non-multichannel EXR region renders would have an option to include a data window set to the region's coordinates, this would solve my problem.


                          ---

                          Since this functionality is not possible and since I need to add multichannel EXR support to our pipeline, I'm now looking into how I can solve this without a data window. I already know all the regions and their coordinates and I can save those to e.g. a JSON file. Then inside of Nuke, this file can be read and parsed. I can put a crop node (instead of a data window) after each region's Read node (with the corresponding coordinates as values) and then I can merge. This will work for both non-multichannel EXRs as well as multichannel EXRs and won't require any "render element recipes".

                          However, if you were to implement a feature where region renders would contain a data window equal to region coordinates, I wouldn't have to develop this kind of merge.


                          (sorry for enourmous post)
                          Last edited by Fredrik Averpil; 31-10-2017, 08:27 AM.
                          Best Regards,
                          Fredrik

                          Comment


                          • #14
                            By the way, I wanted to download the latest stable of 3.60.03 but I noticed there were no Linux installers for the build from Oct 30th. Any chance you could build those? (I need both Win and Linux)
                            Best Regards,
                            Fredrik

                            Comment


                            • #15
                              Hi Fredrik,

                              From what I understood, the new fix should be what you need. Multichannel and non-multichannel EXRs will contain the data window information as metadata and auto-data window won't be necessary. The new stable 3.60.03 however is still not released and we don't have an exact release date yet, but it's coming . Until then, if it is possible, you can try to experiment whether the new fix will work for you if you use deep EXR, because it does not have this issue.

                              Regards,
                              Asparuh

                              Comment

                              Working...
                              X