Announcement

Collapse
No announcement yet.

Windows 10 completes render but hangs at exit

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Windows 10 completes render but hangs at exit

    Hi,

    We're in the process of rolling out Windows 10 64-bit on our workstations and I've been noticing something strange.

    Upon completing a V-Ray for Maya render (mayabatch/commandline), the image files are written properly to disk but the mayabatch process hangs and won't exit. Here's where the log ends:

    Code:
    V-Ray: Starting render
    [2016/Mar/1|02:19:32] V-Ray: Exporting scene to V-Ray.
    [2016/Mar/1|02:19:32] V-Ray: Parsing light links time  0h  0m  0.0s (0.0 s)
    [2016/Mar/1|02:19:32] V-Ray: Translating scene geometry for V-Ray
    [2016/Mar/1|02:19:32] V-Ray: Total time translating scene for V-Ray  0h  0m  0.1s (0.1 s)
    [2016/Mar/1|02:19:32] V-Ray: Rendering.
    [2016/Mar/1|02:19:32] V-Ray: Rendering frames.
    [2016/Mar/1|02:19:32] V-Ray: Updating frame at time 1.00
    [2016/Mar/1|02:19:32] V-Ray:  (Estimated time remaining:  0h  0m  0.1s): 54 % completed
    [2016/Mar/1|02:19:32] V-Ray:  (Estimated time remaining:  0h  0m  0.1s): 57 % completed
    [2016/Mar/1|02:19:32] V-Ray: Total time updating frame  0h  0m  0.1s (0.1 s)
    [2016/Mar/1|02:19:32] V-Ray: Preparing renderer...
    [2016/Mar/1|02:19:32] V-Ray: Using Intel Embree ray server
    [2016/Mar/1|02:19:32] V-Ray: Preparing scene for rendering...
    [2016/Mar/1|02:19:32] V-Ray: 6 lights found.
    [2016/Mar/1|02:19:32] V-Ray warning: Subpixel color mapping is on: rendered result may have incorrect brightness.
    [2016/Mar/1|02:19:32] V-Ray: Max ray intensity is enabled: rendered result may have incorrect brightness.
    [2016/Mar/1|02:19:32] V-Ray: Preparing scene for frame...
    [2016/Mar/1|02:19:32] V-Ray:  (Estimated time remaining:  0h  0m  0.0s): 17 % completed
    [2016/Mar/1|02:19:32] V-Ray: Bitmap file "//192.168.0.226/prod/proj61/maya/sourceimages/tomoco_studio3.exr" loaded.
    [2016/Mar/1|02:19:33] V-Ray: Bitmap file "//192.168.0.226/prod/proj61/maya/sourceimages/light01-2.hdr" loaded.
    [2016/Mar/1|02:19:33] V-Ray:  (Estimated time remaining:  0h  0m  1.4s): 20 % completed
    [2016/Mar/1|02:19:33] V-Ray: Bitmap file "//192.168.0.226/prod/proj61/maya/sourceimages/light02.hdr" loaded.
    [2016/Mar/1|02:19:33] V-Ray:  (Estimated time remaining:  0h  0m  1.6s): 22 % completed
    [2016/Mar/1|02:19:33] V-Ray:  (Estimated time remaining:  0h  0m  1.5s): 25 % completed
    [2016/Mar/1|02:19:33] V-Ray:  (Estimated time remaining:  0h  0m  1.5s): 26 % completed
    [2016/Mar/1|02:19:33] V-Ray: Compiling geometry...
    [2016/Mar/1|02:19:33] V-Ray: Using embree ray tracing.
    [2016/Mar/1|02:19:33] V-Ray: Building embree static accelerator ...
    [2016/Mar/1|02:19:33] V-Ray: Building embree static trees took 37 milliseconds
    [2016/Mar/1|02:19:33] V-Ray: Loading light map from file "//192.168.0.226/prod/proj61/maya/images/scene669/pre-render/vrlmap_precalc_per_frame_VV2_renderCam.0001.vrlmap"...
    [2016/Mar/1|02:19:33] V-Ray: Light map loaded successfully.
    [2016/Mar/1|02:19:33] V-Ray: Prefiltering light cache...
    [2016/Mar/1|02:19:33] V-Ray: Average rays per light cache sample: 137.04 (min 1, max 4274)
    [2016/Mar/1|02:19:33] V-Ray: Rendering image...
    [2016/Mar/1|02:19:33] V-Ray:  (Estimated time remaining:  0h  0m  1.2s): 1 % completed
    [2016/Mar/1|02:19:33] V-Ray:  (Estimated time remaining:  0h  0m  0.0s): 82 % completed
    [2016/Mar/1|02:19:33] V-Ray:  (Estimated time remaining:  0h  0m  0.0s): 99 % completed
    [2016/Mar/1|02:19:33] V-Ray: Number of raycasts: 259121 (0.04 per pixel)
    [2016/Mar/1|02:19:33] V-Ray:   Camera rays: 81887 (0.01 per pixel)
    [2016/Mar/1|02:19:33] V-Ray:   Shadow rays: 0 (0.00 per pixel)
    [2016/Mar/1|02:19:33] V-Ray:   GI rays: 0 (0.00 per pixel)
    [2016/Mar/1|02:19:33] V-Ray:   Reflection rays: 0 (0.00 per pixel)
    [2016/Mar/1|02:19:33] V-Ray:   Refraction rays: 0 (0.00 per pixel)
    [2016/Mar/1|02:19:33] V-Ray:   Unshaded rays: 0 (0.00 per pixel)
    [2016/Mar/1|02:19:33] V-Ray: Number of light evaluations: 0 (0.00 per pixel)
    [2016/Mar/1|02:19:33] V-Ray: Number of intersectable primitives: 163849
    [2016/Mar/1|02:19:33] V-Ray:   SD triangles: 163848
    [2016/Mar/1|02:19:33] V-Ray:   MB triangles: 0
    [2016/Mar/1|02:19:33] V-Ray:   Static primitives: 0
    [2016/Mar/1|02:19:33] V-Ray:   Moving primitives: 0
    [2016/Mar/1|02:19:33] V-Ray:   Infinite primitives: 1
    [2016/Mar/1|02:19:33] V-Ray:   Static hair segments: 0
    [2016/Mar/1|02:19:33] V-Ray:   Moving hair segments: 0
    [2016/Mar/1|02:20:12] V-Ray: Successfully written image file "//192.168.0.226/prod/proj61/maya/images/scene669/slices/scene669_slice8_VV2_renderCam.0001.exr"
    [2016/Mar/1|02:20:12] V-Ray: Total frame time  0h  0m 39.7s (39.7 s)
    [2016/Mar/1|02:20:12] V-Ray: Total sequence time  0h  0m 39.7s (39.7 s)
    [2016/Mar/1|02:20:12] V-Ray: Render complete
    [2016/Mar/1|02:20:12] V-Ray: ========================
    [2016/Mar/1|02:20:12] V-Ray: Clearing exporter memory...
    [2016/Mar/1|02:20:12] V-Ray: Total time clearing exporter memory  0h  0m  0.0s (0.0 s)
    Scene //192.168.0.226/prod/proj61/maya/scenes/scene669.mb completed.
    
    ----------------------------------------------------------
    Time: 2016/3/1 02:20:12
    ----------------------------------------------------------
    This is a very quick render but the command had been running for 12 hours before we noticed it, and we had to kill the mayabatch process manually.

    Since we have multiple V-Ray versions in our pipeline and we just now realized this is happening at least a couple of time a week, it's so far been quite difficult to nail this down to a specific V-Ray version. However, in this particular case V-Ray for Maya 3.05 nightly build 25787 was used. It's quite possible that this issue has appeared with more recent releases but I have yet to have that confirmed.

    This seems to happen to Windows 10 machines only. However, these machines can go on render images for many days before the issue is being hit. To further complicate things, the same machine failing to exit properly may re-render the same command without issues and exit properly.

    Is this anything you've experienced or had reported from other customers?
    Best Regards,
    Fredrik

  • #2
    Just adding that I can confirm this behaviour. Haven't mentioned it before as I wasn't sure if it was renderpal, vray or the processor/mem config on the win 10 machines here.
    on our farm:
    The win 7 machines run all fine (i7 920's)
    The win 10 machines occasionally freeze at end of render (i5 processors)
    All have the same nvidia GTX 980 cards, as the bulk of our rendering is now GPU we assumed new machines didn't require heavier CPU's.

    Comment


    • #3
      Sounds a bit random. Since I upgraded to win 10 I haven't seen this, but I'll try and see if I can reproduce it so we can fix it. Have you tried more recent builds and if yes, is this still happening? Any hint on what's common between the scenes when this happens?
      Alex Yolov
      Product Manager
      V-Ray for Maya, Chaos Player
      www.chaos.com

      Comment


      • #4
        I am now actively collecting data for each case where this happens. We still don't have much to go on. Here's one case where we got the freeze:

        Code:
        maya_version: 2015
        maya_edition: ext1sp6
        vrayscatter_version: 4.197-vray3
        vray_version: 3.05-b25787-nightly
        bonustools_version: 1.0
        
        GPU: Quadro FX 3800
        CPU: Xeon X5570@2.93GHz
        I also know that one machine has been freezing like this, which is using a Geforce GTX 970 also with Xeon X5570@2.93GHz CPU. I doubt this is tied to Geforce drivers as we had the issue with the Quadro card above.
        Best Regards,
        Fredrik

        Comment


        • #5
          Originally posted by yolov View Post
          Sounds a bit random. Since I upgraded to win 10 I haven't seen this, but I'll try and see if I can reproduce it so we can fix it. Have you tried more recent builds and if yes, is this still happening? Any hint on what's common between the scenes when this happens?

          Unfortunately, we haven't collected data on each case so it's hard to see a clear root cause at this moment. We have tried more recent builds and artists are unsure whether they have experienced issues or not. They *think* this also has happened with 3.15 builds... but I can't confirm that.

          We don't yet see enough common denominators between scenes just yet.
          Best Regards,
          Fredrik

          Comment


          • #6
            @ricoholmes - one thing which has crossed my mind... have you experienced this issue on newly-installed Win10 machines?

            It seems here, that when you have installed Win10 and have pending updates from Windows Update, this issue is more likely to be hit. We have machines running Win10 which have had the initial batch of updates just after initial Win10 install applied ... and they have been running just fine for a long time now. One machine which got installed with Win10 and did not have the initial updates applied directly exhibited the freeze issue.

            It could be interesting to explore whether this could be the root cause...
            Best Regards,
            Fredrik

            Comment


            • #7
              Sorry for delayed reply, I've been on the road a spell.. Those machines were added in September and allowed update weekly, so both "old" and up to date. Unfortunately the symptoms seem no more or less apparent. I've been trying to observe commonalities, but it's frustratingly vague stuff. No version of the nightlies has evidenced more or less of the problem.

              Thoroughly useless feedback here I'm sure: but it "feels" like it hangs more frequently when switching between vrScenes on a render list. I can't imagine how that makes sense however.
              "feels" is incredibly poor for bug testing, I know.

              again, The windows 7 nodes never hang. Only Win 10



              edit: as soon as I typed this I noticed one of the win 10 nodes had hung mid sequence (vfb buffer up, displaying black fwiw). so not nescessarily when switching to next vrscene.
              Last edited by ricoholmes; 03-03-2016, 02:59 AM.

              Comment


              • #8
                Originally posted by ricoholmes View Post
                it "feels" like it hangs more frequently when switching between vrScenes on a render list.
                Are you using some batch scripts to render with standalone? What's the place of vrscenes in your workflow?
                Alex Yolov
                Product Manager
                V-Ray for Maya, Chaos Player
                www.chaos.com

                Comment


                • #9
                  Yes, animations are exported from Maya as vrscenes, which are then queued up in Renderpal.
                  on that note: each Renderpal client needs to be restarted when standalone hangs too (i imagine waiting for a "finished" to be returned from standalone.)

                  As mentioned before, I wouldn't completely dismiss Renderpal as the problem either.

                  Comment


                  • #10
                    A bit of a necro to this thread, but a positive one. We gave up fighting the windows 10 upgrade and now the farm is all win 10. This exacerbated the problem of course, so I bit the bullet, killed Renderpal and set up Smedge in its place.

                    The problems seem to have been cause entirely by Renderpal. VRay has been rock solid since migrating to Smedge.

                    Comment


                    • #11
                      Good to hear it's working, but it's still odd that this should happen. Keep us informed on other issues, should any arise.
                      Alex Yolov
                      Product Manager
                      V-Ray for Maya, Chaos Player
                      www.chaos.com

                      Comment


                      • #12
                        I'm really sorry to say, but I'm afraid I spoke too early. Smedge is just a lot better at handling/restarting errors apparently. It can't recover from a "VRay Standalone has stopped working" error however, of which a number of machines suffered overnight.

                        Would happily supply an vray.log file if I could find one %tmp% is only revealing a vray.dmp file.

                        Win 10 professional
                        vray build 26811 (standalone renders)
                        -rtEngine=5 -rtNoise=0.01 -rtSampleLevel=20000 -rtTimeOut=8 -display=1 -autoClose=1 (in case that helps)

                        Comment


                        • #13
                          Do you see a dialog that allows you to send a crash report to Microsoft when V-Ray Standalone crashes?
                          Can you share some logs of the rendering or the render manager?
                          Do you have crash dumps?
                          V-Ray developer

                          Comment


                          • #14
                            Originally posted by t.petrov View Post
                            Do you see a dialog that allows you to send a crash report to Microsoft when V-Ray Standalone crashes?
                            Can you share some logs of the rendering or the render manager?
                            Do you have crash dumps?
                            No crash report option for microsoft I'm afraid
                            Still getting to grips with Smedge, so will see what I can dig up there
                            "Crash dumps" ? - is this the vray.dmp file found in %temp% ?

                            Comment


                            • #15
                              Originally posted by ricoholmes View Post
                              "Crash dumps" ? - is this the vray.dmp file found in %temp% ?
                              Yes, I think this is the file.
                              V-Ray developer

                              Comment

                              Working...
                              X