Announcement

Collapse
No announcement yet.

Multiple crashes

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Multiple crashes

    I've been having a hard time recently trying to get certain files to render.

    I am using a primary, old machine with an i7 Intel processor and 24GB of RAM, as well as three old render slaves with dual Xeon E5 processors with 16GB of RAM each.

    I use the render slaves, when they work, via Distributed Rendering as Backburner won't zip the 3dsmax file as the various geometry and maps are larger than Backburner's 32-bit file size limit.

    Some of the errors I am running into:

    One of the files has a lot of foliage objects in Forest Pack Pro and uses up a large amount of memory. That simply is too much for the slave computers and they've never been able to join in. OK, so be it. So for that foliage file, I simply use my main computer to render, which it's able to do.

    1) However, when I render a large resolution file, marking it to save as an .exr beforehand and using resumable rendering, when vray finishes the image, finishes the render elements, finishes the denoiser pass, and saves everything out, vray then instantly closes all of 3dsmax and dumps me out to a blank Windows desktop. Why is vray shutting down all of 3dsmax when it finishes saving? If I do not check to save the file beforehand and simply let the render finish by itself without saving, vray will not close 3dsmax.

    2) The resumable .vrimg is really helpful, especially since I'm getting a lot of crashes during the rendering process. However, at one point during the night, the computer crashed, probably with a hard blue screen of death - I could not see as I had turned the monitors off and they would not turn back on to let me see the screen, but I was forced to do a cold reboot. After this, the .vrimg did not reload; it seemed to get corrupted. I could not get anything out of it either by loading it into the vray frame buffer nor by using the standalone vrimg to exr converter. If a vrimg gets killed mid-process like this, is there no way to get anything out of it that has already rendered?

    3) When only using my primary computer, trying to render a 4,000 x 3,000 pixel image, I tried pre-calculating the light cache. I even turned off the vray frame buffer and set it to render directly to a vrimg only to save RAM. Vray started the process and hung at the Building Embree Static Accelerator stage. Am I simply too low on memory?

    4) When rendering a slightly smaller file, I can use the slave nodes alone (without using the main computer, as it seems to be more stable without rendering on the host machine at the same time). However, some times the render slaves will get the distributed files, the vray messages window on my host machine will note in purple text that the slave machines are starting, but then one or more of the slave machines will die out/ crash 3dsmax, or will simply never be able to join, but that slave node will stay in a process of RAM swapping near 98%. Is there a message window that I can open on those slave machines to see what they are doing in realtime?

    5) For distributed rendering, are files lighter and easier to render if brute/brute is used, brute/light cache (precalculated and saved by the host machine), or IR/LC?

    6) Is there a way to see if the geometry or the maps are causing the most memory usage?

    7) If none of my texture bitmaps (mostly jpgs, some tifs, nearly no exrs) are huge (all are likely 4K or smaller), would converting all of them to tiled tx files be very useful or not?

    If I am not using any tiled tx or tiled exr bitmaps, can I turn the tiled image cache to something very low (like 1) but not zero (since zero is actually unlimited) and increase the available memory for my render?

    9) Would converting large amounts of my geometry to vray proxies help much?

    10) Any other tips to get these renderings to be more stable and work consistently?

    Thanks,
    Matt


  • #2
    1)It could be due to insufficient resources.
    What the RAM, HDD utilization when the crash take place?
    Are you saving the image locally or on a network location?
    Did you notice on what is the resolution after which this issue starts to appear?
    Does 3dsMax saves minidump file for that crash? If yes would it be possible to send it over for inspection?


    2) I'm afraid this is not possible, if file is corrupted due to instant reboot we can't do anything to save it. It might be a good idea to disable windows sleeping or monitor sleeping- Windows tend to have some issues to resume the session if machine is being heavily used.

    3)Yes it could be. Hanging at LC stage is a good sign about insufficient memory issue. Have you noticed whats the RAM utilization at that time?

    4)There is no such message window separate from the V-Ray Messages window. You can however which the log file locally, it should be in windows temp location.
    One thing is certain if RAM utilization is near 100% it's expected to see such behaviour. You may try to open the scene locally on that machine and see if it could render it or not.

    5) I need more information here in order to answer accurately. By lighter/easier what do you mean - less RAM consumption or something else?

    6) Not directly but you may run a few debug tests to get that information. For example you may disable Maps checkbox from the Global Switches and see how much memory is saved compare to when it's enabled.
    For the geometry you may use default gray shader as Override material and that would give you approximately the memory usage of the Geo.

    7) Converting to tiled files will be useful in any case. Textures might be small in terms of size on disk or resolution side but they might expand quite a lot into the memory while rendering - so converting them is definitely a good idea.

    9) That's why VRayProxy is invented actually If you haven't benefit yet from it maybe it's time to start using it.

    10) Yeap, check this page in the Docs:
    https://docs.chaosgroup.com/display/...whilerendering
    Svetlozar Draganov | Senior Manager 3D Support | contact us
    Chaos & Enscape & Cylindo are now one!

    Comment


    • #3
      Hi Svetlozar,

      Most of the problems seemed to be related to insufficient memory. The slaves were swapping with the HDD and memory use was nearly 99-100% during the crashes. Most of the crashes happened after I raised the render resolution from about 2K (4:3 ratio) to 4K x 3K. I had also tried it with the VFB disabled and writing directly to the vrimg, but that also crashed at 4K.

      I tried five things and they seemed in tandem to help the file render, but I do not know which helped the most. A) The scene did not have many high-poly objects. Rather, the scene had hundreds of instances of small-poly objects placed manually (think screws, bolts, and nuts). These are not scatter objects, but I still placed them into custom-tree Forest Pro objects and hid the original instances. B) I converted nearly all the vrayHDRI textures (jpgs, pngs, tifs) to tiled .tx files. C) I turned on "Conserve memory" in the Embree advanced settings in the V-Ray render dialog. Perhaps that made the most difference. I already had limited the cache for dynamic objects below the maximum amount of memory of the computers, so that was unchanged. D) I converted a few of the larger poly xrefs to individual proxies. E) I tried making a vrscene from one or two of the xrefs, but this really did not seem useful.

      Now the slaves render the scene using about 67% of the available memory.

      Question 5 ) Yes, by "lighter" I did mean using less memory during the rendering phase. Would this be the brute/brute approach since it has no cache to load?

      Question 8 ) If I was not using any tiled tx or tiled exr bitmaps, can I turn the tiled image cache to something very low (like 1) but not zero (since zero is actually unlimited) and increase the available memory for my render?

      Question 8b ) How do the tiled texture cache and dynamic object cache add up? For example, by default the tiled texture cache is set to 4000 MB. If the dynamic object cache is set to 10000 MB, does that mean that on a computer with only 16 GB of memory available, there is only 2 GB left for everything in the scene that is not tiled or dynamic?

      Could you send a screenshot of the V-Ray render message window of the lines that give the approximate memory usage for geometry and maps as you've noted in answer 6? I get lost in all the technical output of that message window.

      Thanks for your help,
      Matt

      Comment


      • #4
        Originally posted by FlynnAD View Post
        Question 8 ) If I was not using any tiled tx or tiled exr bitmaps, can I turn the tiled image cache to something very low (like 1) but not zero (since zero is actually unlimited) and increase the available memory for my render?
        No, it won't make a difference. The texture cache won't take any memory if there aren't any tiled images used. The cache size is a just a limit, only the necessary amount of memory is used.


        Originally posted by FlynnAD View Post
        Question 8b ) How do the tiled texture cache and dynamic object cache add up? For example, by default the tiled texture cache is set to 4000 MB. If the dynamic object cache is set to 10000 MB, does that mean that on a computer with only 16 GB of memory available, there is only 2 GB left for everything in the scene that is not tiled or dynamic?
        Yes, they are separate caches. But this will happen only if both caches are actually full.
        Radoslav Platikanov | V-Ray R&D

        Comment

        Working...
        X