Distributed Rendering X64 -> XP32

[Micha_cg]

Re: Distributed Rendering X64 -> XP32

Next test - push the limit. I installed 6GB RAM (I took from my slave engine). Rendertest -> crash at 3.6GB. Is Vray or Rhino the limit? I copied a scene several times with the goal, to use more than 4GB RAM - crash at 3.6GB. So, it's clear, the Rhino limit is 3.6GB independent from the installed RAM and 5GB installed RAM should be enough for Rhino.

So, if we like to use more RAM, than Joe could help with a spawner trick at the master engine.

[Travis Serio_cg]

Re: Distributed Rendering X64 -> XP32

Micha even with 6 gigs installed Rhino can only access half of that because the kernel will require the same to manage rhino. In order to get rhino to use 4.2 gb (the limit for any 32 bit process) you would need at least 8.4 gb of memory installed. Is the spawner you are using a true 64 bit process?

[Micha_cg]

Re: Distributed Rendering X64 -> XP32

Interesting. Should I get a mem usage difference between 5GB and 6GB installed RAM?

[Travis Serio_cg]

Re: Distributed Rendering X64 -> XP32

Well.. I've heard of some ways to get more virtual memory usage but here's the way it works to my understanding. a 32-bit processor uses 32 bits to refer to the location of each byte of memory. 2^32 = 4.2 billion, which means a memory address that's 32 bits long can only refer to 4.2 billion unique locations (i.e. 4 GB).

In the 32-bit Windows world, each application has its own “virtual” 4GB memory space. (This means that each application functions as if it has a flat 4GB of memory, and the system's memory manager keeps track of memory mapping, which applications are using which memory, page file management, and so on.)

This 4GB space is evenly divided into two parts, with 2GB dedicated for kernel usage, and 2GB left for application usage. Each application gets its own 2GB, but all applications have to share the same 2GB kernel space.

By using the 3gb switch in the boot.ini you are essentially stripping 1gb of ram from the kernel and dedicating it to process usage. I have heard that Windows Server 2003 edition has the ability to allow a 4 gb switch in the boot.ini file. Though I have not tried it, it may help in this situation.

There is also an even more radical approach that may provide useful to your situation. Though I do not think this solution is the easiest approach, people running terminal services often use it on very busy servers. Here is a direct quote from a book on the subject


Here are more details about what booting /PAE means from Chapter 7 of the book "Inside Windows 2000," by David Solomon and Mark Russinovich.

All of the Intel x86 family processors since the Pentium Pro include a memory-mapping mode called Physical Address Extension (PAE). With the proper chipset, the PAE mode allows access to up to 64 GB of physical memory. When the x86 executes in PAE mode, the memory management unit (MMU) divides virtual addresses into four fields.

The MMU still implements page directories and page tables, but a third level, the page directory pointer table, exists above them. PAE mode can address more memory than the standard translation mode not because of the extra level of translation but because PDEs and PTEs are 64-bits wide rather than 32-bits. The system represents physical addresses internally with 24 bits, which gives the x86 the ability to support a maximum of 2^(24+12) bytes, or 64 GB, of memory.

As explained in Chapter 2 , there is a special version of the core kernel image (Ntoskrnl.exe) with support for PAE called Ntkrnlpa.exe. (The multiprocessor version is called Ntkrpamp.exe.) To select this PAE-enabled kernel, you must boot with the /PAE switch in Boot.ini.

This special version of the kernel image is installed on all Windows 2000 systems, even Windows 2000 Professional systems with small memory. The reason for this is to facilitate testing. Because the PAE kernel presents 64-bit addresses to device drivers and other system code, booting /PAE even on a small memory system allows a device driver developer to test parts of their drivers with large addresses. The other relevant Boot.ini switch is /NOLOWMEM, which discards memory below 4 GB and relocates device drivers above this range, thus guaranteeing that these drivers will be presented with physical addresses greater than 32 bits.

Only Windows 2000 Advanced Server and Windows 2000 Datacenter Server are required to support more than 4 GB of physical memory. (See Table 2-2.) Using the AWE Win32 functions, 32bit user processes can allocate and control large amounts of physical memory on these systems.

[Micha_cg]

Re: Distributed Rendering X64 -> XP32

Thanks Travis, I don't see, why I must build in 8GB to get 4GB for Rhino. But I suppose so, I don't understand it.
I wonder me, that Rhino crash at the same limit with 5GB and 6GB RAM. The spawner is a 32bit process yet. But, it could be good to get a Rhino.exe independent render process. So, Rhino could use 3..4GB for Modelling and Vray could use own memory. And I suppose so, we get a 64bit spawner befor we see a 64bit Rhino.

[Travis Serio_cg]

Re: Distributed Rendering X64 -> XP32

i need to correct that. 8 gb would not matter after all. The kernel appears to have a max limit of 2GB. I think the only way you'll get more ram usage from rhino is either through windows server 2003 or configuring PAE mode. If you're getting 3.6GB i am not sure the extra 400 mb is worth the trouble. As for the crash... I am not sure either. It could be possible that by reducing the kernels default 2GB to 1GB using the boot switch, Rhino may not be large memory address aware, or its not working as it should and is crashing. I don't know if the kernel running with only 1GB is enough to manange the large page file swapping and memory management that may be required.

[Micha_cg]

Re: Distributed Rendering X64 -> XP32

Thank you Travis, I think, 3.6GB is a good value for now.

[Micha_cg]

Re: Distributed Rendering X64 -> XP32

Sidenote: my 1Gbit LAN show a 8% max usage only, so, the LAN isn't the bottle neg of DR rendering.