Hi,
is there a real benefit to use .exr versus .tx for 16b map ?
thank you
Tiled EXR etc should help with ram usage and so on… yea go EXR
Do you have more informations about it please ?
unexpectedly, I have some some surprises with .tx
it seems to go faster with some renderfarm… (windows XP) and slower with others… (wondows7)
I continue my tests, but if someone has specifications about memory usage with EXR and TX…
I had allways worked with .exr, but 16b seems to be too much for my current renderfarm 
thank you
The EXR did u use vray convener to make them tillable EXR ?
There is app in vray folder from chaos that u can use and convert images to tiled EXR…
img2tiledexr.exe from Chaos to convert into .exr
and
maketx.exe from oiio to convert in .tx
both are mipmapped.
this is my first time with .tx in Vray. I use it because I have some issues with the memory charge.
I use .tx for 8b texture (img2tiledexr.exe does only 16b)
Humh interesting. Soo what issues are they? Are the textures really big or you just trying to run something on ur smartphone? 
Yes, textures are really big… in fact no… average is 6K, but I have a lot
1/4 are 16b…
as my current renderfarm is mixed (different kind of machines… different OS…) it’s hard to get some real stats.
But on some nodes, .tx seems to go really faster than .exr … so I’m surprise and i’m wondering what is the difference between .tx mipmap and tiled exr in the memory management…
The 8-bit .tx files are more memory efficient; OpenEXR has only 16- or 32-bit support so it has at least twice the memory footprint as an 8-bit .tx file.
Best regards,
Vlado
Hi Vlado, and thanks for your reply.
I understand the difference between 8b and 16b.
What I’m wondering is: are the 16b .tx (mipmap) and the 16b .exr (tiled) acting in the same way during the computation ? I mean, memory usage, network access caching, loading/unloading etc… currently is there a real advantage to use tiled .exr vs 16b mipmap .tx ?
Does the compression option for tiled .exr affect the memory usage? I mean, uncompress the file for each access… does it slow down the render in case of huge quantity of maps ?
is the tile accessed at the bucket level ?
In fact, I’m trying to drastically optimize scenes with massive maps usage… So I try to really understand what depends of memory, network, CPU etc…
Maybe could you give me a link to get some specifications… I can’t find technical documentation on this, specifcally for VRay.
Finally, I have scripted my optimisation: all 8b textures are mipmap 8b .tx, and others are tiled .exr
Why don’t you ditribute a a kind of maketx.exe for 8b maps? This solution seems to be unknown by VRay users… or maybe it’s just me 
The fact that this tool is not given with VRay is not really intuitive.
thanks in advance for your time!
Vlado, do these textures also help out the sampler (speed) or is it purely about memory efficiency?
It’s only about memory.
Best regards,
Vlado
Following your replies, I had a look at BenDel tests to see if high render times with tiled exr files were due to some bottleneck of our renderfarm.
Warning , tech details ahead !
The differences between 16bit uint tx and 16bit half exr are definitely linked to io and bandwidth limitations of our server and the way different Oses ( WinXp64 and Win7x64) are accessing it.
When only a few machines access the server, we don’t see big differences between tx and exr tests.
This is quite different under heavy load. Under heavy load, tx scenes slow down a bit but keep a consistent output rate between machines. This is not the case for exr tests where Win7 machines eat up the bandwidth, keep a consistent rate , but slow down Xp machines by
a huge factor.
So on our renderfarm, for a scene with very high texture count/size, tx are much more efficient ( between 5x and 15x for the test scene ). To investigate a bit more , we ran several tests with BenDel and here are tech details that we found :
. IO read bytes are much higher on scenes with exr files ( very low on tx , memorymap ? )
. the render log seem to show that exr are entirely read on first hit, whereas tx are only ‘mapped’ ?
[log details and end of this post in next post ( too many characters)]
Stephane.
Log with Tif files
Here is a part of the beggining of the logs of the same render scene with tif files, tx files, and a mix of tx and exr (all 16bit (uint for tx , half for exrs), tiled , mipmapped)
scene with TIF files :
[2013/Mar/15|11:51:28] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_color_HIGH_03.tif” loaded.
[2013/Mar/15|11:51:48] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_HIGH_OPA_03.tif” loaded.
[2013/Mar/15|11:52:08] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_color_HIGH_02.tif” loaded.
[2013/Mar/15|11:52:29] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_HIGH_OPA_02.tif” loaded.
[2013/Mar/15|11:52:50] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_facade_COL_03.tif” loaded.
[2013/Mar/15|11:53:02] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_facade_OPA_01.tif” loaded.
[2013/Mar/15|11:53:23] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_fenetresTranches_COL_01.tif” loaded.
[2013/Mar/15|11:53:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements_COL_03.tif” loaded.
[2013/Mar/15|11:54:05] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements_OPA_02.tif” loaded.
[2013/Mar/15|11:54:28] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements2_COL_01.tif” loaded.
[2013/Mar/15|11:54:50] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements2_OPA_02.tif” loaded.
[2013/Mar/15|11:55:13] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_fenetres_COL_03.tif” loaded.
[2013/Mar/15|11:55:19] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batD_f2/Df2_facadeB_color_v01.jpg” loaded.
[2013/Mar/15|11:55:23] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batD_f2/Df2_facadeB_alpha_v01.jpg” loaded.
[2013/Mar/15|11:55:46] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df2/Df2_BFacade_COL_02.tif” loaded.
[2013/Mar/15|11:55:59] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df2/Df2_BFacade_OPA_01.tif” loaded.
Note the time for each line, each texture is loaded sequentially, the time between log is proportional to tif file size. This seems like a normal behaviour as files are loaded when hit.
This is the slowest scene.
Log with tx files
scene with TX files :
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_color_HIGH_03.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_HIGH_OPA_03.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_color_HIGH_02.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_HIGH_OPA_02.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_facade_COL_03.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_facade_OPA_01.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_fenetresTranches_COL_01.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements_COL_03.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements_OPA_02.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements2_COL_01.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements2_OPA_02.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_fenetres_COL_03.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batD_f2/Df2_facadeB_color_v01.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batD_f2/Df2_facadeB_alpha_v01.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df2/Df2_BFacade_COL_02.tx” loaded.
[2013/Mar/19|11:09:44] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df2/Df2_BFacade_OPA_01.tx” loaded.
Note the time for each line, there is no pause when hitting the files, looks like only the requested tile is loaded/mapped.
This is the fastest scene.
Log with exr and tx files
scene with tx and exr files
[2013/Mar/19|18:20:17] V-Ray: Preparing scene for frame…
[2013/Mar/19|18:20:17] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_color_HIGH_03.tx” loaded.
[2013/Mar/19|18:20:17] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_HIGH_OPA_03.tx” loaded.
[2013/Mar/19|18:20:17] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_color_HIGH_02.tx” loaded.
[2013/Mar/19|18:20:17] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batG_b3/Gb3_HIGH_OPA_02.tx” loaded.
[2013/Mar/19|18:20:21] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_facade_COL_03.exr” loaded.
[2013/Mar/19|18:20:21] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_facade_OPA_01.tx” loaded.
[2013/Mar/19|18:20:26] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_fenetresTranches_COL_01.exr” loaded.
[2013/Mar/19|18:20:31] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements_COL_03.exr” loaded.
[2013/Mar/19|18:20:32] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements_OPA_02.tx” loaded.
[2013/Mar/19|18:20:32] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements2_COL_01.exr” loaded.
[2013/Mar/19|18:20:32] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_ornements2_OPA_02.tx” loaded.
[2013/Mar/19|18:20:36] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df1/Df1_fenetres_COL_03.exr” loaded.
[2013/Mar/19|18:20:36] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batD_f2/Df2_facadeB_color_v01.tx” loaded.
[2013/Mar/19|18:20:36] V-Ray: Bitmap file “Z:/…/Modeling/wip/ben/batD_f2/Df2_facadeB_alpha_v01.tx” loaded.
[2013/Mar/19|18:20:39] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df2/Df2_BFacade_COL_02.exr” loaded.
[2013/Mar/19|18:20:39] V-Ray: Bitmap file “Z:/…/Textures/Facades/Df2/Df2_BFacade_OPA_01.tx” loaded.
Once again, note the timed log : all tx files create no pause in rendering, exr files take some time to load at this point ( proportional to their size, doesn’t look like only a tile is loaded)
This is kind of mixed result, much faster than tif on faster Win7 machines, slower than tif on slower WinXp64 machines, but all in all much slower than tx (due to the higher load on our server and the “pauses” on exrs).
End of post and questions
So, to come to a conclusion ( sorry for the very long post), I would like to know if there is a reason for this difference between exr and tx behaviour (both are 16bit, tiled,mipmaped, compressed with standard recommended options).
Are there any drawbacks for the tx files vs exr ones that we might hit for real render scenes ( these were only tests scenes without gi or glossy stuff).
Is there any IT stuff that I can optimize to decrease render time with exr files ( higher bandwidth for server ? higher possible io count for server ? add a second gigEthernet to render machines ? etc).
Best Regards
Stephane.
just some details to complete the post from Stephane Bonnot:
The Maya scene test:
- 152 Maps.
- the resolution is varying between 2K and 6K
- around 2.5 Go in total on disk: .exr with zip default compression, .tx with default compression (no access to the option with maketx.exe from oiio)
- 48 maps are 16b
- 104 are 8b
- only selfIllumination and opacity in the scene… no lights… no env… no GI…
so.. the two tests, comparing .tx and .exr are done with:
- one with: 104 .tx 8b, and 48 .tx 16b
- the other with: 104 .tx 8b, and 48 .exr 16b
both are mipmap filtered
Ok now that is very interesting !
I gotta read in to that tx. format. Is there any tool to convert my textures to it or something?
I’m not sure why exactly this happens. We handle the two formats identically once they get to V-Ray (though they take up different amounts of RAM depending on whether they are 8-bit or 16-bit). Obviously the libraries for loading the two formats are completely different (we use the opensource libtiff for .tx files, and ILM’s OpenEXR library for .exr files); the physical data layout is probably also different.
Are there any drawbacks for the tx files vs exr ones that we might hit for real render scenes ( these were only tests scenes without gi or glossy stuff).I don’t think so.
Is there any IT stuff that I can optimize to decrease render time with exr files ( higher bandwidth for server ? higher possible io count for server ? add a second gigEthernet to render machines ? etc).I don’t have an answer to that either; we could try to do a similar set up and try to profile the whole thing, but it will take a while.
Best regards,
Vlado
Ok, different results make sense since libtiff and openexr lib might not handle disk io the same way(readahead, cache, etc), and the physical layout sounds like another sensible reason. By the way, I would have bet on openexr for the best performance before these tests …
Thanks a lot for the detailed tech info, we’ll stick to tx for this project (very good performance and no issues with this format yet) and we will post here if we find some valuable tech info.
For other members reading this post, note that this specific issue is bound to file IOs, we hit issues with an extreme scene (more textures than RAM) and when our file server serving textures is under heavy load (I don’t know yet if network bandwidth or disk IO count limitations or both are causing the issue), when run locally ( all texture files on the workstation with fast raid) the tests don’t behave the same way.
Best regards
Stephane