Renders are slow to unload

Yes this is an issue, when you have many textures all tiled. Depending on your vray version, if you are in vray 3, you may try option - cache bitmaps between renders. This will keep the textures in ram on the same mayabatch job I believe. If you are in vray 2.x you need to declare this variable for the same result:

SET VRAY_MAYA_KEEP_BITMAPS=1