I think I see the issue, I have more or less the same. Before SP3, it was possible to increase subdiv of the dome HDRI use in direct light for only a small increase in render time. (see PG post, point 8 to illustrate HDR Sky Lighting for Interiors — PETER GUTHRIE ) Now, you just can’t, and the noise of the HDRI is too noticable when working with IR allowing higher noise thresold (and thus faster render times).
If you lower the noise thresold for the whole scene like it’s advised, the render time increases too much (and bascially the use of an IR for speed is then worthless, like you said in another post)
I made some tests and swtching back the HDRI into the IR worked good so far, but for IR user it’s a step back