Yeah I think they went with 4" for layering purposes, people do admit that too
I'm saying the opposite (I think), that I think two inches more, so 6" total chest ease at least, are best on a cross zip because this produces a more flattering line (and yes, the fact that it can be worn with more than a t-shirt is a big plus too!)
I come from a menswear/tailoring background (that's why I joined this site 16 years ago) so my view tends to be that if a piece of outerwear only fits over a t-shirt (which is a form of underwear technically), then it is too small. I own leather jackets that this applies to, but I think by definition they are too small as a result.