Proper casters really only have their magic, so Its important that every hit really counts because they have limited casts and lack the vit to stand against melee oriented builds without magic.
To preserve the image of the cov, if a change is adopted, you could modify the 0 poise uniform or create a caster specific uniform. It would be a tad restricting, but this cov is built on rules and restrictions.