Whenever I try to weaken a silverfish to the extent of not harming players in survival mode, it can always do damage regardless of circumstances.

To reproduce, run the following command -

/summon Silverfish ~ ~ ~ {Attributes:[{Name:generic.attackDamage,Base:0}],ActiveEffects:[{Id:18,Amplifier:10,Duration:20000000}]}

Not only should it deal a default of 0 attack damage, but it also has weakness 10, which should prevent it from damaging altogether.