Affects Version/s: Minecraft 16w38a, Minecraft 16w39a, Minecraft 16w39b, Minecraft 16w39c, Minecraft 16w40a
Fix Version/s: Minecraft 16w41a
Setting it to 2 allows 3 mobs in one spot. They will suffocate when there are 4 mobs in one spot.
As Jeb_ mentioned, one entity more than gamerule should suffocate mobs:
For 1.11 there is a new game rule called "maxEntityCramming" with the default set to 24 (i.e. 25+ entities at the same spot will suffocate)
but they suffocate when there are 2 more mobs.
This behaviour disallows setting this gamerule to "when anything touches anything, suffocate them" by giving value of 1.