Seems like small magma cube's collision box isn't right.

Use the crosshairs to attack the magma, and you will find that the collision box of the small magma has a collision error, and the same as the medium magma.

/summon magma_cube ~ ~ ~ ~ ~ spawn_baby