When a baby zombie naturally spawns with a chicken, it will move, but you will only see the chicken move. You cannot see the zombie, and it will appear as if the chicken is attacking you. When you kill the chicken, the baby zombie will appear back where it first spawned.

Will add screenshots next encounter.