Vines slow you down when flown through in spectator mode. They also slow you down when flown through in creative mode, but that makes sense. However, in spectator mode you cannot interact with any blocks. You should not be slowed down with vines. I assume this happens with ladders too but I can't test because my game crashes when I place one.