Using most items underwater can boot you out of swimming mode, specifically when you hold the use button. This is most noticeable when using tridents or brushing suspicious blocks underwater.