I've found that 99% of the time using realms it works fine, but as soon as I enter a discord voice channel it immediately starts lagging and disconnecting with "java.net.SocketException: Connection reset" and time-out messages.

Also, joining a voice channel while playing a non-realms server does not effect my connection at all.