When ever I load the server list, it takes a good 10-15 seconds for any info to load. Afterwards, if I join a server it also locks up for 10-15 seconds when joining a server.

In 1.3.1/2, there was this (http://www.minecraftforum.net/topic/1461783-fixing-lag-and-freezing-when-joining-a-mp-server/) patch that fixed it. While I don't know the technical reasons very well, I do know that this fixed my issue. Hopefully that can help you guys track it down.