If you take an item (or stack of items) out of a container right as it's removed by redstone, and the server thread is under heavy load, then the client and server can disagree on whether the stack has been removed or not. This means that the client can sometimes take the same stack out twice, making the appearance of item duplication.

While the duplicated items disappear when the container's GUI is closed, they persist for as long as it's open. I've reproduced this with hoppers and droppers in singleplayer, but so far I haven't tested any other containers, or tested the bug in multiplayer.