While making a small house a villager wouldn't get out of my way. I held down right click and the second he moved it placed a block without using the one I had in my inventory.