The vast majority of the time when I try to break something with the appropriate tool (or even just anything) it will switch to a random item in my hotbar upon clicking on what I want to break. I will switch back to the correct to tool, start mining, it'll change my item to something else and I'll do this up to 10 times before it stays on the tool I want. This also happens occasionally when attacking mobs.