If you do shift left click the 'Destroy item' button it will only clear the hotbar. If you open a container or relog it will update your inventory clearing it. It could also be me PC because Minecraft tends to run slow for me.