Cannot punch elytra out of the item frame have tried this with muliple item frames with elytra in

left click makes the noise but hey stay in right click just spins it and if i destroy the block behind and then try to break the frame it just disapears