Before anyone says anything, this isn't one of the many "instant effect arrows do nothing" reports. It's about how the physical damage of the arrow replaces the magic damage of instant harming, even if the instant damage would be hurting the most.

Easiest way to show what I mean: Equip full diamond armor, and then run this command to summon an instant damage II arrow onto yourself:

/summon arrow ~ ~2.5 ~.1 {Motion:[0.0,-1.0],Potion:strong_harming,damage:12}

Note how I set the arrow's damage value to 12; this matches the damage instant harming II does. You'd think that the instant harming would do it's full 6 hearts of damage because it ignores the armor, right? Well it doesn't. The closer the arrow's damage gets to matching the harming effect's damage, the less overall damage it'll do. The armor starts reducing the damage, even though the magic damage of the arrow's effect should be ignoring it.

What seems to be happening is that the damage of the arrow itself is converting the magic damage into physical damage, which starts allowing it to be reduced by armor. Which ironically means the wimpiest possible bow charge is the way to do the most amount of damage against armor with instant harming arrows, and fully charging it leads to the worst damage. This bug applies to armored mobs as well.

To summarize: The physical damage of an arrow of instant harming is replacing its magic damage at a 1:1 ratio regardless of what would ultimately hurt more. What it should be doing is picking whatever the higher damage would be after all relevant defensive calculations are done.

P.S. Instant healing/harming arrows behave normally in regards to restoring HP. The bug only comes into play if the effect would hurt the target (healing vs undead, harming vs living).

P.P.S. Just found MC-107856 (well, pretty sure I saw it before, but had a closer look this time), which sounds the most similar, but like most of the other reports, it mostly just talks as if only the highest damage source is applied (which is normally true, and the expected behaviour). I didn't see any mention in any reports on what seems to actually be going on in the calculations for these arrows. It's not "whichever is stronger overrides the other" (even though it should be). It's "physical damage is 'deleting' magic damage 1 for 1 in all cases". For unarmoured targets, this appears to follow the "highest damage only" rule (and effectively, it does). But technically it's either mixed damage, or pure physical damage if it's equal to or higher than the magic damage value. E.g: 8 damage arrow with an instant damage II effect means 8 physical damage and 4 magic damage, even though it should mean 8 physical and 12 magic damage (and then ideally only applying the one that does more damage after all defense/enchantment defense is taken into consideration). The magic damage is literally being replaced (or somehow being ignored) by the physical damage instead of staying separate for calculations. And therein lies the root of the problem.