What I expected to happen was...:
When I try to teleport the armor stand with tag

{NoGravity:1}

relatively, it doesn't move on the first teleport. It does move on the second tp, relatively to its first position and not relatively to the position it should have. Leaving the world and reentering it fixes it, by taking into account the first tp, without cancelling other tp moves. This bug seems more like a client-side glitch.

What actually happened was...:
The armor stand actually moving, even with

{NoGravity:1}

.

Steps to Reproduce:
See the test_setup.png screenshot (it was taken facing towards negative x). There are 5 command blocks labeled, they contain the following commands:
1 /summon ArmorStand ~ ~1 ~

{CustomName:"armorstandfactory",NoGravity:1}

2 /replaceitem entity @e[c=1,type=ArmorStand] slot.armor.head minecraft:stone
3 /tp @e[rm=4,type=ArmorStand] ~-0.625 ~ ~
4 /tp @e[c=1,type=ArmorStand] ~-5 ~ ~
5 /kill @e[name=armorstandfactory]
Chain command blocks are set to "Always active"

Press the left-most button multiple time: it seems that the fourth command block has no effect, it is the first tp applied to the newly spawned armor stand. Maybe the problem is that the armor stand cannot be teleported client-side within the tick it got created? Remember this only applies to armor stands with

{NoGravity:1}

tag!