Long story short, after acidentally attacking a villager while killing a witch, the gossip eventually made its way to the iron golem and it followed me. Running back into a house, I retreated to the very back wall opposite the door. Was watching the Golem through the door and bang, was one-shotted into the air. There was no attack animation either and I am 100% there was only one golem.

Reproduce: I guess make a room with about 3 blocks between the double doors and the wall.
Attack a villager and wait for the golem to hear the news
Enter the house and retreat to the furthest back wall

I know this could be a duplicate, but they all seem to be attacking through walls and not exactly at this 3 block range. This was a hardcore world and I was on track to going to the end dammit! The salt will never go away