When the player changes its gamemode to Spectator the Giant despawns.
I've tried with other mobs and these don't despawn.
My difficulty is not in Peaceful.

If you spawn a Giant (obviously with /summon) when in Spectator mode, the Giant appears for like a milisecond and then disappears.