When fireworks without NBT tags (defaults, i.e. the /give command with no other parameters) are fired from dispensers, they always face and move upward rather than facing the same as the dispenser is.

Edit: This issue has been patched in version 1.16.201 and should be closed