When firework rockets are selected in the off-hand, trying to boost them while flying does not work. They have to be in the main hand in order for flying to be boosted.

Holding it in the off-hand also does not allow the rocket to be launched, unlike Java Edition, but that's another story for "it's not possible to hold every item in your off-hand."

Note: I had to select "1.16.40 Hotfix" instead of 1.16.20 because it was not listed.