When flying with an Elytra too fast for the game to load the chunks being generated there is a good chance you can hit a block and fall but when the chunks load often times you will see that there was no blocks there but you still fell