When creating a world with the "Caves" world type, it will generate a Netherrack cavern, even with an Overworld biome selected. With "Floating Islands", it will just generate an End Stone plain, not even floating islands as far as I could tell.

Here's what the Ocean biome looks like, in "Caves" and "Floating Islands".

This bug is newly introduced in 21w44a; it does not occur in the previous snapshot.