What I expected to happen were Frozen River biomes in ALL cold biomes.

What actually happens is generation of simple River (not Frozen River) biomes in the neighbourhood of other cold biomes than simple Ice Plains. Namely, rivers are not frozen by Ice Mountains, by Ice Plains Spikes, by Cold Taiga and Cold Taiga Hills biomes. A taiga covered with snow, with frozen lakes, and with not frozen narrow rivers looks really weird. It is even more weird when you walk along a frozen river "flowing" down Ice Plains which suddenly becomes full of water with no ice when the terrain becomes hilly (Ice Mountain biome) or when ice spikes happen.

How to reproduce: just find Ice Plains biome, and a river intersecting it, and go along. River. For example, examine seed -435130594162764786 at the coords. 2954 -3927 (the river is no more covered with ice only because there is Ice Mountains biome on one of the borders).

Note: it looks like a world generator bug.