When upgrading a world with the vanilla datapack disabled (but a datapack that has all of the necessary parts besides banner_patterns is enabled) all banners cease to have patterns or be dyable in the crafting table or loom until the world is reloaded.
This effects both single and multiplayer.

Steps to recreate:

1. Make a world in 1.20.4 or earlier
2. Make a datapack using the version 1.20.6 datapack and remove the banner_patterns folder, or use the attached banner_test datapack (vanilla but with nothing in the banner_patterns folder)
3. Create various banners with various patterns and place them, to see the effect immediately upon reopening the world
   3a. Optionally save the banners to a creative toolbar
4. Disable vanilla and enable your other vanilla based datapack, simply removing the contents of banner_patterns does not replace vanilla's contents with nothing.
5. Open the world in 1.20.6 and observe every banner has gone blank and banners can no longer be dyed in a loom, this is expected
6. Replace banner_patterns in your datapack and do /reload, observe how even saved banners remain blank, no banners can be dyed, and the minecraft:banner_patterns component always gives errors.
   6a. Or simply disable your datapack and re-enable vanilla, observe the same issues persisting.
7. Exit the world and enter it again, observe banners behaving as expected again.

I don't know if the banners losing components is easily fixed, but I'd suggest a possible fix to the greater issue of data driven components losing data could be using the vanilla datapack's components as a backup for when they're missing from the enabled packs as to not lose saved banner patterns (and in the future, enchantments and paintings) and not break them entirely, even if temporarily before a full reload.

I know that simply being better prepared when upgrading a world would largely solve the issue of losing data, but since it doesn't show up as an error until you load a banner I was more concerned about being able to open my world than making sure I didn't lose any data (but with this knowledge I will be absolutely certain to not make this same mistake when upgrading in the future). If for some reason it's impossible to make /reload load the banner_patterns (and later, enchantments), perhaps a "[!] Cannot load banner_patterns from reload, please restart your world for the changes to take effect." to save people the pain of needing to replace all their banners or enchanted items or paintings in this context.