I have created a server using the 22w17a server jar. This bug is not present in multiplayer servers.
Even though I using a skin I made, in 22w17a it is using the default skin.
Steps to Reproduce:
Select a custom skin in the Minecraft Launcher
Open Minecraft Java Edition and join a singleplayer world.
You won't be using your skin and the default skin is used.
You will be using your skin that is selected in the launcher.