Steps to recreate:
1. Create a superflat world
2. Run /worldborder set 31
3. Run against world border and notice that you do not collide with it directly

Code Analysis: (Yarn 1.18.1)
net.minecraft.world.border.WorldBorder$StaticArea

private void recalculateBounds() {
   ...
   this.shape = VoxelShapes.combineAndSimplify(
      VoxelShapes.UNBOUNDED,
      VoxelShapes.cuboid(
         Math.floor(this.getBoundWest()),
         Double.NEGATIVE_INFINITY,
         Math.floor(this.getBoundNorth()),
         Math.ceil(this.getBoundEast()),
         Double.POSITIVE_INFINITY,
         Math.ceil(this.getBoundSouth())
      ),
      BooleanBiFunction.ONLY_FIRST
   );
} 

Basically the world border is being rounded using Math.floor() & Math.ceil()
Although entity collisions work perfectly fine without the rounding. The voxelShape bounds are only used for entity collisions

The fix is simply to remove Math.floor() & Math.ceil()