The flags list accepts float and integer numbers list while you run command, Judging by the name "flags" , It should only accept boolean list.

How to reproduce:

1. Take any item in your main hand.
2. Run those commands:

   /item modify entity @s weapon.mainhand {"function":"minecraft:set_custom_model_data","flags":{"values":[0.2,-0.9,1.0,-100.0,23.0],"mode":"replace_all"}} 
   /item modify entity @s weapon.mainhand {"function":"minecraft:set_custom_model_data","flags":{"values":[-2,-1,0,1,2],"mode":"replace_all"}}

    

3. Check the command result, or use this command to check the result:

   /data get entity @s Inventory[0].components."minecraft:custom_model_data".flags

Expected Result:

It should only accept boolean list.

Observed Result:

The command allows number list.