Trying to take any armor with all possible enchants (except thorns as of yet) and add thorns III with the anvil kicks back the original item and book after attempting to generate. Attempts on single player dont have this problem, only on the server. Searching a couple of minecraft server forums i have noticed other people with the same problem. They say if you try enough times (over several hours) it will work with the pro4unb3 chest and legs and even the pro4feather4unb3 boots, but it won't work ever with a pro4resp3aqua1unb3 helm.

If it works in single player, then I guess it is something about how the server handles the item generation. I hope this can be resolved.