Loading and unloading a resource pack does not release the memory allocated for those resources anymore, it seems. This was already described e.g. in MC-11439

On initial startup my ram usage is at 2.2 GB, loading and unloading a resource pack then bumped it up by a few hundred MB, only dropping down slightly after unloading. This also consecutively adds up, such that I could load and unload the same texture pack multiple times, pushing the RAM usage up to 5 GB. Waiting a few minutes afterwards dropped it down to 4.5GB again, where it then stayed.

All tested without even loading up a world, in the start menu.