The inventory system acts weird on a lot of servers I play but I don't think the servers are the problem. I was trying to build on a OP only area while I wasn't OP. The blocks that I placed got as expected immediately removed. But they didn't appear inside my inventory untill I updated that inventory slot by clicking on it or collecting an item that stacks with that slot. It is very strange.