I had total 14 villagers in the trading hall, some of them were masters in trades, they all had their different beds and workstations, only one of them was roaming and the rest were inside the trading hall only within one Minecraft day all the villagers inside the hall disappeared except the one roaming outside and stuck in the animal pen.