Being stuck at your current population limit so no new units can be trained. Each House adds 5 to the limit, the Town Center supports 5 and a Castle supports 20.
Getting housed stops villager production, which delays everything else. Players build a House slightly before they need it, often by watching the population count rather than waiting for the warning.