My last two forts, I'd started digging quick -- by summer I'd opened up a barracks, three large (10x10) stockpile areas, a dining hall, a workshop area (14x14) with a 10x10 storage area underneath, and a tunnel to the river. None of it was smoothed or anything, but most of it was in rock, not loam or silt or clay or whatever.
Helps that I had 5 miners, but still. ;)
Anyway, I doubled my population overnight, and then got the usual immigration wave later in the year after the caravan, which took me to 28, I think. Crazy.