Wow, I guess I got cocky there. I had ~30 zombies with me, I slept in the open...
"Halt in the name of the Quiet Devil!" or something like that. Seemed like a (well named) authority for the area. 10 goblin lashers, crossbowmen, bowmen, macemen, an so on. Devastated my zombies and me
Oh well, time to try again
EDIT: I think I became an enemy of theirs because I accidentally drained a guy of blood when feeding. I should've stuck to suffocating and feeding on the dwarves in lairs