After looking at the dev reports lately, I've been thinking about something that may help even out the conflicts a bit and keep the world a little less conquered by the time the actual game rolls around.
Native terrain bonuses. Basically, Dwarves and Goblins do best in mountainous regions, humans on the plains, and elves in forests. Swamps should always give the defender an advantage...hills too, probably. I imagine you're already giving defenders a bonus anyway, but maybe that could be made a little more specific? For example, give sites features like walls, archer towers, traps, moats, etc, that all make it harder for an attacker to win. Might be interesting to decide which races do what to help their defenses. Dwarves are obviously expert trap builders. Perhaps humans rely more on walls and moats? Elves on scattered archer towers and pit traps? Goblins I could see having long walls of spikes around their towers to funnel enemies into kill pockets.
Historically, I think it was a rule of thumb that you had to have 3-4 times the number of attackers to take over a castle. Obviously, a small unwalled village is a different story.
Thinking about this sort of thing, it might also make sense in the normal combat system to give a combatant that is higher up then their opponent some sort of bonus (if they don't already get one). That way, when fighting on a slope or up a ramp/stairs, the fighter on higher ground is more likely to win. This is true in real life, after all. I've done enough real-life sword/spear fighting to know that having the high ground isn't just an expression!