My own code is pretty crufty and I've never worked with threading, so I'll leave that aspect of the problem to more qualified persons.
However, I feel I should interject on the debate over how the pathing could be made more efficient.
It seems to my observations, that a Dwarf only recalculates his path if it becomes blocked, or another moving unit collides with him and grounds him. Until then some system just checks to be sure his existing path is still valid (since he changes routes before reaching a freshly locked door, and can go around other units, if the hallway is wide enough).
Somewhere in every fortress is a hallway one tile narrower than it should be, just waiting for the population to grow large enough for it to completely destroy your FPS with a continuous traffic jam. I can't count the number of times I've had a fortress humming along, only to have the latest litter of kittens or the new wave of migrants overload my road system and drive my FPS down between 3 and 9, from 35 to 50.
I can see "thinking" mark appear on the grounded dwarves, so I know they're re-pathing whenever they trip over a kitten. I believe that if they would just try their old path again they would reach their destinations without rendering my game unplayably slow. Surely whatever gains we get from discarding paths that result in collisions can't outweigh the cost of re-calculationf them everything time we bump into someone going the other way. Once a dwarf is caught in a traffic jam, the only way out is to keep pressing forward, anyway.
Dwarves might have to be given a way do differentiate between grounded by traffic and grounded by other means, and their old path should probably be re-checked; we don't have the code, so it's hard to say anything definite about how to actually do the thing.
Really, there's no way for anyone other than Toady to implement this, so the argument is pretty much academic, but it seems to me that eliminating all these pathings would be more effective than making each pathing slightly faster.
I'm a multithreaded, single core and unlikely to upgrade in the next few years, BTW.