you should be able to cross reference an Adventurer who starts from the fort for the adventurer's who don't but really it just aligning the Citizen's id (the unit's Civ, Population, and probably group ids to match with the starting citizen's ids everything else seems to be not really needed) with the rest of the fort once your in Fort mode.
I do remember someone making a script that converts units to your Fort might be Called Makeown.
Thanks. Googling found the makeown command in the tweak plugin, but that is not sufficient to make my adventurer accept labor assignments. (it sets the civ and some flags on the unit - resident, etc) I suspect something has changed in the game that renders that command irrelevant. Based on a hint in the forums I found what I posted previously, which does work, however I'm not sure if I'm just missing something that is not completely working. If anyone can tell me whether it's possible to change an existing reference using gui/gm-editor, that would be helpful.
edit: I just had an epiphany what a nemesis is: it is the corresponding unit id for the historical figure... not sure why anything worked without it, but hey.