Bay 12 Games Forum

Dwarf Fortress => DF Modding => Topic started by: IndigoFenix on February 27, 2020, 06:02:44 am

Title: Historical figure personality analysis science
Post by: IndigoFenix on February 27, 2020, 06:02:44 am
I've been trying to figure out why historical figures do the things they do, the better to mod their behaviors through secrets (that modify personalities) or base creature personality traits.

I haven't done much exploring yet, but I did write a DFHack script for scanning the brain of historical figures in Legends Mode.  Feel free to run it and see if you can find any patterns!  Unfortunately it seems DFHack has not been updated for the new version yet (I think?), so villain behaviors are still a no-go.  It should work once we get an update though, and for now it's possible to use in previous DF versions.

Here is the script. (https://github.com/IndigoFenix/Brainscan/blob/master/brainscan.lua)

EDIT: Updated the script to allow getting the average of all units in the world that meet particular criteria.  Results seem disappointing, so far vampires, werebeasts, and necromancers all seem to happen randomly, with no impact from the unit's personality.

EDIT2: Updated to take even more variables, and to also check average ethical values, but to no avail; as far as statistics suggest forming relationships of any kind or writing books are entirely random; when scanning all units with these criteria no personality trait or value is more than a fraction of a percent away from average.  Books do suggest some of the author's personality traits, but I think we could have already guessed that.  Maybe the villain update has more in-depth behavior, the fact that generated necromancy secrets alter their holder's personality suggests that personality does something, at least.

EDIT3: Added scanning for entity positions and war leaders, still nothing.  On the whole, this has been a very disappointing project, it appears that personality and values are basically cosmetic, as far as historical figures doing things in worldgen is concerned, unless things were changed substantially for the villain update, but that will have to wait until DFHack has been updated to test properly.

EDIT4: Tested it out with the alpha DFHack build.  It still seems like personalities and values don't have a significant effect on worldgen events, at least not in ways that can be viewed by looking for average trends.  I guess the script could still be nice for storytelling purposes though, since you can use it to find out things about legendary figures that aren't directly apparent from the legends screen.
Title: Re: Historical figure personality analysis science
Post by: Dwarf_Fever on February 27, 2020, 05:41:25 pm
I've been trying to figure out why historical figures do the things they do, the better to mod their behaviors through secrets (that modify personalities) or base creature personality traits.

I'm ok with worldgen events not being at the whims of the leaders, though in either case it would be nice to have some political depth to it.

Incidentally, if you're just taking a census, how would you relate that to the events, or why would you expect leaders to be noticeably different from their species, outside of maybe the results of a tendency for inbreeding?
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on February 28, 2020, 02:04:03 am
I've been trying to figure out why historical figures do the things they do, the better to mod their behaviors through secrets (that modify personalities) or base creature personality traits.

I'm ok with worldgen events not being at the whims of the leaders, though in either case it would be nice to have some political depth to it.

Incidentally, if you're just taking a census, how would you relate that to the events, or why would you expect leaders to be noticeably different from their species, outside of maybe the results of a tendency for inbreeding?

Species personality variations only impact the average - the actual distribution is highly variable so each creature is unique.  You'll find units with 100 or 0 of each trait (apart from goblins, who have some traits locked above or below 50) but if you average them all out, you'll arrive at the species average.  (This is how I tested the averaging script; if you scan all -race DWARF or -race ELF the average value is very close to the species average as defined in the raws.)  Genetics does not appear to play a role in personality, since if it did you'd wind up with skewed values when averaging out all units in a given world, but as it is the values are all close to neutral.

The reason I would expect to find trends is because I would expect that some personality traits would be responsible for units making particular choices.  For example, I expected to find above-average AMBITION among position holders, because I expected that units with higher ambition would be more likely to create plots that would put them in power, especially with the villain update.  Similarly I would expect to find above-average BRAVERY or VIOLENT values among those who lead battles, or PRIDE and VANITY among those who become obsessed with immortality and therefore become necromancers.

It seems that worldgen basically chooses these units at random, and then sometimes uses their personalities to retroactively invent a reason for them doing the things they do to spice up the flavor text.  Unless there's a test I haven't thought of yet.

Toady has said that personality traits and values figure into villain behavior, which is why I wanted to modify personalities in the first place, but with no explanation of which traits cause which behaviors the only way to check scientifically is to look for average trends among units who perpetrate certain deeds, and so far I have found no such trends.

To be fair I've been searching for the results of villainous plots, not the number of attempts.  It's possible that villains with high AMBITION (or some other traits) regularly scheme to put themselves into power, but so few succeed that any trends are lost in the noise.  It is possible that skills may impact the success of these plots, but the problem with scanning for skills is that it's a chicken-and-egg problem.  Units with positions do have above-average social skills, but I can't tell if they reached their positions due to their skills or if they have developed social skills due to their position.
Title: Re: Historical figure personality analysis science
Post by: Rumrusher on February 28, 2020, 08:04:08 am
scheming is a skill a unit can learn so it's possible that you could force this by making some naturally incline to scheme.
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on February 28, 2020, 10:11:07 am
scheming is a skill a unit can learn so it's possible that you could force this by making some naturally incline to scheme.

Skills can't be added through syndromes though.  And making a species with natural scheming ability wouldn't help, because the secret-learners would be competing with the other schemers in their civ.

Actually, secret keepers have a higher-than-average likelihood to take positions of power already (usually about a third to half of the secret keepers in a world also have entity positions), though it isn't really clear why this happens, since non-secret-keepers with the same natural personality traits don't have a greater tendency to get into power.  Unless it's hard-coded.  Or unless I have things backwards, and people in power are more likely to get access to secrets.

Right now I'm more interested in figuring out how to make them more likely to start wars, make peace agreements, take apprentices or write books to spread their secret further, and so on.  Unfortunately I haven't figured out how to scan all of the historical events a unit has been involved in, which would help figure out if personalities impact those.
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on March 08, 2020, 05:26:53 am
So lacking a better method, I've just been scouring Legends Mode by hand to try and find patterns.
It's extremely annoying, because sometimes I can't tell if DF is being brilliant or if it's just random.
For example, I found a human civ leader who was constantly making war on a nearby elven civ.  So I scanned the leader's brain and found that he had the lowest VIOLENT level possible, a dream of BRING_PEACE_TO_THE_WORLD, and no notable traits or values that would suggest any kind of war-happy behavior.  He did have high BRAVERY and EXCITEMENT_SEEKING but that was all.
Meanwhile, the queen of the elves had a very high VIOLENT level, and the princess had a very high CRUELTY level.  Yet the humans were the aggressors every time.  (In fact several times, the elves tried to create a treaty to support the humans in war...only to be attacked the next year.)
So...did the peace-loving human lawgiver recognize that the elf queen was a threat to world peace and bravely act to destroy the threat?  Or is it just randomness?
Title: Re: Historical figure personality analysis science
Post by: FantasticDorf on March 08, 2020, 05:59:31 am
I have a dwarf migrant with skilled schemer (as a lowly woodcutter) upon arrival and a 'Gain Rank in Society' goal so the things they do are definitely driven by their long-term life goals. Most of all the migrants with 'produce a great work of art' / 'master a skill' are in strictly non-craft fields but still have a manual profession like pressing, while all the ones with masterwork usually are somewhat versed in those fields; I would not be suprised in my own experience whether if you had any 'legendary warriors' join a adveturer or merceary group (its common amongst humans with their broad beliefs)

These ones on this list are still interesting, batheing the world in chaos seems like something Demons or a [POWER] creature like a minotaur would have (http://dwarffortresswiki.org/index.php/DF2014:Personality_trait#Goals). People who want to raise a family usually already do have families, quite extensive ones.

Obsession upon achieving immortality usually precludes, whether by right mixture of beliefs or a preassigned thing.
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on November 27, 2021, 06:25:48 pm
So, getting back to try a different approach.

Rather than operate through DFHackery, I decided to try using advanced worldgen parameters to spin up a small worldgen paramset with minimal features - a flat plain, with glaciers in the north, tropics in the south, and flat plains in between.  I removed all megabeasts, night creatures, secrets and savagery in an effort to make the setup as invariable as possible.  I then created two civilizations, Ice Giants and Fire Giants, both of them physically identical, with no need for food and drink, and gave them the ability to settle anywhere; differentiated only by one starting in the polar regions and the other in the tropics.  The perfect setting to watch their civs spread across the world and clash in the middle during worldgen.

From there, I began the process of adjusting their traits in an effort to see what would impact worldgen events.

Adjusting ethics to make them opposed to each other worked fine.  If their ethics were similar, wars rarely occurred in worldgen; if their ethics were very different, they would be more likely to go to war.

Differing VALUES didn't appear to do anything noticeable.

After a very many, many tests, I finally found something besides ethics that seemed to make an impact.  IF there was hatred due to differing ethics, giving BOTH sides max-level CONFIDENCE, BRAVERY, VENGEFUL, and AMBITION seemed to increase the chances of conflicts when the two civs met.  This could be viewed by the expansion of both civs grinding to a halt as they met in the middle, the world population would show a noticeable increase in deaths, and this would often cause forts to pop up along the front lines.  Sometimes, one civ would end up taking over the other.  If these personality traits were low, it would be more likely that the civs would simply cross over and build sites on each others' territories, even if they hated each other.

Notably, if only ONE civ had these traits, they would not clash visibly.

This is, so far, the first evidence I have found of personality traits making an impact on worldgen behavior.
Title: Re: Historical figure personality analysis science
Post by: FantasticDorf on November 28, 2021, 07:09:32 am
If these personality traits were low, it would be more likely that the civs would simply cross over and build sites on each others' territories, even if they hated each other.

Notably, if only ONE civ had these traits, they would not clash visibly.

This is, so far, the first evidence I have found of personality traits making an impact on worldgen behavior.

I think this may also be why player fortresses are often clogged by elven and goblin economic sites especially who settle nearby, which is a very annoying ongoing issue, while humans and at times dwarves (the intended hill dwarves) don't seem to be all that bothered to come near, and frustratingly lock players out of ordering extra dwarf population when they become a barony.

Other stuff i've heard about but not been able to substantiate are personal leader propensities like anger_propensity, acting in a similar vein to those cultural values to override strength considerations to throwing their lot into wars they've sized up they probably can't win.

Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on November 28, 2021, 07:45:05 am
It's very hard to demonstrate a difference in an objective manner, because of the complexity of stuff that can happen in worldgen. I'll run more tests on these bare-bones settings to see if I can find any other patterns.

What's upsetting is that ethic differences are by far the most significant factor in determining whether two civs would fight. I would hope that sufficiently power-hungry leaders would just start wars for territory or personal grudges even without a moral excuse to justify it.

Though I don't think that's entirely impossible. Wars over land ownership, livestock grazing or water rights, and formalized agreements are in the game, but I haven't figured out what triggers them, or if there's a way to make them more likely.
Title: Re: Historical figure personality analysis science
Post by: brewer bob on November 28, 2021, 08:15:22 am
PTW, seems interesting -- been wondering if personalities have any effect during worldgen.
Title: Re: Historical figure personality analysis science
Post by: FantasticDorf on November 28, 2021, 12:53:54 pm
It's very hard to demonstrate a difference in an objective manner, because of the complexity of stuff that can happen in worldgen. I'll run more tests on these bare-bones settings to see if I can find any other patterns.

What's upsetting is that ethic differences are by far the most significant factor in determining whether two civs would fight. I would hope that sufficiently power-hungry leaders would just start wars for territory or personal grudges even without a moral excuse to justify it.

Though I don't think that's entirely impossible. Wars over land ownership, livestock grazing or water rights, and formalized agreements are in the game, but I haven't figured out what triggers them, or if there's a way to make them more likely.

Formalized agreements tend to be a declaration of war in general, prompted by attack trigger. Retro-looking at wars put upon the player by goblins (because dwarves dont declare war, the goblins have to make the first move in reasoning the right time to attack) reveals often this is the casus belli. Disturbing eternal rest comes from mummies and deep risen sieges.

The others (water and grazing rights) i've seen to be inter-civ wars, like groups of humans where the values are randomized enough where they occasionally be abrasive (maybe territorially too close?) and military strategy is much more reduced away from the 'monarch' into site generated lords around towns and keeps for them to make a individual decision for theselves, though i have actually only seen it predominantly with human towns so it might say something for site-type to matter irregardless of who's in charge of it.

You can have a deep risen siege come and occupy a mountain-hall for instance, wherin the enemy will be inside the mountain being capstoned by a dwarf-fort blocking the exit because mountainhomes have a internal tunnel-road leading between forts, as do individual forts link to each other in this way. Point being that unless they're particularly brave, often they get trapped and stay within the mountain forever as a threat kept within.
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on November 28, 2021, 04:58:03 pm
So I've found some more stuff.

Although ethics play the largest role in determining whether two civs go to war, with the right personalities it is possible to make civs fight reliably even when they don't have an ethical issue with each other.  The game will generally look for an ethical excuse to explain the cause of the conflict, but if it can't find one it will say "the cause of the conflict is debated by scholars and little is truly known."  I have managed to generate worlds where one race completely exterminates the other though a series of wars, none of which are given an explicit reason.

The likelihood of total war increases greatly if one side significantly outnumbers the other when they first encounter each other (which tends to be pretty random).  This creates an unstable equilibrium, as the side that is more numerous is also more likely to win.  Most worlds either result in both sides colliding and remaining at a level of occasional skirmishes forever, or one side completely overwhelming the other within a century or two.

Personality traits that might improve chances of conflict: HATE_PROPENSITY, ENVY_PROPENSITY, GREED, AMBITION, THOUGHTLESSNESS, and CONFIDENCE.  It's hard to narrow down, but wars definitely seem more likely with these traits involved.

Beyond simple disagreement, ethics appear to make an impact on worldgen behaviors, which impact the chances of wars.  Torture and theft related ethics - possibly others - all seem to increase the chances of conflict if ACCEPTABLE, even if both races are okay with it.  I suppose that finding these ethics ACCEPTABLE means that they don't care about doing it to others - but having their items stolen and their citizens tortured by foreigners still makes them angry.  Hypocrites.

This creates a kind of balance - races with strict ethics are more likely to be outraged by races that don't follow them, but those with loose ethics are more likely to aggravate each other through their general behavior.  Presumably NOT_APPLICABLE, rather than ACCEPTABLE, is intended for societies that have no concept of things like property, territory, or pain.

KILL_NEUTRAL and KILL_ENEMY ethics are, not surprisingly, disproportionately important when it comes to starting or continuing wars.  Making both REQUIRED greatly increases the chances of conflict.  Making KILL_ENEMY forbidden turns battles rather interesting - pacifists will never be the aggressors and they will never be recorded as attacking an enemy, but somehow the attackers can still suffer losses, possibly animal-inflicted.

Conflicts over territory, water, grazing rights, and livestock ownership seem to occur mostly between sites within a civilization, but I have seen some inter-civ conflicts.  It is not used as a reason when no other reason is available (that would be the above "debated by scholars" line), it appears to be a real in-game event.  Loose property-related ethics may play a role here - I have seen these wars erupt while testing loose ethic behavior, but when both races have strict ethics they do not happen.
Title: Re: Historical figure personality analysis science
Post by: brewer bob on November 28, 2021, 06:04:17 pm
Personality traits that might improve chances of conflict: HATE_PROPENSITY, ENVY_PROPENSITY, GREED, AMBITION, THOUGHTLESSNESS, and CONFIDENCE.  It's hard to narrow down, but wars definitely seem more likely with these traits involved.

So, adding boosts to these with a custom secret might increase the chances of the civ going to war, if the leader of the civ has learned the secret (presuming the leader's personality matters)?
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on November 29, 2021, 01:13:09 pm
Personality traits that might improve chances of conflict: HATE_PROPENSITY, ENVY_PROPENSITY, GREED, AMBITION, THOUGHTLESSNESS, and CONFIDENCE.  It's hard to narrow down, but wars definitely seem more likely with these traits involved.

So, adding boosts to these with a custom secret might increase the chances of the civ going to war, if the leader of the civ has learned the secret (presuming the leader's personality matters)?

See, that's what's baffling me.  While my latest tests do demonstrate that personality - at least of an entire species - can have an impact on worldgen events, it doesn't change the fact that my original method of testing - actively scanning the personality of every unit associated with a particular life event in Legends mode and then averaging them out in order to see if they deviated from the average - completely failed to find ANY connection between a personality trait and making any kind of decision, be it writing a book, angering a god, getting married, taking a position, and so on.  (I know the code is finding the right units, because it DID find connections between SKILLS and events - nobles had above-average social skills, warriors had above-average combat skills, and the average of units with a particular profession had high skill levels in that profession.)

That script would be a much better way of doing personality science, but either there's something really weird about how worldgen associates behaviors with units, or something is wrong with the script itself.  I'd like to see if any talented DFHackers could maybe check it out and see if they could find out what I'm doing wrong.
Title: Re: Historical figure personality analysis science
Post by: Quietust on November 30, 2021, 01:13:33 pm
Something that might be of interest to you: during my analysis of version 0.28.181.40d, I discovered that a few specific personality traits did indeed have an impact on worldgen behavior:
* High values of EXCITEMENT_SEEKING / ACHIEVEMENT_STRIVING and low values of CAUTIOUSNESS would make civ leaders more likely to declare wars in general
* High values of EXCITEMENT_SEEKING / ACHIEVEMENT_STRIVING / ANGER / SELF_EFFICACY would make civ leaders more likely to declare wars, even in the face of insurmountable odds

I managed to locate the same logic in version 0.47.05, and it's very similar:
* High values of CONFIDENCE, AMBITION, THOUGHTLESSNESS, and EXCITEMENT_SEEKING increase the chances of declaring wars in general
* High values of CONFIDENCE, AMBITION, THOUGHTLESSNESS, and ANGER_PROPENSITY increase the chances of declaring "unwinnable" wars
There's an additional check against EXCITEMENT_SEEKING and AMBITION by itself, and it seems to be in the general proximity of code for establishing alliances.
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on November 30, 2021, 03:46:34 pm
Something that might be of interest to you: during my analysis of version 0.28.181.40d, I discovered that a few specific personality traits did indeed have an impact on worldgen behavior:
* High values of EXCITEMENT_SEEKING / ACHIEVEMENT_STRIVING and low values of CAUTIOUSNESS would make civ leaders more likely to declare wars in general
* High values of EXCITEMENT_SEEKING / ACHIEVEMENT_STRIVING / ANGER / SELF_EFFICACY would make civ leaders more likely to declare wars, even in the face of insurmountable odds

I managed to locate the same logic in version 0.47.05, and it's very similar:
* High values of CONFIDENCE, AMBITION, THOUGHTLESSNESS, and EXCITEMENT_SEEKING increase the chances of declaring wars in general
* High values of CONFIDENCE, AMBITION, THOUGHTLESSNESS, and ANGER_PROPENSITY increase the chances of declaring "unwinnable" wars
There's an additional check against EXCITEMENT_SEEKING and AMBITION by itself, and it seems to be in the general proximity of code for establishing alliances.

Are you able to actually see the code?  How did you figure all this out?
Also, can you figure out why the brainscan script can't find any patterns?
Though, I guess if personality is only relevant for leaders making whole-civ or whole-site decisions, then it would make sense (there are never enough leaders to find a meaningful average).  Still, I would expect that personality should play a role in individual behaviors other than starting wars or making alliances.
Title: Re: Historical figure personality analysis science
Post by: Quietust on November 30, 2021, 04:25:34 pm
Are you able to actually see the code?  How did you figure all this out?
I identified the function for creating history_event_collection_warst (which is extremely easy to find because of runtime type information, or RTTI), and I traced that backwards to find the function used for declaring wars (or accepting/refusing peace treaties) during worldgen, then I found the function call that takes a unit_personality and an array of personality facet weights and figured out which weights were being assigned.

Of course, this was only possible because I had already located the relevant code in version 0.28.181.40d so I knew exactly what I was looking for.

The code itself is just x86 assembly language with some annotations, looking mostly like this:
Code: [Select]
1405031ea ad8    48 8d 8d b0     LEA      RCX=>weights2,[RBP + 0x1b0]
                   01 00 00
 1405031f1 ad8    e8 2a 91 f8 ff  CALL     init_personality_weights                                    void init_personality_weights(in
 1405031f6 ad8    b8 01 00 00 00  MOV      EAX,0x1
 1405031fb ad8    89 85 68 02     MOV      dword ptr [RBP + weights2[46]+0x9d8],EAX
                   00 00
 140503201 ad8    89 85 fc 01     MOV      dword ptr [RBP + weights2[19]+0x9d8],EAX
                   00 00
 140503207 ad8    89 85 04 02     MOV      dword ptr [RBP + weights2[21]+0x9d8],EAX
                   00 00
 14050320d ad8    89 85 50 02     MOV      dword ptr [RBP + weights2[40]+0x9d8],EAX
                   00 00
 140503213 ad8    4c 8b 6d f0     MOV      R13,qword ptr [RBP + local_9e8+0x9d8]
 140503217 ad8    49 8b 8d 30     MOV      RCX,qword ptr [R13 + 0x130]
                   01 00 00
 14050321e ad8    48 8d 95 b0     LEA      RDX=>weights2,[RBP + 0x1b0]
                   01 00 00
 140503225 ad8    48 8b 49 18     MOV      RCX,qword ptr [RCX + 0x18]
 140503229 ad8    e8 b2 d8 40 00  CALL     unit_personality::getOverallScore                           int getOverallScore(unit_persona
and this:
Code: [Select]
14050326e ad8    48 8d 8d b0     LEA      RCX=>weights2,[RBP + 0x1b0]
                   01 00 00
 140503275 ad8    e8 c6 90 f8 ff  CALL     FUN_14048c340                                               undefined FUN_14048c340()
 14050327a ad8    b8 01 00 00 00  MOV      EAX,0x1
 14050327f ad8    89 85 68 02     MOV      dword ptr [RBP + weights2[46]+0x9d8],EAX
                   00 00
 140503285 ad8    89 85 fc 01     MOV      dword ptr [RBP + weights2[19]+0x9d8],EAX
                   00 00
 14050328b ad8    89 85 04 02     MOV      dword ptr [RBP + weights2[21]+0x9d8],EAX
                   00 00
 140503291 ad8    89 85 c4 01     MOV      dword ptr [RBP + weights2[5]+0x9d8],EAX
                   00 00
 140503297 ad8    49 8b 8d 30     MOV      RCX,qword ptr [R13 + 0x130]
                   01 00 00
 14050329e ad8    48 8d 95 b0     LEA      RDX=>weights2,[RBP + 0x1b0]
                   01 00 00
 1405032a5 ad8    48 8b 49 18     MOV      RCX,qword ptr [RCX + 0x18]
 1405032a9 ad8    e8 32 d8 40 00  CALL     unit_personality::getOverallScore                           int getOverallScore(unit_persona
(the indices on the "weights" and "weights2" arrays identify which traits are being evaluated - the "unit_personality::getOverallScore" function is being called against historical_figure.info.personality)

Also, can you figure out why the brainscan script can't find any patterns?
I haven't looked at your script, so I don't know why it wouldn't have found anything.
Title: Re: Historical figure personality analysis science
Post by: IndigoFenix on December 01, 2021, 12:54:24 am
--snip--

Oh, wow. So is there a way of translating that into math? Is it just adding up the values of those four traits?

And is there a way of doing something similar for other events?  War was one that, as I said, I wasn't able to average out properly because there are too few war declaring rulers per world. But I did try looking for history_event_collection_theftst and history_event_collection_abductionst and could find no statistically significant deviation from the average kobold or goblin personality.

I'm not sure exactly which events are tied to book writing, seeking immortality, or defiling temples, but when I scan a specific race for units that wrote at least one book, know at least one secret, or have a vampire curse (provided there are enough of them) I could find no deviations from the norm of that race, suggesting that whatever factors are involved in selecting these, personality and values are not among them, which seems wrong.  There were also no connections between personality and whether one had a spouse or a lover (I thought maybe LOVE_PROPENSITY would play a role), a grudge (ditto for HATE_PROPENSITY), the presence of any other relationship, or the number of said relationships.

All my script does is scan all historical figures for individuals that meet a certain set of parameters (for instance, scan all units with race HUMAN and who wrote at least one book), add up all their personality, value, and skill traits, and then divide those by total units scanned to find the average.  The theory being that units associated with a particular action should have the traits involved in that decision be higher/lower than their racial average.  I know I'm getting the right units because their SKILLS deviate from the average as expected (if I scan for all cheesemakers, they have, on average, a very high cheesemaking skill) but as far as I can tell, personality and values are irrelevant.
Title: Re: Historical figure personality analysis science
Post by: Quietust on December 01, 2021, 08:44:23 am
So is there a way of translating that into math? Is it just adding up the values of those four traits?
The actual math is done inside this pair of functions:
Code: [Select]
                             int __thiscall unit_personality::getOverallScore(unit_pe
             int               EAX:4           <RETURN>
             unit_personali    RCX:8 (auto)    this
             int32_t[50] *     RDX:8           weights
                              unit_personality::getOverallScore
 140910ae0   0    48 89 5c 24 08  MOV      qword ptr [RSP + local_res8],RBX
 140910ae5   0    48 89 6c 24 10  MOV      qword ptr [RSP + local_res10],RBP
 140910aea   0    48 89 74 24 18  MOV      qword ptr [RSP + local_res18],RSI
 140910aef   0    57              PUSH     RDI
 140910af0 008    48 83 ec 20     SUB      RSP,0x20
 140910af4 028    33 f6           XOR      ESI,ESI
 140910af6 028    48 8b fa        MOV      RDI,RDX
 140910af9 028    0f b7 de        MOVZX    EBX,SI
 140910afc 028    48 8b e9        MOV      RBP,RCX
 140910aff 028    90              NOP
                              LAB_140910b00                                      XREF[1]:   140910b34(j) 
 140910b00 028    44 8b 07        MOV      R8D,dword ptr [RDI]
 140910b03 028    45 85 c0        TEST     R8D,R8D
 140910b06 028    7e 0f           JLE      LAB_140910b17
 140910b08 028    0f bf d3        MOVSX    EDX,BX
 140910b0b 028    48 8b cd        MOV      RCX,RBP
 140910b0e 028    e8 3d 00 00 00  CALL     unit_personality::checkTrait                                int checkTrait(unit_personality
 140910b13 028    03 f0           ADD      ESI,EAX
 140910b15 028    eb 12           JMP      LAB_140910b29
                              LAB_140910b17                                      XREF[1]:   140910b06(j) 
 140910b17 028    79 10           JNS      LAB_140910b29
 140910b19 028    41 f7 d8        NEG      R8D
 140910b1c 028    0f bf d3        MOVSX    EDX,BX
 140910b1f 028    48 8b cd        MOV      RCX,RBP
 140910b22 028    e8 29 00 00 00  CALL     unit_personality::checkTrait                                int checkTrait(unit_personality
 140910b27 028    2b f0           SUB      ESI,EAX
                              LAB_140910b29                                      XREF[2]:   140910b15(j), 140910b17(j) 
 140910b29 028    66 ff c3        INC      BX
 140910b2c 028    48 83 c7 04     ADD      RDI,0x4
 140910b30 028    66 83 fb 32     CMP      BX,0x32
 140910b34 028    7c ca           JL       LAB_140910b00
 140910b36 028    48 8b 5c 24 30  MOV      RBX,qword ptr [RSP + local_res8+0x28]
 140910b3b 028    8b c6           MOV      EAX,ESI
 140910b3d 028    48 8b 74 24 40  MOV      RSI,qword ptr [RSP + local_res18+0x28]
 140910b42 028    48 8b 6c 24 38  MOV      RBP,qword ptr [RSP + local_res10+0x28]
 140910b47 028    48 83 c4 20     ADD      RSP,0x20
 140910b4b 008    5f              POP      RDI
 140910b4c   0    c3              RET

...

                             int __thiscall unit_personality::checkTrait(unit_persona
             int               EAX:4           <RETURN>
             unit_personali    RCX:8 (auto)    this
             personality_fa    EDX:4           type
             int               R8D:4           rolls
                              unit_personality::checkTrait
 140910b50   0    48 89 5c 24 10  MOV      qword ptr [RSP + local_res10],RBX
 140910b55   0    57              PUSH     RDI
 140910b56 008    48 83 ec 20     SUB      RSP,0x20
 140910b5a 028    41 8b d8        MOV      EBX,R8D
 140910b5d 028    4c 63 c2        MOVSXD   R8,EDX
 140910b60 028    48 8b 91 58     MOV      RDX,qword ptr [RCX + 0x158]
                   01 00 00
 140910b67 028    42 0f b7 84     MOVZX    EAX,word ptr [RCX + R8*0x2 + 0x80]
                   41 80 00 00 00
 140910b70 028    48 85 d2        TEST     RDX,RDX
 140910b73 028    74 16           JZ       LAB_140910b8b
 140910b75 028    66 42 03 04 42  ADD      AX,word ptr [RDX + R8*0x2]
 140910b7a 028    79 04           JNS      LAB_140910b80
 140910b7c 028    33 c0           XOR      EAX,EAX
 140910b7e 028    eb 0b           JMP      LAB_140910b8b
                              LAB_140910b80                                      XREF[1]:   140910b7a(j) 
 140910b80 028    66 83 f8 64     CMP      AX,0x64
 140910b84 028    7e 05           JLE      LAB_140910b8b
 140910b86 028    b8 64 00 00 00  MOV      EAX,0x64
                              LAB_140910b8b                                      XREF[3]:   140910b73(j), 140910b7e(j),
                                                                                             140910b84(j) 
 140910b8b 028    33 ff           XOR      EDI,EDI
 140910b8d 028    85 db           TEST     EBX,EBX
 140910b8f 028    7e 5d           JLE      LAB_140910bee
                              LAB_140910b91                                      XREF[4]:   14106e98c(*), 14106e99c(*),
                                                                                             141ef2ec0(*), 141ef2ec8(*) 
 140910b91 028    48 89 74 24 30  MOV      qword ptr [RSP + local_res8+0x28],RSI
 140910b96 028    0f bf f0        MOVSX    ESI,AX
 140910b99 028    ff c6           INC      ESI
 140910b9b 028    0f 1f 44 00 00  NOP      dword ptr [RAX + RAX*0x1]
                              LAB_140910ba0                                      XREF[1]:   140910be7(j) 
 140910ba0 028    83 fe 01        CMP      ESI,0x1
 140910ba3 028    77 04           JA       LAB_140910ba9
 140910ba5 028    33 c0           XOR      EAX,EAX
 140910ba7 028    eb 38           JMP      LAB_140910be1
                              LAB_140910ba9                                      XREF[1]:   140910ba3(j) 
 140910ba9 028    e8 12 b5 0e 00  CALL     mt_trandom                                                  int32_t mt_trandom(void)
 140910bae 028    44 8b c0        MOV      R8D,EAX
 140910bb1 028    b8 03 00 00 00  MOV      EAX,0x3
 140910bb6 028    41 f7 e0        MUL      R8D
 140910bb9 028    41 8b c0        MOV      EAX,R8D
 140910bbc 028    2b c2           SUB      EAX,EDX
 140910bbe 028    d1 e8           SHR      EAX,1
 140910bc0 028    03 c2           ADD      EAX,EDX
 140910bc2 028    33 d2           XOR      EDX,EDX
 140910bc4 028    c1 e8 1e        SHR      EAX,0x1e
 140910bc7 028    69 c0 ff ff     IMUL     EAX,EAX,0x7fffffff
                   ff 7f
 140910bcd 028    44 2b c0        SUB      R8D,EAX
 140910bd0 028    b8 ff ff ff 7f  MOV      EAX,0x7fffffff
 140910bd5 028    f7 f6           DIV      ESI
 140910bd7 028    33 d2           XOR      EDX,EDX
 140910bd9 028    8d 48 01        LEA      ECX,[RAX + 0x1]
 140910bdc 028    41 8b c0        MOV      EAX,R8D
 140910bdf 028    f7 f1           DIV      ECX
                              LAB_140910be1                                      XREF[1]:   140910ba7(j) 
 140910be1 028    03 f8           ADD      EDI,EAX
 140910be3 028    ff cb           DEC      EBX
 140910be5 028    85 db           TEST     EBX,EBX
 140910be7 028    7f b7           JG       LAB_140910ba0
 140910be9 028    48 8b 74 24 30  MOV      RSI,qword ptr [RSP + local_res8+0x28]
                              LAB_140910bee                                      XREF[3]:   140910b8f(j), 141ef2ecc(*),
                                                                                             141ef2ed4(*) 
 140910bee 028    8b c7           MOV      EAX,EDI
 140910bf0 028    48 8b 5c 24 38  MOV      RBX,qword ptr [RSP + local_res10+0x28]
 140910bf5 028    48 83 c4 20     ADD      RSP,0x20
 140910bf9 008    5f              POP      RDI
 140910bfa   0    c3              RET

Also, I mentioned that it's all x86 assembly, but Ghidra (https://ghidra-sre.org/) is also able to produce a reasonable approximation of C code:
Code: [Select]
int __thiscall unit_personality::getOverallScore(unit_personality *this,int32_t (*weights) [50])

{
  int iVar1;
  short sVar2;
  int iVar3;
 
  iVar3 = 0;
  sVar2 = 0;
  do {
    iVar1 = (*weights)[0];
    if (iVar1 < 1) {
      if (iVar1 < 0) {
        iVar1 = checkTrait(this,(int)sVar2,-iVar1);
        iVar3 -= iVar1;
      }
    }
    else {
      iVar1 = checkTrait(this,(int)sVar2,iVar1);
      iVar3 += iVar1;
    }
    sVar2 += 1;
    weights = (int32_t (*) [50])(*weights + 1);
  } while (sVar2 < 0x32);
  return iVar3;
}

int __thiscall
unit_personality::checkTrait(unit_personality *this,personality_facet_type type,int rolls)

{
  uint16_t uVar1;
  uint uVar2;
  int iVar3;
 
  uVar1 = (&(this->traits).LOVE_PROPENSITY)[type];
  if (this->temporary_trait_changes != NULL) {
    uVar1 += (&this->temporary_trait_changes->LOVE_PROPENSITY)[type];
    if ((short)uVar1 < 0) {
      uVar1 = 0;
    }
    else {
      if (100 < (short)uVar1) {
        uVar1 = 100;
      }
    }
  }
  iVar3 = 0;
  if (0 < rolls) {
    do {
      if ((int)(short)uVar1 + 1U < 2) {
        uVar2 = 0;
      }
      else {
        uVar2 = mt_trandom();
        uVar2 = (uVar2 % 0x7fffffff) /
                ((int)(0x7fffffff / (ulonglong)((int)(short)uVar1 + 1U)) + 1U);
      }
      iVar3 += uVar2;
      rolls += -1;
    } while (0 < rolls);
  }
  return iVar3;
}
The actual trait checks are random, but during worldgen they will be under the influence of one of the worldgen seeds (probably the History one) so they'll still be consistent.

I should note that all of these functions started out with names like "FUN_140910ae0" and "FUN_140910b50" - it was only through manual analysis that the types of all the different function parameters were identified (typically by starting from code where the data types can be trivially identified, and then tracing outwards from there) and meaningful names could be assigned to the functions themselves.

It also greatly helps that a small part of the source code for the Linux version is available (the "g_src" directory, which mainly only contains the code for rendering graphics and handling keyboard/mouse input).