Bay 12 Games Forum

Dwarf Fortress => DF Modding => Utilities and 3rd Party Applications => Topic started by: Roses on December 18, 2015, 04:20:36 pm

Title: Research
Post by: Roses on December 18, 2015, 04:20:36 pm

I wasn't exactly sure where I should put this, so I decided to start a new thread, in hopes people will see it. This topic will concern the inner workings of DF as they relate to things we can change with DFHack and the research I have been doing/have done on them. (Others please feel free to post your own findings)

First up, tokens in the unit.counters and unit.counters2 vectors (e.g. pain, suffocation, stunned, winded, etc...)

• Pain - Altering this value directly in the counters vector will have no effect. If you wish to inflict pain on a unit with DFHack you need to create a wound with the pain in the wound itself. Note that the pain value in the wound will not directly translate to the pain value in the counters, it will be adjusted based on the pain receptors of the specific body part.
• Nausea - Same as Pain (minus pain receptors)
• Dizziness - Same as Pain (minus pain receptors)
• Winded - Winded can be set to whatever you want, but it seems to cap at 50 (at least for the creatures I tested on). It then decreases by between 4 and 10 each tick (again for the creatures I tested). For each tick that Winded > 0 Suffocation increases by one.
• Stunned - Stunned can be set directly in counters, it's value will decrease by one each tick. Note that no matter what you set Stunned too you can never make a unit unconscious with just setting this.
• Unconscious - Same as stunned, in addition when Unconscious reaches 0 it will set Stunned to ~75 and will set Unconscious to -1. While Unconscious is -1 the unit will ignore increases certain counters.
• Suffocation - Suffocation will automatically reset to 0 if Winded is 0 (or presumably if the lungs are damaged, more testing is needed for this).
• Webbed - Same as Stunned except that it decreases by 2 each tick instead of 1.
• Paralysis - Same as Pain (minus pain receptors)
• Numbness - Same as Pain (minus pain receptors)
• Fever - Same as Pain (minus pain receptors)
• Exhaustion - Same as Stunned with the caveat that certain threshold values will change the health display of the unit (e.g. Tired, Over-Exerted, etc...)

In general there are two types of counters, those that have an effect without adding a wound, and those that require a wound to function. Note that just having a wound is not enough. You can't, for instance, just increase the value of Pain in unit.counters, you must increase the Pain value in unit.body.wounds. The game will then automatically adjust the pain value in unit.counters.

Next Up;

I will try and get an idea of values for emotions, there effects, and the possible effects traits have on them.
Determine effect of density on spawned flows
More as I come across it

Title: Re: Research
Post by: Roses on December 18, 2015, 04:20:53 pm

(Reserved just in case)

Title: Re: Research
Post by: Meph on December 19, 2015, 10:28:27 am

Not sure if this is what you are looking for, but adding NODRINK or NOEAT to creatures does not reset their hunger/thirst counters.

Title: Re: Research
Post by: Roses on December 19, 2015, 11:07:23 pm

Quote from: Meph on December 19, 2015, 10:28:27 am

Not sure if this is what you are looking for, but adding NODRINK or NOEAT to creatures does not reset their hunger/thirst counters.

Good to know, for both DFHack modding and normal modding.

Title: Re: Research
Post by: Bearskie on December 19, 2015, 11:36:40 pm

What about NOSLEEP?

Title: Re: Research
Post by: Max™ on December 20, 2015, 04:30:00 am

Hmmm, I noticed with my retired adventurers that they had 0 on the counters for those, but [NOEXERT] permanently sets exhaustion to 3000 for some reason which drove me nuts in fort mode.

Title: Re: Research
Post by: Roses on February 26, 2016, 05:21:55 pm

Preliminary research into attacks and the values in dfhack.

The attack action is currently filled with unknown variables. And currently looks like

Code: [Select]

target_unit_id           = 28
unk_4                    = <unit_action.T_data.T_attack.T_unk_4: 0x17234f0c>
attack_item_id           = 15
target_body_part_id      = 0
attack_body_part_id      = -1
attack_id                = 1
unk_28                   = 0
unk_2c                   = 1
unk_30                   = 57
flags                    = 66
unk_38                   = 0
unk_3c                   = 58
timer1                   = 3
timer2                   = 4
It has been "established" that unk_30 and unk_3c are attack_velocity and attack_hitchance (or somehow related to hit chance) and I believe the names have even been changed in the new DFHack. The following research tries to investigate the other components.

Tests done. By having several groups of dwarves with the same stats attack eachother with differing skills I have been analyzing their attack actions. Currently found results.

attack.flags
Flags is a specific number, meaning  you won't find just a random number between say 0 and 100. Instead it appears to be tied to the type of attack done (i.e. fast attack, wild attack, etc...). My reasoning behind this is that the attack.flag and attack.timer1/attack.timer2 is fixed, independent of attack preformed (with the exception of weapon vs non-weapon attacks). This means that if you were to tell me that the flags for a short sword attack was 66 I could tell you that the timer1 = 3 and timer4 = 4. In the couple dozen or so tests I ran with short swords, the flags values could be 2, 34, 66, 130, or 258. For the unnarmed attacks the flag values were 70, 134, or 262.

attack.unk_38
This number appears constant depending on if it is a weapon attack or creature attack. With all tested weapon attacks having unk_38 = 0 and all tested creature attacks having unk_38=102 or unk_38=103 (no idea why the difference all dwarves tested used 103, but some other animals used both, some only used 102, possibly other animals use other numbers?)

attack.unk_3c or hitchance
A highly variable number, as expected increase skill, on average, increases hit chance. The attack.flags also impacts hit chance greatly. With larger timer1 values and smaller timer2 values increasing hit chance (4,3 appearing to give the best increase for the tests done). Still need to do more work concerning varying agility, speed, and enemy dodge skills.

attack.unk_30 or velocity
Velocity values have been analyzed by Urist DaVinci in his disassemble work and appear to follow those values well.

attack.unk_28 and attack.unk_2c
No idea. Doesn't seem to be tied to anything I could think of, with unk_28 varying between 0 and 4 (usually 0, 1, or 2) and unk_2c varying between -2 and 5 (usually -2, -1, 0, or 1). Possibly related to the attack.unk_4 vector, as I have not studied the numbers in there yet.

That is my small update for now.

EDIT: After looking through about 50,000 attacks it appears that velocity and hit chance are related to eachother. For basic attacks (not fast, wild, hard, etc...) you have a higher velocity = higher hit chance (the particular equation I looked at was hit chance = velocity*0.8 - 40, but I haven't looked at the impact skills and attributes have yet). Now when you throw in different types of attacks the relationship changes, but it's a nice start.

EDIT2: Here are two simple graphs showing the velocity and hit chance of different types of attacks for dwarves with 1000 strength and 1000 agility at different skill levels.

Spoiler (click to show/hide)

(https://i.img.ie/jWp.png)

You will notice that the hit chance increases linearly with skill level, but the velocity seems to plateau.
Other interesting notes
Quick and Heavy have the same hit chance as Normal attacks, with Precise having more and Wild having less
Normal and Precise have the same velocities, Wild and Heavy have the same velocities, and Quick has the lowest velocities
Next step will be to check the effect attributes have on this distribution. Presumably the general shape will remain the same, with just the absolute values changing.

EDIT3: Last update for now. The reason the velocity is plateauing is because the max velocity of all attacks doesn't increase as skill goes up, just the minimum and the average. I believe you can think of there being an optimum velocity that you can reach based on your attributes and weapon (this is the velocity Urist DaVinci calculates), then the game uses a fraction of that velocity depending on how skilled you are. The more skilled, the closer that fraction goes to reaching 1. This is different from hitchance, which doesn't seem to have an upper cap, and whose max just keeps increasing along with the average at each skill level.

Title: Re: Research
Post by: Hesperid on February 27, 2016, 11:00:49 am

Quote from: Roses on February 26, 2016, 05:21:55 pm

attack.unk_28 and attack.unk_2c
No idea. Doesn't seem to be tied to anything I could think of, with unk_28 varying between 0 and 4 (usually 0, 1, or 2) and unk_2c varying between -2 and 5 (usually -2, -1, 0, or 1). Possibly related to the attack.unk_4 vector, as I have not studied the numbers in there yet.

I can answer as I did this same research a few versions ago. These numbers represent how easy the hit is to land and how well it can connect. Hence they vary between -3 and 3. (-3, -3) is an impossibly difficult hit that cannot connect even if the hit roll succeeds. (3, 3) is a simple hit (hit roll automatically succeeds) that can connect squarely, i.e. will typically dismember the opponent if it hits.

Title: Re: Research
Post by: Roses on February 27, 2016, 01:24:24 pm

Quote from: Hesperid on February 27, 2016, 11:00:49 am

Quote from: Roses on February 26, 2016, 05:21:55 pm

attack.unk_28 and attack.unk_2c
No idea. Doesn't seem to be tied to anything I could think of, with unk_28 varying between 0 and 4 (usually 0, 1, or 2) and unk_2c varying between -2 and 5 (usually -2, -1, 0, or 1). Possibly related to the attack.unk_4 vector, as I have not studied the numbers in there yet.

I can answer as I did this same research a few versions ago. These numbers represent how easy the hit is to land and how well it can connect. Hence they vary between -3 and 3. (-3, -3) is an impossibly difficult hit that cannot connect even if the hit roll succeeds. (3, 3) is a simple hit (hit roll automatically succeeds) that can connect squarely, i.e. will typically dismember the opponent if it hits.

Ah perfect! I didn't even think about that.

Title: Re: Research
Post by: Atomic Chicken on February 27, 2016, 06:46:21 pm

Quite unrelated to the research on attacks, but I haven't seen documentation on this anywhere, so here's my run-through on how to replicate the "true name" mechanics utilised by worldgen demon slabs:

Spoiler (click to show/hide)

Creating a "true name" and linking it to your target:

1) obtain your target unit's hist_figure_id
2) insert a new "identity" in df.global.world.identities.all and remember to increase the value of df.global.identity_next_id by +1
3) in the "identity" :

Code: [Select]

id: insert the "identity" index value
name: edit as desired and set "has_name" to true (this shall be the hidden "true name" of your target unit)
histfig_id: insert the target unit's hist_figure_id
unk_4c: not sure what this does, but apparently must be set to "2" for this to work

all other fields can be set to -1   
4)

Code: [Select]

df.historical_figure.find(histfigId).info.reputation.all_identities:insert("#", newidentityId)where histfigId = the hist_figure_id of your target unit
and newidentityId = the id of your previously created identity

Note that df.historical_figure.find(histfigId).info.reputation and possibly even df.historical_figure.find(histfigId).info may need to be newly created in some cases.
Of course, an entirely new histfig will need to be created for non-historical figures.

Creating a functional "true name" slab:

5) obtain or create a slab
6) in your target slab item:

Code: [Select]

engraving_type: set to "25" (DemonIdentity)
topic: insert your target unit's hist_figure_id
At this point, any unit who reads the slab will gain knowledge of the target's secret identity, and the invoke/banish conversation options will become available when talking to the target.

The effect of reading a "true name" slab:

Upon reading the slab, the following occurs:

Code: [Select]

df.historical_figure.find(x).info.secret.anon_2:insert("#", y)where 'x' is the hist_figure_id of the reader
and 'y' is the id of the identity linked to the hist_figure_id on the slab

Incidentally, this means that the slab can be entirely omitted from the procedure if so desired, by directly modifying the histfig info of the unit who is to learn the target's identity.

Title: Re: Research
Post by: Putnam on February 28, 2016, 01:03:44 am

unk_38 is attack_bp AFAIK... I thought that was already found, at least I posted it alongside the attack_velocity findings.

Title: Re: Research
Post by: Roses on February 28, 2016, 01:52:57 am

Quote from: Putnam on February 28, 2016, 01:03:44 am

unk_38 is attack_bp AFAIK... I thought that was already found, at least I posted it alongside the attack_velocity findings.

unk_38 is definitely not attack_bp, as it varies from 102 to 104 for body part attacks and is 0 for item attacks for dwarves. There is already a field attack_body_part_id that is seperate from unk_38.

From my tests on dwarves it appears unk_38 has to do with a relation between attack_body_part_id, target_body_part_id, flags and attack_id. Not quite sure what the exact number means, but in over 100,000 dwarf attacks it is only ever 102, 103, and 104, where as a dwarf only has ~80 body parts.

From what I can tell, for dwarves and ignoring wrestling attacks, 102 is used for bites, 103 is used for punchs and scratches, and 104 is used for kicks. Meaning 102 is for attack # 6, 103 is for attack # 0, 1, 4, and 5, and 104 is for attack # 2 and 3. This break down when the attack is a wrestling attack.

Title: Re: Research
Post by: scamtank on February 28, 2016, 02:27:05 am

Quote from: Roses on February 28, 2016, 01:52:57 am

Not quite sure what the exact number means, but in over 100,000 dwarf attacks it is only ever 102, 103, and 104, where as a dwarf only has ~80 body parts.

Hey, peep the skill index. (http://dwarffortresswiki.org/index.php/DF2014:Skill_token)

Title: Re: Research
Post by: Roses on February 28, 2016, 03:59:21 pm

Quote from: scamtank on February 28, 2016, 02:27:05 am

Quote from: Roses on February 28, 2016, 01:52:57 am

Not quite sure what the exact number means, but in over 100,000 dwarf attacks it is only ever 102, 103, and 104, where as a dwarf only has ~80 body parts.

Hey, peep the skill index. (http://dwarffortresswiki.org/index.php/DF2014:Skill_token)

Well then, that's that. Get's weird with wrestling attacks, and for some reason is always 0 for weapon attacks, but that solves that puzzle. That solves all the puzzles for non-wrestling based attacks. Go us!

Title: Re: Research
Post by: expwnent on February 29, 2016, 01:01:56 am

Posting to watch.

Title: Re: Research
Post by: Isngrim on February 29, 2016, 01:27:28 pm

PTW

Title: Re: Research
Post by: Roses on March 03, 2016, 08:25:49 pm

Full breakdown of the attack action. Note that the unk_4.unk_10 and unk_4.unk_14 fields are the only two things I have no ideas about. The seem to be nonesense, but I doubt that is the case. They could possibly be related to very specific circumstances.

• target_unit_id = Target unit id
• unk_4 -> Wrestling Vector
  • wrestle_type = -1 if not a wrestling attack, 1 if a grab, 0 if anything else (throw, break, slam)
  • unk_4 -> Attackers body part id (-1 if wrestling with an item)
  • unk_6 -> Defenders body part id (-1 if targeting an item with wrestling)
  • unk_8 -> Attackers item id (-1 if wrestling with a body part)
  • unk_c -> Defenders item id (-1 if targeting a body part with wrestling)
  • unk_10 -> No freaking clue, possibly related to breaks
  • unk_14 -> Again, no clue
• attack_item_id = Attack item id (-1 if attacking with a body part)
• target_body_part_id = Target body part id
• attack_body_part_id = Attack body part id (-1 if attacking with an item)
• attack_id = Attack id
• unk_28 -> Ease of strike (e.g. Fairly Easy, etc...)
• unk_2c -> Level of connection (e.g. Connects Squarely, etc...)
• unk_30 -> Attack velocity
• flags = Based off of attack_id, attack_body_part_id, target_body_part_id, and type of attack (Wild, Heavy, etc...). Comes in groups of numbers, exact meaning of the number and it's effect is unknown.
• unk_38 -> Body part attack skill
• unk_3c -> Attack hit chance
• timer1 = Preparation timer
• timer2 = Recovery timer

Title: Re: Research
Post by: Putnam on March 03, 2016, 09:44:38 pm

I'm reasonably sure the flags were fully mapped out by IIRC me and Quietust about a year ago (earliest stuff relating to ki in Sparking was April 2015, and that's when I figured out unk_30 and the flags IIRC)

Title: Re: Research
Post by: Roses on March 03, 2016, 10:06:30 pm

Quote from: Putnam on March 03, 2016, 09:44:38 pm

I'm reasonably sure the flags were fully mapped out by IIRC me and Quietust about a year ago (earliest stuff relating to ki in Sparking was April 2015, and that's when I figured out unk_30 and the flags IIRC)

Well, from my little work you get things like flags = 4 is a normal bite attack against body parts <= 15, flags = 5 is a normal bite attack against body parts > 15, flags = 6 is a normal grasp attack against body parts <= 15 and flags = 7 is a normal grasp attack against body parts > 15. 1030 and 1031 are normal stance attacks against body parts <= 15 and > 15 respectively. This is true, at least, for dwarves, where 36, 37, 38, 39, 1062, and 1063 are the corresponding flag numbers for quick attacks. It's possible that all the numbers are filled in by different attack types from different units or that there are intentional gaps. I don't know. If you know more please do share. Right now I can't seem to find any difference in outcomes when I change the flags number from one thing to another without changing other values.

Title: Re: Research
Post by: Roses on March 10, 2016, 04:38:07 pm

Just some final research to wrap up the attack research I have been doing. This will reference Urist DaVinci's (UDV) work which can be found here (http://www.bay12forums.com/smf/index.php?topic=142372.0) and here (http://www.bay12forums.com/smf/index.php?topic=131995.0). Note that his work and this work has all been done in DF 40.xx. I am unsure how much has changed with combat in 42.xx, but will be doing similar research to double check everything when I start using 42.xx.

1. The velocity UDV found is not the same as the velocity input into attack_velocity(unk_30). Instead it is some factor of that velocity, capped at the velocity. This is dependent on skill, strike ease, level of connection, and random factors. With, on average, greater skill, greater striking ease and greater level of connection resulting in higher velocities.

2. UDV didn't post any results for hit chance from his dissasembly reading, but from what I have found, all of the factors that influence the velocity are the same as those that influence attack_accuracy (unk_3c). Even the randomness found in the attack_velocity is the same randomness found in attack_accuracy. This means that whatever random roll is made is applied to both, instead of a different roll for each. The main difference between attack_velocity and attack_accuracy is attack_accuracy does not seem to be capped, instead increasing linearly with skill.

3. Type of attack and multipliers

Code: [Select]

         velocity     accuracy
normal       x1          x1
quick        x0.5        x1
heavy        x1.5        x1
wild         x1.5        x0.75
precise      x1          x1.5
Ending Note:
The actual equation that determines attack_accuracy is still unknown, but suitable temporary value ranges can be found numerically. In the future I may attempt to find an actual equation like UDV has done for attack_velocity.

Next up for research:
With the inclusion of emotion changing syndromes I plan on using 42.xx to test the various numbers in unit.current_soul.personality.emotions and determine how they translate from the syndromes to the structures. In addition I hope to test the effects these emotions have on units. Possibly even seeing if the emotional state has any effect on combat (besides the extreme states where the unit "flees in terror" and such).

Title: Re: Research
Post by: Putnam on March 10, 2016, 07:22:52 pm

attack_velocity is capped at 10000 IIRC

Sparking can bring it up to... I think I cap it off at 2000000000. It has proper effects there, too, sending even larger opponents flying.

Title: Re: Research
Post by: Roses on March 10, 2016, 07:26:06 pm

Quote from: Putnam on March 10, 2016, 07:22:52 pm

attack_velocity is capped at 10000 IIRC

Sparking can bring it up to... I think I cap it off at 2000000000. It has proper effects there, too, sending even larger opponents flying.

Ah yes, that is important to note. The game caps the attack_velocity at 10000 when calculating it, but larger numbers (if put in manually or via script) do have an effect. As far as I can tell it is a linear effect, although to figure that out exactly it would require many more tests than I have done.

Title: Re: Research
Post by: expwnent on March 11, 2016, 04:31:22 am

Great work!

Title: Re: Research
Post by: Max™ on March 16, 2016, 12:35:52 am

In case anyone knows more about working with offsets, when you pull up the world_data screen in gm-editor there is a region_map entry but it never changes and doesn't have a list so I can't manually make it open region_id[k] or whatnot, but playing with the offsets option (thank you for putting that in, whoever added it, it's exactly the kind of derpy vaguely dangerous thing I like to play with!) I was able to determine that 1 through 30 (probably 33 as it is a 33x33 map) changed the map opened to the regions along the west edge of the world.

That map has the fields for evil, savagery, volcanism, drainage, etc, and directly editing the values in there is enough to reliably change a serene site to a terrifying one which for some reason has bubble bulbs and unicorns. There's also stuff related to weather so that should be tweakable through there.

Title: Re: Research
Post by: Roses on March 16, 2016, 02:11:00 pm

If you instead just open up the lua command line by typing 'lua' in the DFHack command line, you can access all the different region_map_entry values with df.global.world.world_data.region_map[n].

Title: Re: Research
Post by: Max™ on March 16, 2016, 07:34:53 pm

>.>

Naturally, I was trying to open it up through the region map entry nesting because the value line screwed me up. That string works with gm-editor too, so yeah, you can change region evilness fine then.

Title: Re: Research
Post by: Roses on June 02, 2017, 03:53:36 pm

I'm bumping this thread, both to bring interest back to it and in preparation of a new DF release. I've noticed a lot of threads with people performing their own !SCIENCE! and I am planning on starting to gather those all together and provide links to it from here. If you have any research you would like to post, or any links to research feel free to post them.

As for me, I am starting back up my research into syndromes and how they are transferred to units. Currently a CE_NAUSEA:SEV:100 syndrome doesn't necessarily mean an increase in 100 in the unit.counter.nausea counter. Similarly some syndromes actually work by applying a wound to unit.body.wounds, this is why some syndromes applied via modtools/add-syndrome don't work correctly. The syndrome is added but the wound is not.

Also, I am wondering if anyone knows what tool(s) Urist DaVinci used here (http://www.bay12forums.com/smf/index.php?topic=131995.0) to obtain the equations he did. I know he mentions disassembly reading and DFHack, but my knowledge of the former is rather limited.

Title: Re: Research
Post by: Quietust on June 03, 2017, 08:14:26 am

In reply to your rather old research about attack flags, you were incorrectly looking at them as a number when you should have been looking at them as a bitfield (as nearly all "flags" values are), and had you looked directly at the df-structures XML files (https://github.com/DFHack/df-structures/blob/master/df.units.xml#L1832), you would have seen that bits 5-9 mean "quick attack", "heavy attack", "wild attack", "precise attack", and "charge attack" (with bit 10 being related to multi-attacks), while the lower 5 bits were never properly identified.

Title: Re: Research
Post by: Roses on June 03, 2017, 11:25:50 am

Quote from: Quietust on June 03, 2017, 08:14:26 am

In reply to your rather old research about attack flags, you were incorrectly looking at them as a number when you should have been looking at them as a bitfield (as nearly all "flags" values are), and had you looked directly at the df-structures XML files (https://github.com/DFHack/df-structures/blob/master/df.units.xml#L1832), you would have seen that bits 5-9 mean "quick attack", "heavy attack", "wild attack", "precise attack", and "charge attack" (with bit 10 being related to multi-attacks), while the lower 5 bits were never properly identified.

Interesting, so why do values change based on type of attack (bite, grasp, stance, etc...) and targeted body part? Sorry, my knowledge of bitfields is lacking.

Title: Re: Research
Post by: Quietust on June 03, 2017, 12:45:04 pm

Quote from: Roses on June 03, 2017, 11:25:50 am

Interesting, so why do values change based on type of attack (bite, grasp, stance, etc...) and targeted body part? Sorry, my knowledge of bitfields is lacking.

A bitfield is a single number used to store multiple "on/off" values, distinguished by position - if you convert the number to binary, you can see each individual value as a 0 or 1.

For example:

Code: [Select]

  2 = 0000000010 -> bit 1 set
 34 = 0000100010 -> bits 1 and 5 set
 66 = 0001000010 -> bits 1 and 6 set
130 = 0010000010 -> bits 1 and 7 set
258 = 0100000010 -> bits 1 and 8 set
In this particular case, bits 5-9 were already identified (as I mentioned previously, "quick", "heavy", "wild", "precise", and "charge"), but bits 0-4 also have some special meaning which I wasn't able to determine at the time (and of those, bit 1 is being actively used).

In other parts of Dwarf Fortress, multiple bits might be combined together to form a numeric value - for example, the Tile Designation flags use 3 consecutive bits to store the Liquid Level (i.e. 0-7) as well as another 3 bits to specify one of up to 8 types of Dig designations present in a given tile (none, tunnel, up/down stair, channel, ramp, down stair, or up stair - the 8th type isn't used). In most other places, though, they're just used to store individual on/off values, since it uses much less memory than storing them in separate variables (and memory usage has historically been a big problem in Dwarf Fortress, at least until Toady started making 64-bit builds).

See this article (https://en.wikipedia.org/wiki/Bit_field) for more information.

Title: Re: Research
Post by: Roses on June 03, 2017, 02:20:07 pm

Ah, that makes a lot of sense, thank you! So if I follow, for my previous testing with dwarves

Normal Bite Attacks (maybe more specifically, normal attacks with the attack skill as BITE?)
4 -> bit 2 set
5 -> bit 2 and 0 set

Normal Grasp Attacks
6 -> bit 2 and 1 set
7 -> bit 2 and 1 and 0 set

Normal Stance Attacks (Kick has ATTACK_FLAG_BAD_MULTIATTACK, maybe that's why bit 10 is set?)
1030 -> bit 10 and 2 and 1 set
1031 -> bit 10 and 2 and 1 and 0 set

Quick Bite Attacks
36 -> bit 5 and 2 set
37 -> bit 5 and 2 and 0 set

Quick Grasp Attacks
38 -> bit 5 and 2 and 1 set
39 -> bit 5 and 2 and 1 and 0 set

Quick Stance Attacks
1062 -> bit 10 and 5 and 2 and 1 set
1063 -> bit 10 and 5 and 2 and 1 and 0 set

Wrestling Attack
12 -> bit 3 and 2 set

It looks like;

• bit 0 is set when targeting body parts >15 on Dwarves, which corresponds (I believe) to small body parts.
  
• bit 1 is set for grasp and stance attacks and not bite attacks
• bit 2 is set for all the attacks (maybe bit 2 is set for body part attacks and not set for item attacks? I will test)
• bit 3 is set for wrestling attacks
• bit 5 is set for quick attacks (which is what is listed in the DF-Structures
• bit 10 is listed for stance attacks (possibly because the KICK attack has ATTACK_FLAG_BAD_MULTIATTACK)

I didn't find any cases where bit 4 was set, but I will do a more exhaustive test now that I know what I am looking at/for.

EDIT: All of the above information is now out of date since the attack action structure changed a bit and now has correctly identified all of the necessary items.

Title: Re: Research
Post by: Roses on February 08, 2018, 12:30:34 pm

So a new version of DF means new research to be done! I am going to start with this updates major feature, sending units out on raids. I'll start by identifying some of the anon fields

Starting with the [c] screen (the one with the world map on the left and different "tabs" on the right). Inspecting it with dfhack.gui.getCurViewscreen we get a list of entires

Code: [Select]

child = nil
parent = <viewscreen_dwarfmodest>
breakdown_level = 0
option_key_pressed = 0
page = 2
sel_idx = 0
entities = <vector<historical_entity*>[1]>
map_x = 11
map_y = 17
site = <world_site>
anon_1 = <vector<void*>[0]>
anon_2 = <vector<void*>[0]>
anon_3 = <vector<void*>[0]>
anon_4 = 0
anon_5 = <vector<void*>[1]>
anon_6 = <vector<void*>[0]>
anon_7 = 0
anon_8 = <vector<void*>[0]>
anon_9 = 0
anon_10 = 0
anon_11 = 0
anon_12 = <vector<void*>[0]>
anon_13 = <vector<void*>[0]>
anon_14 = 0
anon_15 = 0
anon_16 = 0
artifact_desc = <vector<string*>[0]>
anon_17 = 0
anon_18 = 0
anon_19 = 0
anon_20 = 0
anon_21 = <vector<void*>[#]>
anon_22 = <vector<void*>[0]>
anon_23 = 0
anon_24 = 0
anon_25 = 0
anon_26 = 0
anon_27 = 0
anon_28 = 11
anon_29 = 13
anon_30 = 587
anon_31 = 0
anon_32 = 0
anon_33 = 0
unk1 = <viewscreen_civlistst.T_unk1>
anon_34 = <vector<string*>[#]>
This is what pops up for me when I use it on a freshly created world/fort. Let's go through them and see if we can identify them

• page <= This is the "tab" you are on. 0 is Civilizations, 1 is News and rumors, 2 is World, 3 is Missions, 4 is the squad select for Missions 5 is People, and 6 is Artifacts
• entities <= I only ever got one entry in this table and that was my entity, possibly the number of entries in 'Civilizations', which might make set_idx the index of the entry in 'Civilizations' that you are looking at
• map_x, map_y <= These are the x, y coordinates your cursor is currently over on the world map
• site <= This is a link to the site you are currently selecting, nil if not selecting a site
• anon_1 and anon_2 <= While these were always blank for my, I assume they have something to do with 'News and rumors'
• anon_3 <= While this table is initially empty, every mission you create will create a new entry in this table
• anon_4 <= The index of the entry from anon_3 that you are currently selecting
• anon_5 <= A list of currently active missions (or possibly squads)
• anon_8 <= Presumably a list of missing citizens from 'People' (I never had any, but based on anon_10 and anon_11 and the structure of the other anons it makes sense)
• anon_9 <= Presumably the index you are selecting from anon_8
• anon_10 and anon_11 <= The x and y coordinates for your cursor when using 'People'
• anon_12 and anon_13 <= These tables have something to do with artifacts. While initially empty the moment you go to page 6 they will be populated with a number of entries equal to the number of artifacts. Presumably one is for the actual item entry and one is for the artifact entry
• anon_14 <= The index of the artifact you are currently selecting
• anon_15 and anon_16 <= The x and y coordinates of the artifact you are currently selecting
• artifact_desc <= A table of strings for the description of the artifact you are currently selecting
• anon_21 <= No idea, but do not try to access, will cause a crash
• anon_28 and anon_29 <= The x and y coordinates selected while using 'News and rumors'

The rest I have no idea about, especially unk1. it contains 11 anons each of which are 1000 long tables of integers. anon_21 and anon_34 are also weird because they can be vectors that have, supposidly, 100000+ entries.

Next up I will look at the squads and armies entries to figure out how the missions actually work.

Title: Re: Research
Post by: Rumrusher on February 09, 2018, 04:11:53 am

anon_20 allows you to bypass the lacking a leader check though it might be there to prevent folks from just Crashing the game I havent figure out what the numbers lead to but I do know -1 the value will lock you out of sending folks out to raids.
wanted to mess with this  menu more but ended up just fiddling with the army and the army's controller instead.

Title: Re: Research
Post by: Roses on February 09, 2018, 11:48:22 am

Quote from: Rumrusher on February 09, 2018, 04:11:53 am

anon_20 allows you to bypass the lacking a leader check though it might be there to prevent folks from just Crashing the game I havent figure out what the numbers lead to but I do know -1 the value will lock you out of sending folks out to raids.
wanted to mess with this  menu more but ended up just fiddling with the army and the army's controller instead.

Interesting, I started looking in to the army, army controllers, and squads last night. I'll be posting my findings today, For the most part it seems the additions to the newest DF version are the new orders for squads that tell them they are going out on a raid. I still want to see what happens if I artificially give a squad the raiding order without going through the world map interface.

Title: Re: Research
Post by: lethosor on February 09, 2018, 02:10:38 pm

Is it actually printing "vector<void*>[#]", or did you replace some number with "#"?

I think I identified most of the fields with names... are the names not clear enough? The various page IDs are defined in df.viewscreen_civlistst.T_page. Also, in both gui/gm-editor and the lua interpreter, you can use "scr" or "screen" instead of "dfhack.gui.getCurViewscreen()".

I'm pretty sure I added vectors in the right places on Linux/macOS, so if anon_22 is misaligned on Windows, that means everything after it is also wrong on Windows, and maybe things as early as anon_17 (since anon_17-21 are all integers).

Title: Re: Research
Post by: Roses on February 09, 2018, 02:30:48 pm

Quote from: lethosor on February 09, 2018, 02:10:38 pm

Is it actually printing "vector<void*>[#]", or did you replace some number with "#"?

I think I identified most of the fields with names... are the names not clear enough? The various page IDs are defined in df.viewscreen_civlistst.T_page. Also, in both gui/gm-editor and the lua interpreter, you can use "scr" or "screen" instead of "dfhack.gui.getCurViewscreen()".

I'm pretty sure I added vectors in the right places on Linux/macOS, so if anon_22 is misaligned on Windows, that means everything after it is also wrong on Windows, and maybe things as early as anon_17 (since anon_17-21 are all integers).

Yeah, I just replaced the number printed with #, it was usually >100000, but seemed rather random. I don't see any names for any of the anon stuff. Similarly looking at armies and squads I see a lot of anon stuff as well. It's worth noting that the tables (like anon_13) just contain <user_data> entries, not anything actually accessible.

Title: Re: Research
Post by: Roses on February 09, 2018, 04:11:47 pm

Alright, squads, armies, and army controllers time!

Small note, I use the nomenclature World map x/y and Block map x/y. World map refers the coordinates when looking at the world map, so for a small world they can be 0-31 (32x32). Block map refers to the entire block a single world map tile takes up, I believe this is always 48*32? Maybe, actually maybe I don't know what block map means, but you typically see numbers between a few hundred up to about 1300 so I think it's the smaller grained x,y information.

For squads all we really need to look at are the squad.orders (although anon_1 in squad is the army controller ID). I'm going to just be looking at the df.squad_order_raid_sitest order.

Code: [Select]

<squad_order_raid_sitest>
unk_v40_1 = -1
unk_v40_2 = -1
year = 23
year_tick = 17122
unk_v40_3 = 52                       -- The army controller ID of the squad
anon_1 = 0
squad_order_raid_sitest.anon_1 = 4   -- The ID of the site being raided
anon_2 = <coord>                     -- This is the coordinate your squad left the forts map from
Short and sweet.

Now armies, nothing really of note here, most things are identified and pretty self explanatory. So I will just do a snip

Code: [Select]

<army>
id = 128
<snip>
unk_pos_x = <vector<int32_t>[9]>  -- Block map x coordinates left in the current blocks travel
unk_pos_y = <vector<int32_t>[9]>  -- Block map y coordinates left in the current blocks travel
unk_70 = <vector<int32_t>[11]>    -- World Map x coordinates left in the armies travel
unk_80 = <vector<int32_t>[11]>    -- World Map y coordinates left in the armies travel
<snip>These were already known from others work and used in mifki's script to force a siege, they are just here for completeness

Moving on to army_controllers, this gets a little confusing so I won't snip anything

Code: [Select]

<army_controller>
id = 53
entity_id = 75
unk_8 = 10                       -- ID of the site the army is going to
pos_x = -1
pos_y = 616                      -- Are we sure this isn't a block pos_x
unk_14 = 1177                    -- And this isn't a block pos_y
unk_18 = 0
unk_1c = 0
unk_20 = <vector<int32_t>[0]>
year = 23
year_tick = 17122
unk_34 = -1
unk_38 = 53                      -- ID of the army controller again?
unk_3c = 286
unk_40 = -1
unk_44 = <vector<int32_t>[0]>
unk_50 = 8
unk_54 = <vector<int32_t>[1]>    -- I think this is the ID of the mission being done, squads going on raids have a number here, other squads don't
unk_60 = <army_controller_unk60> -- Another construct, will discuss below
unk_64 = <army_controller.T_unk_64>
type = 2


<army_controller_unk60>
anon_1 = <vector<army_controller_unk60.T_anon_1*>[1]>         -- Another construct! Discussed below
anon_2 = 0
anon_3 = 25                                                   -- Armies current world map x position
anon_4 = 20                                                   -- Armies current world map y position
anon_5 = -1
anon_6 = Mission Report: Raid Dreadfought (Set out Spring 23) -- The description of the mission
anon_7 = 0
anon_8 = 23                                                   -- Year
anon_9 = 17790                                                -- Year tick
anon_10 = vector<int32_t>[0]>
anon_11 = -1
anon_12 = -1


<army_controller_unk60.T_anon_1>
anon_1 = <int32_t[]>  -- World map x coordinate of army, gets filled in as the unit moves
anon_2 = <int32_t[]>  -- Cooresponding world map y
anon_3 = <int32_t[]>  -- Year of the cooresponding move to new x,y
anon_4 = <int32_t[]>  -- Year tick to go with the year
anon_5 = 4            -- Number of steps taken
anon_6 = <int32_t[]>  -- Looks like it could be the x or y block coordinate of the site explored, but only having one is weird, probably something else
anon_7 = <int32_t[]>  -- A list of years the site was explored
anon_8 = <int32_t[]>  -- A list of year ticks the site was explored
anon_9 = 0            -- Number of filled in integers in anon_6,7,8

That is as far as I have gotten, I was thinking there would be some df.global.world.missions or something of the sort, so I went through some of the unknowns in world but wasn't able to find anything that would match, except maybe anon_1.

As for my last stuff;
My guess for anon_8,9,10,11 was correct
My guess for anon_5 was not correct, it may instead be the number of known entities?
I think anon_6 is actually the list of currently active missions
Which makes anon_7 the currently highlighted active mission maybe?

Title: Re: Research
Post by: Roses on February 20, 2018, 12:17:35 pm

Well research continues with squads, armies, and missions. Right now I am trying to create a mission completely through DFHack. That is, create a squad, an army, and an army controller, assign a unit to said squad, and then send the squad out on a squad_order_raid_sitest order. So far I haven't gotten it to work so I thought I would ask to see if anyone else has tried something similar.

Title: Re: Research
Post by: Roses on March 14, 2018, 11:01:31 am

I finally got "custom" missions to sort of work, that is to say I was able to create a mission and send a unit off on it by creating a squad, army, army controller, and squad order and linking them all together all with a script. The trouble is at some point (maybe when the unit arrived at the destination, maybe when they got back, maybe somewhere in between) the game crashed. I'll keep investigating on my end, there may have been an incorrectly set flag or variables (there are a lot of them), but after seeing the 44.07 update I'm hopeful that maybe what I saw was one of the crashes that Toady fixed. Once a new version of DFHack is available I'll try again and see if it's fixed.

Title: Re: Research
Post by: bloop_bleep on March 14, 2018, 12:23:25 pm

PTW.

I really want to do some kind of research into this at some point, since it seems like the kind of puzzle I would like.  :D

Title: Re: Research
Post by: Roses on March 15, 2018, 11:50:10 am

Quote from: bloop_bleep on March 14, 2018, 12:23:25 pm

PTW.

I really want to do some kind of research into this at some point, since it seems like the kind of puzzle I would like.  :D

There's always more research that can be done!

Title: Re: Research
Post by: Atomic Chicken on March 26, 2018, 03:21:10 pm

creature_raw.flags.unk_69

Was set to true for every race except for the following creatures in a vanilla test world:

Spoiler: unk_69 == false (click to show/hide)

blood man
gabbro man
amethyst man
bronze colossus
fire man
magma man
iron man
mud man
forgotten beast
forgotten beast
forgotten beast
forgotten beast
brute of brine
specter of salt
specter of steam
white ghost
steam haunt
snow haunt
fiend of ash
specter of embers
boiling demon
Being of The Hatchet

Seems to me that this flag refers to whether or not a creature is organic as opposed to inorganic.

creature_raw.flags.unk_49

Set to true for the following creatures:

Spoiler: unk_49 == true (click to show/hide)

dwarf
human
elf
goblin
kobold
cave fish man
olm man
bat man
cave swallow man
amphibian man
reptile man
serpent man
antman
rodent man

These are all the creatures which form civilisation-type entities at the beginning of worldgen.

creature_raw.flags.unk_6e

Spoiler: unk_6e == true (click to show/hide)

toad man
worm man
bluejay man
cardinal man
grackle man
oriole man
red-winged blackbird man
penguin man
peregrine falcon man
kiwi man
ostrich man
crow man
raven man
cassowary man
kea man
snowy owl man
sparrow man
white stork man
loon man
barn owl man
parakeet man
kakapo man
grey parrot man
puffin man
swan man
lorikeet man
wren man
osprey man
emu man
cockatiel man
peach-faced lovebird man
magpie man
kestrel man
albatross man
great horned owl man
eagle man
hornbill man
masked lovebird man
bushtit man
damselfly man
moth man
grasshopper man
bark scorpion man
mantis man
tick man
louse man
thrips man
slug man
mosquito man
jumping spider man
moon snail man
brown recluse spider man
snail man
leopard gecko man
desert tortoise man
gila monster man
fly man
roach man
beetle man
monarch butterfly man
firefly man
dragonfly man
mountain goat man
hoary marmot man
walrus man
narwhal man
hippo man
platypus man
grizzly bear man
black bear man
deer man
fox man
raccoon man
rhesus macaque man
cougar man
wolf man
groundhog man
alligator man
buzzard man
panda man
capybara man
badger man
moose man
red panda man
elephant man
warthog man
lion man
leopard man
jaguar man
tiger man
cheetah man
gazelle man
mandrill man
one-humped camel man
two-humped camel man
saltwater crocodile man
vulture man
rhinoceros man
giraffe man
honey badger man
giant tortoise man
armadillo man
muskox man
elk man
polar bear man
wolverine man
chinchilla man
gorlak
plump helmet man
octopus man
crab man
leopard seal man
cuttlefish man
orca man
sponge man
horseshoe crab man
sperm whale man
elephant seal man
harp seal man
nautilus man
lizard man
skink man
chameleon man
anole man
iguana man
otter man
snapping turtle man
beaver man
leech man
axolotl man
mink man
pond turtle man
rat man
gray squirrel man
red squirrel man
chipmunk man
hamster man
hedgehog man
flying squirrel man
squid man
cave fish man
olm man
bat man
cave swallow man
wild boar man
coyote man
kangaroo man
koala man
adder man
echidna man
porcupine man
kingsnake man
gray langur man
bobcat man
skunk man
green tree frog man
hare man
rattlesnake man
weasel man
copperhead snake man
ibex man
wombat man
dingo man
coati man
opossum man
mongoose man
hyena man
anaconda man
monitor lizard man
king cobra man
ocelot man
jackal man
capuchin man
sloth man
spider monkey man
pangolin man
black mamba man
sloth bear man
aye-aye man
bushmaster man
python man
tapir man
impala man
aardvark man
lion tamarin man
stoat man
lynx man

Mostly animal people, but the list also includes gorlaks and plump helmet men. Not entirely sure what to make of this.

creature_raw.flags.unk_6f

Spoiler: unk_6f == true (click to show/hide)

human

Refers to OUTSIDER_CONTROLLABLE.

creature_raw.flags[114]

Spoiler: 114 == false (click to show/hide)

wagon
chimera
centaur
griffon
sea lamprey
great white shark
frill shark
spiny dogfish
spotted wobbegong
whale shark
basking shark
nurse shark
shortfin mako shark
longfin mako shark
tiger shark
bull shark
blacktip reef shark
whitetip reef shark
blue shark
hammerhead shark
angelshark
common skate
manta ray
stingray
coelacanth
sturgeon
conger eel
milkfish
cod
opah
giant grouper
bluefish
ocean sunfish
swordfish
marlin
halibut
great barracuda
bluefin tuna
narwhal
giant narwhal
longnose gar
carp
tigerfish
pike
pond grabber
octopus
giant octopus
cuttlefish
giant cuttlefish
orca
giant orca
sponge
giant sponge
sperm whale
giant sperm whale
nautilus
giant nautilus
fluffy wambler
mussel
oyster
salmon
clownfish
hagfish
brook lamprey
bat ray
thornback ray
spotted ratfish
herring
shad
anchovy
steelhead trout
hake
seahorse
glasseye
white-spotted puffer
sole
flounder
mackerel
sea nettle jellyfish
squid
gigantic squid
clown loach
brown bullhead
yellow bullhead
black bullhead
banded knifefish
char
rainbow trout
sailfin molly
guppy
perch
sea serpent
sea monster
cave fish

Set to false for sea creatures, wagons and the non-existent chimera, centaur and griffon.

creature_raw.flags[115]

Spoiler: 115 == false (click to show/hide)

blue jay
bluejay man
giant bluejay
cardinal
cardinal man
giant cardinal
grackle
grackle man
giant grackle
oriole
oriole man
giant oriole
red-winged blackbird
red-winged blackbird man
giant red-winged blackbird
peregrine falcon
peregrine falcon man
giant peregrine falcon
crow
crow man
giant crow
raven
raven man
giant raven
kea
kea man
giant kea
snowy owl
snowy owl man
giant snowy owl
sparrow
sparrow man
giant sparrow
white stork
white stork man
giant white stork
loon
loon man
giant loon
barn owl
barn owl man
giant barn owl
parakeet
parakeet man
giant parakeet
grey parrot
grey parrot man
giant grey parrot
puffin
puffin man
giant puffin
swan
swan man
giant swan
lorikeet
lorikeet man
giant lorikeet
wren
wren man
giant wren
osprey
osprey man
giant osprey
cockatiel
cockatiel man
giant cockatiel
peach-faced lovebird
peach-faced lovebird man
giant peach-faced lovebird
magpie
magpie man
giant magpie
kestrel
kestrel man
giant kestrel
albatross
albatross man
giant albatross
great horned owl
great horned owl man
giant great horned owl
eagle
eagle man
giant eagle
hornbill
hornbill man
giant hornbill
masked lovebird
masked lovebird man
giant masked lovebird
bushtit
bushtit man
giant bushtit
damselfly
damselfly man
giant damselfly
moth
moth man
giant moth
mantis
mantis man
giant mantis
thrips
thrips man
giant thrips
mosquito
mosquito man
giant mosquito
duck
goose
fly
fly man
giant fly
large roach
roach man
giant roach
ant
monarch butterfly
monarch butterfly man
giant monarch butterfly
firefly
firefly man
giant firefly
dragonfly
dragonfly man
giant dragonfly
honey bee
bumblebee
buzzard
buzzard man
giant buzzard
vulture
vulture man
giant vulture
hungry head
bugbat
cave floater
acorn fly
blood gnat
fairy
pixie
nightwing
harpy
roc
giant bat
bat
bat man
cave swallow
cave swallow man
giant cave swallow
antman
forgotten beast
forgotten beast
forgotten beast
forgotten beast
forgotten beast
forgotten beast
forgotten beast
forgotten beast
forgotten beast
forgotten beast
forgotten beast
brute of brine
specter of salt
specter of steam
white ghost
steam haunt
snow haunt
fiend of ash
specter of embers
boiling demon
wicked man
Being of The Hatchet

Appears to to be set to false for creatures which have at least one caste capable of flight.

creature_raw.flags[116]

Spoiler: 116 == true (click to show/hide)

blind cave ogre
manera
troll
ogre
giant
blizzard man
troglodyte
dusk man
shadow creature
wicked man
woman of shadow
freak of twilight

Probably refers to SLOW_LEARNER.

creature_raw.flags[117]
to
creature_raw.flags[135]
Appear to be identical to creature_raw.flags[116].