Bay 12 Games Forum

Dwarf Fortress => DF Modding => Topic started by: Aedom on September 20, 2016, 11:48:47 am

Title: Reversing combat damage
Post by: Aedom on September 20, 2016, 11:48:47 am
Hello one and all.

I have a personal !science! project of some possible interest to the community. 

I'm am building on Urist da Vinci's wonderful analysis of the material property influences. http://www.bay12forums.com/smf/index.php?topic=131995.0 (http://www.bay12forums.com/smf/index.php?topic=131995.0).

What I find missing is a relation to map the input momentum, and tissue layer results to the sustained damage that is stored for each individual tissue layer.

Presently, I am engaged in building a "combat sniffer" df hack script that will record data about combat sessions.  I was wondering if anyone in the community would be willing to run the sniffer in arena mode, and send me back logged data, in hopes that I can aggregrate it all into one consistent model.

So far, it is looking like getting the data will involve creating interesting fights in arena mode after running a df hack command.  The output will be a log file that can be harvested.

I hope that I have picqued someone's interest :)
Thanks for reading
Title: Re: Reversing combat damage
Post by: Grimlocke on September 20, 2016, 01:12:46 pm
Ooh, consider my interest peaked!

This could end up being a pretty great debug tool for combat and creature mods, especially if it could also somehow keep track of momentum stopped by clothing and armor.
Title: Re: Reversing combat damage
Post by: Aedom on September 20, 2016, 02:51:53 pm
Thus far, I have a first prototype version that subscribes to OnUnitAttack.  Then it looks up the relevant junk and dumps anything relevant about it.

The next phase is to iterate a couple of times, while driving parser development.  I'll probably write that in C#.

Once I have the combat sniffer and parser hardened, I'll post back here with instructions to download and use.
Title: Re: Reversing combat damage
Post by: Aedom on September 20, 2016, 02:55:32 pm
Ooh, consider my interest peaked!

This could end up being a pretty great debug tool for combat and creature mods, especially if it could also somehow keep track of momentum stopped by clothing and armor.

Already experimenting with that ;)  The momentum as it passes though the layers isn't in memory at the time we are dumping stuff.  But if you refer the link I have in the opening comment, you can see Urist Da Vinci's work, describing the momentum effects in detail.

This tool will be able to show how much damage is done to each tissue layer.  If you wish you dress your units in the arena, then those clothings/armors will be taken into account.  The raw dump will also contain relevant information so you can see which clothings/armors affected each and every action that was logged.
Title: Re: Reversing combat damage
Post by: Grimlocke on September 20, 2016, 06:31:50 pm
Cool, will keep track of this thread.

And yes, I'm quite familiar with Urist da Vinci's sciences. I used it to make a mod that pretty much overhauls the way weapons and armor work, which is one of the reasons I'm interested in this. Balancing the mod right now is a matter of making an immobile, blind dummy creature, dressing it up in armor and hitting it with various things to see the effect but its kind of a cumbersome and slow process.
Title: Re: Reversing combat damage
Post by: Aedom on September 21, 2016, 12:58:36 pm
Cool, will keep track of this thread.

And yes, I'm quite familiar with Urist da Vinci's sciences. I used it to make a mod that pretty much overhauls the way weapons and armor work, which is one of the reasons I'm interested in this. Balancing the mod right now is a matter of making an immobile, blind dummy creature, dressing it up in armor and hitting it with various things to see the effect but its kind of a cumbersome and slow process.

I can probably help with that.  I am working on the parser.  Since you seem to find the project useful, I'll put some extra hours into making a forms gui to display the logged data.
Title: Re: Reversing combat damage
Post by: Aedom on September 21, 2016, 02:48:26 pm
I am starting to suspect that the EventManager that is polling for these "events" is missing some. 

While I haven't ruled out an error in my stuff, I also have not been able to prove that every weapon strike in the simulation gets a single onUnitAttack event and one or more onReport events.  Work continues, but this has been a major slow down.
Title: Re: Reversing combat damage
Post by: Aedom on September 22, 2016, 08:59:19 am
Here's a screenie of the display tool prototype

Spoiler (click to show/hide)

It's about 80% functional atm and needs an actual UI design, but I am already finding it useful.
Title: Re: Reversing combat damage
Post by: Aedom on September 22, 2016, 05:51:18 pm
Here's the close of day screenshot.  As you can see, it is coming right along. 

Turns out that if you have large arena battle, the lua engine can miss some events, so that will just be part of life.  I don't think it will be an issue to only use a handful of combatants at a time, since this is a very detailed and "zoomed in" tool.

I'm almost ready to open it up for beta testing after a few more UI features and polish.

Spoiler (click to show/hide)

Any thoughts? Wishes? Outrage?
Title: Re: Reversing combat damage
Post by: Grimlocke on September 23, 2016, 09:05:13 am
Cool, looking good so far.

Seems it can display pretty much all information the game retains of tissues and bodyparts. Curious to see how it will handle some of my modded creatures.
Title: Re: Reversing combat damage
Post by: Aedom on September 23, 2016, 09:43:03 am
The underlying event system that I'm using to get all the data, has some real problems.  It will fail to create some attack events, and it will also incorrectly duplicate the last attack event, in place of the new one, in some cases.

These are the biggest problems.  In my sniffer lua script, I just dump the information given by the event system (eventful/EventManager)

It is somewhat disappointing to know that no matter how much polish and hard work I put in here, I will not have a completely accurate tool.

The other big annoyance is that death strikes don't get wound information in the attack callback.  My guess is that the game doesn't bother fully committing wounds to a body that has died this frame. 

Regardless of these deficiencies, I should be able to give you a tool that is correct 90% of the time.  The longer the recorded battle, the bigger the magnitude of the errors. 

I had to write a heuristic system to tie the combat report text to the actual strike events, since they aren't directly related in the lua environment.  This system works by scanning the combat log from beginning to end, and removing entries that are found to be matches (only allow each report line to be used once).  So if an event gets dropped or duplicated, it can throw off the entire stream of future events for the (attacker, defender) pair. 

I have split the log file up into sessions, that are easy to control via the df hack command line.  Usage will involve pausing and starting/stopping a session around the combat actions that you want to analyze.  This session scoping can be used to still have lots of data in a file, but it will be separated out, so that these event system errors can be localized.

It is still very usable.
Title: Re: Reversing combat damage
Post by: Aedom on September 24, 2016, 02:20:36 pm
A major factor, in the missing and duplicated events phenomena, is the speed at which the attacks occur.    This is clear, if you pit two bronze collosuses against one another.  They end up hitting the same body parts over and over, in rapid succession.  This is the optimal scenario for dropped and duplicated attack events.

This is easy to see if you take an accounting of the UNIT_ATTACK events that eventful produces, and cross reference that with the REPORT events or in-game combat log.  This type of error can be corroborated without my custom tools.

Title: Re: Reversing combat damage
Post by: Grimlocke on September 24, 2016, 03:27:50 pm
This may be a stupid suggestion, but would it help to reduce the maximum framerate to something like 10 instead of 100?
Title: Re: Reversing combat damage
Post by: Roses on September 26, 2016, 12:37:03 am
Have you thought about just having it output the attack information every tick by reading unit.action (I think that's the one). It gets a little slow if you have more than a couple creatures fighting, but you won't get any missing or duplicated attack events that way. It's how I started looking at attack numbers to try and better understand the system. Although I was mainly trying to figure out what the values for the various things were. I made a quick script that just outputs the attack information. So I got an output like,

Code: [Select]
target_id item_id target_bp attack_bp attack_id unk_28 unk_2c unk_30 flags unk_38 unk_3c timer1 timer2 strength agility unit race caste
58 0 -1 7 -1 0 0 0 12 0 20 1 2 1000 1000 52 465 0
57 0 -1 5 -1 0 0 0 12 0 24 1 2 1000 1000 53 465 0
55 -1 8 9 0 2 -1 88 6 103 21 2 3 1000 1000 54 465 0
54 -1 15 9 0 0 -2 144 134 103 18 1 5 1000 1000 55 465 0
63 -1 2 9 0 2 0 97 6 103 26 2 3 1000 1000 56 465 0
53 -1 13 9 0 2 0 48 38 103 26 1 2 1000 1000 57 465 0
52 -1 11 9 0 0 0 78 262 103 24 4 3 1000 1000 58 465 0
52 -1 74 82 6 3 4 96 5 102 25 2 3 1000 1000 59 465 0
66 -1 11 9 0 2 0 84 6 103 19 2 3 1000 1000 60 465 0
65 -1 9 8 1 1 0 80 262 103 25 4 3 1000 1000 61 465 0
64 -1 0 8 1 1 0 97 6 103 26 2 3 1000 1000 62 465 0
56 -1 60 58 4 5 3 138 71 103 23 3 4 1000 1000 63 465 0
Since you have the target_id and target_bp available you can easily get wound information from the target as well.
Title: Re: Reversing combat damage
Post by: Aedom on September 26, 2016, 08:46:11 am
This may be a stupid suggestion, but would it help to reduce the maximum framerate to something like 10 instead of 100?

Sounds like a great idea.  I'll also experiment with keeping things paused and stepping forward one step manually.  Either might be a plausible workaround. Will let you know.
Title: Re: Reversing combat damage
Post by: Aedom on September 26, 2016, 08:47:49 am
Have you thought about just having it output the attack information every tick by reading unit.action (I think that's the one). It gets a little slow if you have more than a couple creatures fighting, but you won't get any missing or duplicated attack events that way. It's how I started looking at attack numbers to try and better understand the system. Although I was mainly trying to figure out what the values for the various things were. I made a quick script that just outputs the attack information. So I got an output like,

Code: [Select]
target_id item_id target_bp attack_bp attack_id unk_28 unk_2c unk_30 flags unk_38 unk_3c timer1 timer2 strength agility unit race caste
58 0 -1 7 -1 0 0 0 12 0 20 1 2 1000 1000 52 465 0
57 0 -1 5 -1 0 0 0 12 0 24 1 2 1000 1000 53 465 0
55 -1 8 9 0 2 -1 88 6 103 21 2 3 1000 1000 54 465 0
54 -1 15 9 0 0 -2 144 134 103 18 1 5 1000 1000 55 465 0
63 -1 2 9 0 2 0 97 6 103 26 2 3 1000 1000 56 465 0
53 -1 13 9 0 2 0 48 38 103 26 1 2 1000 1000 57 465 0
52 -1 11 9 0 0 0 78 262 103 24 4 3 1000 1000 58 465 0
52 -1 74 82 6 3 4 96 5 102 25 2 3 1000 1000 59 465 0
66 -1 11 9 0 2 0 84 6 103 19 2 3 1000 1000 60 465 0
65 -1 9 8 1 1 0 80 262 103 25 4 3 1000 1000 61 465 0
64 -1 0 8 1 1 0 97 6 103 26 2 3 1000 1000 62 465 0
56 -1 60 58 4 5 3 138 71 103 23 3 4 1000 1000 63 465 0
Since you have the target_id and target_bp available you can easily get wound information from the target as well.

Thanks for the fantastic suggestion.  I'll be able to play with the lua side of things either tonight or tomorrow.

If it ends up being significantly slower than usual, I don't see that as a problem.  Our use case is to be able to run arena mode and then export the data for external analysis.

This looks very promising!  Could post your lua script here in a spoiler tag, maybe?  I'd love to see it.
Title: Re: Reversing combat damage
Post by: Aedom on September 26, 2016, 03:39:04 pm
I should also say that I've discovered cases where the attack event order does not agree with the combat log order.  This is very rare, but it can happen.  I have been working on the analyzer app, so that it can detect uncorrelated combat log message. 

In the end, I'm hoping Roses's solution saves me from having to rely on the mere detection of errors.
Title: Re: Reversing combat damage
Post by: Roses on September 26, 2016, 09:04:39 pm
I'm traveling for the next couple of days so I don't have access to my DF stuff. But I can post my code on Wednesday or Thursday for you to look at.
Title: Re: Reversing combat damage
Post by: Aedom on September 27, 2016, 12:57:31 pm
I'm basically finished with set of 1.0 features that I want, but before I show off the final teaser screenshots, I should report on my analysis regarding the event system accuracy.

The event system has been shown to be missing events, duplicating events, and sometimes giving the event out of order.  The heuristic system in the external analyzer app has been able to cope with this to the point that I can reasonably blame the apparent errors on the df hack lua eventful system.  It's a great tool, but unfortunately with its polling mechanics, I don't think it will be able to achieve 100% accuracy.  Repeat, I think that is OKAY for our uses.

I did experiment with turning down the calculation fps to 1, and it had no effect on the error rate.
I did experiment with pausing and stepping forward one frame at a time, it had no effect.
I did experiment with the unit.actions pattern that Roses suggested.  In the end, I decided that our accuracy level is fine without having to resort to taking live samples every tick to ascertain my own high level events.

That said, I just made the external analyzer app detect missing events and highlight them for you.  If we are going to be wrong, at least we can know we are wrong :)  Furthermore, if any great and altruistic person ever took it upon themselves to try to increase the accuracy of eventful, then our tool might be a helpful aide.

With out further ado, I will now show the design I finally went with. 

There are two parts to the solution. 

The first is a combat sniffer lua script that you drop in %DF%/hack/scripts.  To use the script, you go into arena mode and design whatever kind of battle you would like to record.  When ready to start, you run the sniffer in the df hack window.  This creates a "session", which is just a helpful way to group your combat data.  You can also optionally give the session a name.

The result of running the sniffer is a combat-sniffer-log.txt that gets dropped in your %DF% directory next to game log.

The second part of the solution is the external analyzer app.  With it, you can open sniffer log file, and it will show the results in exploded form for your analysis.  I will let the screenshots answer the big questions for me, and you guys can point anything confusing.  The ui design is basically just a 3 tab display, and here they are in order.

Report log display:

Spoiler (click to show/hide)

The Report Log tab highlights any combat log text that looks like combat information.  If we were able to map it to a discrete event, then it is highlighted green.  If we failed to find an event for the line, it is highlighted red.  Sometimes it is red just because I haven't written a black list rule for things like "the elephant gives into pain", but most of the time it is because of the lua event system dropping data.  My analysis of nearly 40 fights has shown ~95% accuracy, which I think is good enough.

The highlighted lines hyper link to the discrete strike data.

Strikes display:

Spoiler (click to show/hide)

This tab shows the stream of recognized attack strikes (no ranged yet), and you can see the resulting wound information, per body part tissue layer.

The attacker/defender lines hyper link to the units display.

Units display:

Spoiler (click to show/hide)

This tab lets you see all the body part information about the units involved, including body parts, tissue layer properties, attacks, armors, and weapons.


Anyway, thanks for anybody still hanging on to this thread.  I can't wait to get it out in beta test so I can see how you guys use/break it :P  I will report back with links and things once I'm ready for launch.
Title: Re: Reversing combat damage
Post by: Aedom on September 27, 2016, 12:59:35 pm
I'm traveling for the next couple of days so I don't have access to my DF stuff. But I can post my code on Wednesday or Thursday for you to look at.

No need.  I settled on my level of accuracy vs amount of data trade off and am moving towards wrapping up 1.0.   I did check out unit.actions, which I did not know about.  Thanks for the pointer.
Title: Re: Reversing combat damage
Post by: Aedom on September 27, 2016, 01:43:27 pm
Combat Analyzer v0.5 is out!

I created a separate thread regarding the tool itself.  That thread can be found here : http://www.bay12forums.com/smf/index.php?topic=160821.0 (http://www.bay12forums.com/smf/index.php?topic=160821.0).

Download : https://github.com/DigitalLibrarian/DfCombatSniffer/releases (https://github.com/DigitalLibrarian/DfCombatSniffer/releases)
Meager Instructions: https://github.com/DigitalLibrarian/DfCombatSniffer/wiki (https://github.com/DigitalLibrarian/DfCombatSniffer/wiki)

The remainder of this thread should be focused on coming up with a mathematical description of how the combat effects are calculated.
Title: Re: Reversing combat damage
Post by: Aedom on September 29, 2016, 08:41:44 am
Here is something interesting that I noticed using the sniffer analyzer tool.  In the Strikes tab, if you click on a wound, there is a property called STRAIN.  I had originally thought these was just the stricken material's strain at yield, chosen from the appropriate material property set. This describes the situation most of the time, however, there cases were this value is clearly dynamic, having values that are fractions of the material strain at yield.

One outstanding example was a case where the strain for a part with torn apart skin was 50010.  The extra 10 shows that some type of calculation is going on.  Other times I have found the strain to be between 0-50000 (for a skin wound), indicating that the calculation is typically done as a fraction of the material strain at yield.

Currently working towards a formula to predict this strain value. 

If you read through the pre-existing literature regarding how momentum passes through the layers, you'll find that the soft tissues (with strain at yield >= 50k) all trigger a "blunt bypass" effect, thereby effectively skipping layers for momentum costs and what not.  If I understand it, it should not be possible to "dent the skin" under this model, because the "blunt bypass" would always be invoked first.  However, if you run enough arena battles, you will see blunt damage dent the skin. 

My hunch is that this blunt bypass feature is triggered off a dynamically calculated strain, rather than picking the value from the material raws, which opens up the possibility to dent the skin in some circumstances.
Title: Re: Reversing combat damage
Post by: Aedom on September 29, 2016, 02:04:23 pm
Here is an interesting case.  Look at the two expanded strike results.  This is from a big free for all where I gave a bunch of dwarves silver maces.  The two highlighted are both the first strikes on an upper arm.  Dwarf 2 swings with a momentum of 136 and Dwarf 7 swings with 145.  However, the results are very different.  In both strikes, each of the tissue results were the same so I only highlighted one in each strike result. 

The big question is:  If Dwarf 7 had more momentum, then how come Dwarf 2's swing managed to bruise the muscle, but Dwarf 7's swing only bruised the fat?

Spoiler (click to show/hide)

Spoiler (click to show/hide)

Clearly there is at least one more factor at play here.  They both did the exact same computed damage to all layers involved.  Perhaps there is an RNG roll at each layer that might stop the attack, or something...
Title: Re: Reversing combat damage
Post by: Roses on September 29, 2016, 04:31:29 pm
Well there are random factors that get applied on each swing, effecting not just velocity and momentum, but the "squareness" of the strike (i.e. if it hits with full force or partial force). In addition there appears to be a sort of damage range that is selected from randomly on top of all of the other factors. Another difference could be the layer volume, was one dwarf fatter than the other? Did they have different attribute values? (i.e. STRENGTH, TOUGHNESS, etc...). Were there exhaustion levels the same? There are also different types of attack (i.e. wild, precise, strong, etc...). Some of these attacks have the same momentum modifiers but do different levels of damage.

From my limited testing I have had a single dwarf make the same attack against a dozen different dwarf clones (same attributes, same fat, same sizes, same everything) and have gotten slightly different results for each attack. Sometimes it's just a small difference with slightly different numbers, but sometimes it can be the difference between damaging different layers, all the way to cutting off an arm versus just cutting down to the muscle. The variability can be astonishing. That's why I started to use code that tracked thousands of attacks and performed statistical analysis on them.

Here is my simple code that tracks attacks
Code: [Select]
dir = dfhack.getDFPath()
io.output(dir..'/test.txt')
io.write('target_id,item_id,target_bp,attack_bp,attack_id,unk_28,unk_2c,unk_30,flags,unk_38,unk_3c,timer1,timer2,unk_4-wrestle_type,unk_4-4,unk_4-6,unk_4-8,unk_4-c,unk_4-10,unk_4-14,strength,agility,unit,race,caste\n')
output = {}
function callback()
 n = #output
 for _,unit in pairs(df.global.world.units.active) do
  output[n] = {}
  for _,action in pairs(unit.actions) do
   if action.type == 1 then
    table.insert(output[n],action.data.attack.target_unit_id)
    table.insert(output[n],action.data.attack.attack_item_id)
    table.insert(output[n],action.data.attack.target_body_part_id)
    table.insert(output[n],action.data.attack.attack_body_part_id)
    table.insert(output[n],action.data.attack.attack_id)
    table.insert(output[n],action.data.attack.unk_28)
    table.insert(output[n],action.data.attack.unk_2c)
    table.insert(output[n],action.data.attack.unk_30)
    table.insert(output[n],action.data.attack.flags)
    table.insert(output[n],action.data.attack.unk_38)
    table.insert(output[n],action.data.attack.unk_3c)
    table.insert(output[n],action.data.attack.timer1)
    table.insert(output[n],action.data.attack.timer2)
    table.insert(output[n],action.data.attack.unk_4.wrestle_type)
    table.insert(output[n],action.data.attack.unk_4.unk_4)
    table.insert(output[n],action.data.attack.unk_4.unk_6)
    table.insert(output[n],action.data.attack.unk_4.unk_8)
    table.insert(output[n],action.data.attack.unk_4.unk_c)
    table.insert(output[n],action.data.attack.unk_4.unk_10)
    table.insert(output[n],action.data.attack.unk_4.unk_14)
    table.insert(output[n],unit.body.physical_attrs.STRENGTH.value)
    table.insert(output[n],unit.body.physical_attrs.AGILITY.value)
    table.insert(output[n],unit.id)
    table.insert(output[n],unit.race)
    table.insert(output[n],unit.caste)
    table.insert(output[n],'\n')
    action.type = -1
    io.write(table.concat(output[n],','))
    n = n + 1
   end
  end
 end
 if n < 10000 then
  dfhack.timeout(1,'ticks',callback)
 else
  print('Done')
  io.close()
 end
end

dfhack.timeout(1,'ticks',callback)
Title: Re: Reversing combat damage
Post by: Aedom on September 30, 2016, 09:47:43 am
Well there are random factors that get applied on each swing, effecting not just velocity and momentum, but the "squareness" of the strike (i.e. if it hits with full force or partial force). In addition there appears to be a sort of damage range that is selected from randomly on top of all of the other factors. Another difference could be the layer volume, was one dwarf fatter than the other? Did they have different attribute values? (i.e. STRENGTH, TOUGHNESS, etc...). Were there exhaustion levels the same? There are also different types of attack (i.e. wild, precise, strong, etc...). Some of these attacks have the same momentum modifiers but do different levels of damage.

From my limited testing I have had a single dwarf make the same attack against a dozen different dwarf clones (same attributes, same fat, same sizes, same everything) and have gotten slightly different results for each attack. Sometimes it's just a small difference with slightly different numbers, but sometimes it can be the difference between damaging different layers, all the way to cutting off an arm versus just cutting down to the muscle. The variability can be astonishing. That's why I started to use code that tracked thousands of attacks and performed statistical analysis on them.

Here is my simple code that tracks attacks
Code: [Select]
dir = dfhack.getDFPath()
io.output(dir..'/test.txt')
io.write('target_id,item_id,target_bp,attack_bp,attack_id,unk_28,unk_2c,unk_30,flags,unk_38,unk_3c,timer1,timer2,unk_4-wrestle_type,unk_4-4,unk_4-6,unk_4-8,unk_4-c,unk_4-10,unk_4-14,strength,agility,unit,race,caste\n')
output = {}
function callback()
 n = #output
 for _,unit in pairs(df.global.world.units.active) do
  output[n] = {}
  for _,action in pairs(unit.actions) do
   if action.type == 1 then
    table.insert(output[n],action.data.attack.target_unit_id)
    table.insert(output[n],action.data.attack.attack_item_id)
    table.insert(output[n],action.data.attack.target_body_part_id)
    table.insert(output[n],action.data.attack.attack_body_part_id)
    table.insert(output[n],action.data.attack.attack_id)
    table.insert(output[n],action.data.attack.unk_28)
    table.insert(output[n],action.data.attack.unk_2c)
    table.insert(output[n],action.data.attack.unk_30)
    table.insert(output[n],action.data.attack.flags)
    table.insert(output[n],action.data.attack.unk_38)
    table.insert(output[n],action.data.attack.unk_3c)
    table.insert(output[n],action.data.attack.timer1)
    table.insert(output[n],action.data.attack.timer2)
    table.insert(output[n],action.data.attack.unk_4.wrestle_type)
    table.insert(output[n],action.data.attack.unk_4.unk_4)
    table.insert(output[n],action.data.attack.unk_4.unk_6)
    table.insert(output[n],action.data.attack.unk_4.unk_8)
    table.insert(output[n],action.data.attack.unk_4.unk_c)
    table.insert(output[n],action.data.attack.unk_4.unk_10)
    table.insert(output[n],action.data.attack.unk_4.unk_14)
    table.insert(output[n],unit.body.physical_attrs.STRENGTH.value)
    table.insert(output[n],unit.body.physical_attrs.AGILITY.value)
    table.insert(output[n],unit.id)
    table.insert(output[n],unit.race)
    table.insert(output[n],unit.caste)
    table.insert(output[n],'\n')
    action.type = -1
    io.write(table.concat(output[n],','))
    n = n + 1
   end
  end
 end
 if n < 10000 then
  dfhack.timeout(1,'ticks',callback)
 else
  print('Done')
  io.close()
 end
end

dfhack.timeout(1,'ticks',callback)

Thanks for those tips on run time variations.  That is exactly the type of stuff that I'm trying to chart out.

I've got a simulator that I use to test these ideas out.  It can read all the df raws and create prototypical creatures, but it doesn't do any of the creature instance-level variation.  All members of a creature species are identical in my sim.  I am capable of creating the lower bounds of strike results that I see if the combat analyzer.  I think I'm going to have to move to a statistical analysis model as you did, in order to make significant progress.
Title: Re: Reversing combat damage
Post by: Aedom on September 30, 2016, 09:53:15 am
Here's another thing I'm stuck on.  I have been focusing on understanding naked dwarves fighting with wooden swords.  I am able to predict most of the behavior I see, except that the combat analysis tool shows me a very surprising value for the sharpness of a wagon wood short sword.

Spoiler (click to show/hide)

It has a value of 500.  However, the WOOD_TEMPLATE has [MAX_EDGE:1000].  I don't understand how it gets this value.  My best guess is that the sharpness might be multiplied by a quality factory, and the arena creates average quality items with this factor = 0.5. 

That's a conjecture.  Does anybody know for sure how this 500 is arrived at as the sharpness for a wagon wood short sword?
Title: Re: Reversing combat damage
Post by: Roses on October 04, 2016, 12:48:40 pm
I think I do remember seeing somewhere that quality affects sharpness.

So I never did ask, what is your end goal for this project?
Title: Re: Reversing combat damage
Post by: Aedom on October 04, 2016, 04:22:10 pm
I think I do remember seeing somewhere that quality affects sharpness.

So I never did ask, what is your end goal for this project?

Oh I'm just having fun and practicing.  I have a parser for the raws and a time simulation that I think is comparable to what df can do.  No promises of anything ultimately coming out of my simulation experiment, but that isn't the whole project.

Along the way, I hope to make some tools to enable the modding effort and also open the model of how the maths operate.  Once this exploration thread has run its course, I was going to see about updating the df wiki with accurate combat model and statistics as another community aid.

One thing that I am going to release in the coming months is an open source df raws parser, that deserializes to .net pocos, suitable for linq queries.  I'm still feeling out the api I want to expose, but I have enough functionality to drive interesting applications already.  My sim is the current test bed for that.
Title: Re: Reversing combat damage
Post by: Roses on October 04, 2016, 08:01:12 pm
Interesting. The reason I asked is because I have always wanted a way to simulate in game combat properly. This would allow us to write scripts in DFHack that would "send" units out on missions and such and have them come back with appropriate wounds.
Title: Re: Reversing combat damage
Post by: Aedom on October 05, 2016, 09:00:58 am
Interesting. The reason I asked is because I have always wanted a way to simulate in game combat properly. This would allow us to write scripts in DFHack that would "send" units out on missions and such and have them come back with appropriate wounds.

My sim is written in .net and is a standalone game engine.  Don't think I can help you there.  However, the model that we come up with could theoretically enable that.

I bet you could get pretty far by just using random wounds, maybe with probability assigned based on combat skills.  Players loves to infer meaning where there isn't any.
Title: Re: Reversing combat damage
Post by: Roses on October 05, 2016, 11:38:54 am
I do have a rough system that basically assigns combat scores to units based on their equipment, stats, etc... and then does some randomization for assigning wounds and such, but I was hoping to make it more consistent with in-game fights. (Ideally we would find out how the world gen battles are simulated and use that same code, but I don't foresee that happening)
Title: Re: Reversing combat damage
Post by: Aedom on October 07, 2016, 10:31:46 am
Here's another one of the interesting cases that I'm currently analyzing.  In this case, a kick attack (which does blunt damage) was performed on a toe.  The nail was shattered.  The interesting part is that the skin was "torn apart", flavor text usually reserved for edged attacks.  The sniffer shows that indeed the nail and the skin layers took cutting damage.  I am starting to suspect when subsequent layers are tested, the previous layer is treated as the strike material, and the stress mode (blunt or edged) might be picked dynamically based on the material/tissue properties.  I suspect this kind of interlayer damage mechanic would only make sense for blunt damage inputs.  Another alternative is that there is an unknown feature that can translate blunt damage back into cutting damage in some cases.


Spoiler (click to show/hide)
Title: Re: Reversing combat damage
Post by: Roses on October 07, 2016, 04:02:38 pm
I have seen instances where I blunt attack breaks a bone and the subsequent injuries are treated as a piercing attack (edged attack really since there are only the two) but usually that information is included in the combat log. Maybe since it was a toe nail it didn't warrant a combat log report? I suppose it would be worth checking if the nail could do the damage to the skin you are seeing.
Title: Re: Reversing combat damage
Post by: Aedom on October 12, 2016, 12:46:02 pm
Does anybody know in what units the strike velocities are reported by Da Vinci's combat.lua script? 

I used the same code to track this number in my sniffer, and I have a .net engine, driven by raws, that correctly produces it in comparable scenarios. I am aiming to be able to cover all the upper bound cases, for "normal" attacks, before adding in those scaling factors.  Now that I'm doing my wrap-up analysis and getting ready to present a model, I realize this is a major hole in my knowledge.
 
The velocities are calculated as a result of the object's mass, the attacker's strength, and the attacker's size, so it isn't obvious to me how to pin distance units to this calculation based on the inputs alone.


An example:

An "average" dwarf can perform a slash move using a copper sword at velocity = 26.  I have been assuming that this was in m/s, so that works out to 58 mph.  I know that golfers can easily swing 100 mph and baseballs can be thrown well upwards of that.  This seems like a reasonable assumption.
Title: Re: Reversing combat damage
Post by: Aedom on October 12, 2016, 12:49:50 pm
I have seen instances where I blunt attack breaks a bone and the subsequent injuries are treated as a piercing attack (edged attack really since there are only the two) but usually that information is included in the combat log. Maybe since it was a toe nail it didn't warrant a combat log report? I suppose it would be worth checking if the nail could do the damage to the skin you are seeing.

I'm still thinking about this.  It seems unlikely that the nail damage was hidden from the combat log.  I say this because hair typically has this pattern.  Damage to hair is typically not ever reported via the combat log.  My understanding is that this is triggered by the HAIR_TEMPLATE having [COSMETIC].  The NAIL_TEMPLATE does not have this attribute. 

While nothing I just said is hard evidence, it suggests to me that something more interesting is going on.
Title: Re: Reversing combat damage
Post by: Roses on October 12, 2016, 03:17:07 pm
I'm not entirely sure that the velocities reported can be taken as a straight m/s, but lacking any other information I would say that is fine.
Title: Re: Reversing combat damage
Post by: Grimlocke on October 15, 2016, 04:20:46 pm
I have seen instances where I blunt attack breaks a bone and the subsequent injuries are treated as a piercing attack (edged attack really since there are only the two) but usually that information is included in the combat log. Maybe since it was a toe nail it didn't warrant a combat log report? I suppose it would be worth checking if the nail could do the damage to the skin you are seeing.

I'm still thinking about this.  It seems unlikely that the nail damage was hidden from the combat log.  I say this because hair typically has this pattern.  Damage to hair is typically not ever reported via the combat log.  My understanding is that this is triggered by the HAIR_TEMPLATE having [COSMETIC].  The NAIL_TEMPLATE does not have this attribute. 

While nothing I just said is hard evidence, it suggests to me that something more interesting is going on.

If I remember correctly, strand-like layers with a material with [IMPACT_STRAIN_AT_YIELD:50000] or higher will be ignored combat-wise, its not just not reported but remains undamaged entirely. This is how the ridiculous invincible 'walking hair' zombies happened.
Title: Re: Reversing combat damage
Post by: Aedom on October 20, 2016, 04:18:03 pm
Hey everybody.  I've been silent for a while, so I'll just post a teaser to give you an idea of my progress. 

This is a screen shot of the list of combat simulation tests that I've been developing for my engine, and the output is within acceptable levels of variation from recorded arena data.  They act like spot checks for all the numerical dynamics we are uncovering.

Spoiler (click to show/hide)

I'm still adding and exploring features, but I think you can see my general method.  Once I have an engine with capabilities on these test points, which are meant to be low bounds, then I can set about formalizing everything into a comprehensive mathematical model that doesn't live only in my code.  Said model documentation will either live here or on the wiki in a simplified form.

Finally, here is some sample combat log output from my program.  I think it is very comparable to DF's output.  Currently, I have a constructive model regarding all the mechanics you can see here.  I still have lots of ancillary features to add, but the broad strokes are there, as you can see.  I am currently only targeting the first strike consequences for a body part.  Cumulative damage will be added as a separate stage of the analysis project.

Spoiler (click to show/hide)


Title: Re: Reversing combat damage
Post by: Aedom on October 21, 2016, 02:46:47 pm
In other forum threads regarding combat damage, there is a common topic of "strain" and, so far, I haven't been able to find any canonical definitions for DF's concept of strain. I will describe my interpretation of this number, which I give a high confidence since my engine uses it to produce output which agrees with DF. It has been working for me, and I use this to calculate dynamic strain (a value different for each strike), as opposed to the static strain that is implied from reading on forums. 

Spoiler (click to show/hide)

Anywho, I just wanted to share this tidbit, because it is a topic that comes up quite a bit, without any known definitive answers.  Again, this will all be formalized, I hope you aren't put off by the little bit of c# code I gave to illustrate the calculation. 

The only thing that this dynamic strain influences in my simulation is that it acts like an extra test before triggering the blunt bypass.  If the stricken layer has STRAIN_AT_YIELD > 50k, then I check to see if the dynamic strain is greater than the STRAIN_AT_YIELD.  If it is, then I trigger the blunt bypass case.  If it is not, then there is no blunt bypass.  This is how you can get wooden sword slashes and stabs that only "dent" the skin.  If it only used the statically defined material property, then it would not be possible to dent the skin.

I also did some major deviations regarding the momentum deductions as damage passes through the layers, but I gotta stop myself somewhere :)

For completeness, you can find the referenced Da Vinci thread here : http://www.bay12forums.com/smf/index.php?topic=142372.0 (http://www.bay12forums.com/smf/index.php?topic=142372.0)

Title: Re: Reversing combat damage
Post by: Roses on October 24, 2016, 08:33:51 pm
It's looking good, just wanted to say that almost all the work DaVinci did was several versions ago, and before much of the attack prepare and recovery, multiple types of attacks, pulping, and several other things. So it's not surprising that you are finding different results and I hope you haven't been following his work too religiously. I was always hoping he would come back and do another pass at the combat mechanics, since he actually delved into the code itself and found the actual equations that were used, but your work seems to be just as good, if not better, since you have taken it a step further and have reproduced the results.
Title: Re: Reversing combat damage
Post by: Aedom on November 04, 2016, 01:35:22 pm
I've haven't forgotten about this project, I have just been real busy with work.  There will be more to come as soon as I can get to it :)
Title: Re: Reversing combat damage
Post by: Aedom on November 16, 2016, 04:26:42 pm
I am only now starting to pick this back up.  The short break allowed me to quickly find a pesky bug that I had been looking for :)

At this point, my combat clone passes a battery of 414 automated tests, testing everything from my raw parser to the finer points of how damage moves through the layers.  I am also generating the combat log text in a manner that is basically identical to DF.

Remaining goals:
 * Discover exact set of rules for how persistent damage (injuries) feeds back into future tissue/material layer performance
 * Add the "coupe" feature where body parts can be jammed into other body parts.
 * Tracking extraneous body state that doesn't directly affect the material simulation (pain, blood, nausea, etc..)
 * Of course testing the engine in wider variety of scenarios, including novel creatures I wouldn't have imagined.  I might use modded raws for this kind of testing.

Once I achieve the stated goals, then I'll have a framework for doing all these calculations, that is able to be fed from the same set of raws that DF consumes.  It is at that point that I will consider my df combat clone to be complete and I'll be ready to formalize and generalize all my findings.  In addition to documentation, this framework and its components could be used to create a great many modder-friendly applications.

Thanks for sticking with us if you are still watching :)
Title: Re: Reversing combat damage
Post by: Aedom on November 18, 2016, 11:53:11 am
Does anybody know of any resources regarding the  "jamming A through B" feature.  I would be interested in anything regarding momentum/stress passing through the layers as well as any info about the resulting wound/contact areas.  I have noticed several interesting outputs around this feature, including a case where the contact area actually increased when jamming a rib through a lung.

Another interesting development regarding this, is that I previously had the belief that a part could only be jammed into other parts "AROUND" it, by first defeating all the layers in that part.  However, if you use the sniffer to examine the wounds on the jammed parts, you can see that they consistently don't take enough damage to actually defeat all the layers in the part being jammed.   They consistently all take a little bit of cutting and denting damage.  I haven't analyzed nearly enough situations to confidently make this claim, but the data is starting to suggest a cap on damage to the jammed part at ~20%.

One thing that does look pretty solid, is that in order to make a part jam into another part, you must at least qualify for the middle tier of material stress result.  For example, if you are doing a blunt attack, then you at least need to be able to initiate a fracture in that rib, in order to get it to jam into the lung.  At this point, the damage seems to always be converted to edge damage, but I wouldn't be surprised if I find some edge cases that take material properties into account.  This is easy to see if you spend some time with wooden whips.  Eventually you'll see a combat line, where the apparent damage to the internal body parts was treated just like it was extra tissue layers.  This only happens because each layer was never technically defeated, only dented or bruised.

It also interesting to note that if you examine jammed parts in the the analyzer, then you can see what appear to be round-off errors on the strain.  The strain is usually taken directly from the material properties, but for these cases you can see that there is an extra small component (0 < x < 10) that seems to be added.  I have noticed this previously, but now I can see that it happens often in the jamming scenario.  It isn't consistent, but may end up being consistent, if we can uncover the rule.  Another explanation is that it might just be small bug in one branch of DF code.  No user is supposed to see this data, and the "error" is small enough to be all but meaningless.

Currently working towards a description of when things are jammed (according to what rule).  So far, I've been able to reproduce all the effects you have seen on this thread without using any random number generation.  I've been pushing that as far back in the development as I can, for obvious reasons.  However, that may very well be why I have been unable to describe the jamming condition.

Title: Re: Reversing combat damage
Post by: Aedom on November 18, 2016, 03:39:37 pm
*redacted*
Title: Re: Reversing combat damage
Post by: Aedom on November 22, 2016, 11:48:23 am
Today's mystery is regarding the wound contact size for blunt attacks.  Wounds have contact area, as well as a "layer wound area".  I haven't fully sussed out the difference yet.  What I find interesting is the case of a blunt strike when the weapon contact area < tissue layer contact area.  When this case happens, you can see a discrepancy between the weapons contact area, and the resulting wound's contact area.  Usually the wound is 1 contact area larger, but I've found cases where it deviated as much as 8/22. I've even found a case where this term seems to be negative.  It always seems to be capped at a percentage of the striker contact area.  I'd guess with a hard-limit of 25% of that for the absolute value.

Contrast with the same situation but with edge damage, and the wound contact area always uses the weapon contact area as an upper bound.

Does anybody know of any features that would result in a blunt weapon making a slightly larger wound than the actual weapon contact area?

I'm starting to see a pipeline.  The jamming condition happens depending on the calculated damage.  The calculated damage comes about as a result of the material stress mechanics + these wounding event transforms, which seem to play a minor role, by tweaking strike contact areas.
Title: Re: Reversing combat damage
Post by: LtGreeneyes on November 22, 2016, 08:38:27 pm
I personally don't have any need for this, but it is extremely interesting to read and I am excited to see what other people do with what you're building. Great work so far. :D It is also interesting to think about other applications of the skills you're developing during this process. Data analysis can be a very interesting field! :D
Title: Re: Reversing combat damage
Post by: Aedom on May 02, 2017, 11:34:29 am
*Redacted because I figured out my own answer*
Title: Re: Reversing combat damage
Post by: Aedom on October 31, 2018, 01:17:30 pm
Well, it has been a long time since I posted here.  The project is still alive and well, but I am only able to work on it sporadically as life imposes.  I am very comfortable with the tissue layer mechanics and I have a working implementation that covers most of the basic injury/material model effects.  Hundreds of striking scenarios are tested, and conform to the measured upper boundary post conditions that I see in DF arena.  I still have issues with very large or small creatures, where my accuracy falls off, but it has only been a serious problem for very small creatures (specifically gremlins with wood daggers are far too deadly in my engine vs their df counterpart)   

I am happy to field general questions about the model, or share code.  There are no immediate plans to publish my formulas as they mostly exist in my tested code at the moment (and I'm just having too much damn fun coding)

I have since moved on to reversing other features to fill out the engine.  I just finished recreating the liquids cellular automaton, with pressure included :) 

Presently, I'm trying to come up with a predictive model for the bleeding values that you can find by examining the injured layers after a cutting attack. Does anybody have any guidance on this?  Surely, there must be some modding guidance that helps to explain these values.   All that I have been able to find is that "bigger values in the [VASCULAR] tag means more blood, as well as having [ARTERIES] and [MAJOR_ARTERTIES]", but I'm interesting in knowing how much blood can come from a given injury, so that I can model blood loss effectively.