Bay 12 Games Forum

Dwarf Fortress => DF Dwarf Mode Discussion => Topic started by: Bjiip on August 01, 2019, 12:28:34 am

Title: Some more military xXScienceXx
Post by: Bjiip on August 01, 2019, 12:28:34 am
Last year, in a previous post (http://www.bay12forums.com/smf/index.php?topic=170878.msg7774920), I did some military science in the arena, using moderately high numbers of 1v1 matches and seeing who wins.

This year, I did some more science.  Some of it was inspired by replies to the previous post- thank you to all who had replied there.

First, a simple bit of recording.  I took a totally unskilled dwarf in the arena, equipped him with basic equipment, and had him fight unarmed goblins one at a time.  As he fought, he gained skills- but not at the same rates.

(https://i.imgur.com/lvW3JTc.png)

Fighter skill advanced the fastest, followed by the weapon skill, as you see.  I did some additional experiments (not so formal, not recorded), but it also seemed that the skills that advanced the fastest were the most valuable in combat (dwarves with skill in Fighter tended to beat dwarves with skill in Axedwarf, and both beat those skilled in Shield user).

Second: how important is skill?  Obviously, more is better: how much better?

(https://i.imgur.com/tOCymmH.png)

The x-axis shows the skill differential, and the y-axis is the win rate.  So if you are 2 skill levels below your opponent, your skill differential is -2.  If you are 3 skill levels above your opponent, your skill differential is +3.  As you might expect, the result is looks like a Sigmoid Function (https://en.wikipedia.org/wiki/Sigmoid_function)- you don't get to a point where you have no chance to win, but it gets smaller and smaller.

In last year's science, I tended to use dwarves.  As the commenters mentioned, dwarves can get a martial trance, which can really skew the results.  You see that in the orange points on the win rate graph; even with a high skill differential, with dwarves, the win rate can't really get above 90%.  That's why I later switched to goblins, who can't trance; there, you see the win rates trend toward 0% and 100%, as you would expect.

The skill differential battles involved one goblin with all Proficient skills, and one goblin with a different skill level (ranging from None to Expert); I calculate the skill differential from that.  It's possible that these curves look slightly different depending on your reference point (for instance, if I were to re-run it with one goblin always having Adept skill, the curve might look different).

Finally, some commenters suggested that I test picks.  I have added those to the win rate by weapon type, for armored dwarves and unarmored dwarves separately; unfortunately, for these tests I had not yet started using goblins.

(https://i.imgur.com/RyWBvVO.png)
(https://i.imgur.com/aIBiPjm.png)

As many commenters expected, the pick performs well in both armored and unarmored cases.  I calculated an overall average win rate (against all enemies in the tests), and the results are:

Armored:
------------------
Hammer: 60.5%
Pick: 53.6%
Axe: 47.7%
Sword: 43.5%
Spear: 42.4%

Unarmored:
-------------
Sword: 58.2%
Pick: 54.0%
Axe: 52.2%
Spear: 49.4%
Hammer: 33.3%

Combined:
------------
Pick: 53.8%
Sword: 50.8%
Axe: 49.9%
Hammer: 46.9%
Spear: 45.9%
Edits:
Additional results:
Effectiveness of Armor, measured in skill levels (http://www.bay12forums.com/smf/index.php?topic=174389.msg8009603#msg8009603)

Recommended reading:
From Iron to Steel (http://www.bay12forums.com/smf/index.php?topic=162260.0)

Spoiler: Methodology (click to show/hide)
Title: Re: Some more military xXScienceXx
Post by: Fleeting Frames on August 01, 2019, 01:23:47 am
 - Your skill progression measurement would be much smoother when using individual exp values, while also getting the benefit of not having to consider how each skill level needs 100 more exp. That requires an utility to get those values, though.

 - It's probably wise to mention methodology; for example, if you didn't mention trances, I wouldn't know to think dwarf faced more than 1 enemy at a time.

That aside, the win rate by skill differential is interesting. Mid-battle powerup can give nearly two to one edge, huh? It's harsher curve than I expected.

I wonder why sword has so good results in unarmored category. Spear stabs better, axe lops off better, pick mines with stronger strikes...I'd expect it to have in-between performance, but it shows an edge over everything else there.
Title: Re: Some more military xXScienceXx
Post by: Bjiip on August 01, 2019, 10:42:15 am
Quote from: Fleeting Frames on August 01, 2019, 01:23:47 am
Your skill progression measurement would be much smoother when using individual exp values, while also getting the benefit of not having to consider how each skill level needs 100 more exp. That requires an utility to get those values, though.
Good point.  Unfortunately, I don't think Dwarf Therapist can connect to arena mode.  That's the only tool I'm familiar with that shows individual exp values.

Quote from: Fleeting Frames on August 01, 2019, 01:23:47 am
It's probably wise to mention methodology; for example, if you didn't mention trances, I wouldn't know to think dwarf faced more than 1 enemy at a time.
As you wish; I'll add it in a spoiler to the original post.  It's mostly the same as in my 2018 thread.

Each dwarf faced only 1 enemy at a time.  Nonetheless, a lucky trance can skew the result.  I think that's why, when testing skill differential with dwarves, even those with high skill differentials can't break 90% or so.  I think some of the low-skill dwarves are getting trances and beating combats against higher-skilled enemies.

Incidentally, this sort of snowball effect is why I focus on 1v1 combats, rather than 10v10 as some do.  The latter would be affected too much by who gets the first kill.

Quote from: Fleeting Frames on August 01, 2019, 01:23:47 am
That aside, the win rate by skill differential is interesting. Mid-battle powerup can give nearly two to one edge, huh? It's harsher curve than I expected.
Harsher than I expected, as well.  A mid-battle powerup may not give quite a two to one edge, depending on how many intermediate injuries happen before the powerup.  If someone is half dead by the time they level up...

Quote from: Fleeting Frames on August 01, 2019, 01:23:47 am
I wonder why sword has so good results in unarmored category. Spear stabs better, axe lops off better, pick mines with stronger strikes...I'd expect it to have in-between performance, but it shows an edge over everything else there.
Nice pun!
I guess people had written swords off as a competitive choice; in From Iron to Steel (http://www.bay12forums.com/smf/index.php?topic=162260.0), I don't see that he even tested them.
Title: Re: Some more military xXScienceXx
Post by: NordicNooob on August 01, 2019, 11:28:36 am
Wonderful !!SCIENCE!!, fellow Urist!

I expect swords are doing really well due to their large contact area, so while the wounds won't be as deep on an unarmored target, they'll bleed much more.

Also, what weapon material/s did you use?
Title: Re: Some more military xXScienceXx
Post by: Josher on August 03, 2019, 01:56:06 am
I'm wondering about how to explain some of the results as well... I wonder if maybe dwarfs have a shorter interval between attacks when using a sword? I also wonder how often getting "first blood" results in victory in the unarmored category. I imagine it's most of the time.
Title: Re: Some more military xXScienceXx
Post by: Ulfarr on August 03, 2019, 03:18:11 am
Quote from: Josher on August 03, 2019, 01:56:06 am
I wonder if maybe dwarfs have a shorter interval between attacks when using a sword?

They shouldn't. Skill levels might affect the interval between attacks but by default all weapons, except whips, have the same values for the [ATTACK_PREPARE_AND_RECOVER] token in the raws.


@Bjiip, would it be possible to test the effect of the different types of attacks that each weapon has? As is in vanilla DF, sharp weapons have both edge and blunt attacks with the blunt type of attacks being arguably suboptimal in most situations* due to the worse area + penetration combo compared to dedicated blunt weapons. My suggestion is to mod in duplicate items (say battle axe and battle axe 2) where the modded ones are identical to their base weapon but without the blunt attacks and then test them against each other.

*at extreme strength levels even these attacks are powerful enough to cause their target to explode in gore.
Title: Re: Some more military xXScienceXx
Post by: anewaname on August 03, 2019, 01:23:59 pm
The sword's Size and a Velocity rating is probably having an effect during the "defensive response" phase. It would have a higher acceleration, giving higher odds of bypassing the block and dodge attempts.

After that, it is all pain.
Title: Re: Some more military xXScienceXx
Post by: Superdorf on August 03, 2019, 02:25:11 pm
I hear getting cut by a really really sharp blade actually doesn't hurt so much when it first happens. Too sudden, too subtle. Not like breaking a bone, say.
Title: Re: Some more military xXScienceXx
Post by: Fleeting Frames on August 03, 2019, 11:36:05 pm
@NordicNooob: Axes have even larger contact area than swords.

@anewaname: Expect swords have same velocities as spears and axes (it's pick that has higher acceleration).

Though, hm, they are the smallest weapons tested. Expect spear performs worse in unarmed than axe and pick, both of which are bigger, so it shouldn't be overpowering.
Title: Re: Some more military xXScienceXx
Post by: TheEqualsE on August 04, 2019, 01:09:03 am
The take away I'm getting from this is no matter what kind the target, spears are only a few percent worse than axes.  I like spears.
Title: Re: Some more military xXScienceXx
Post by: Fleeting Frames on August 04, 2019, 08:33:01 am
I believe the OP only posted about same-size opponents(it'd call that really for fighting a siege; for that situation in particular I'd recommend dual-wielding, but the testing is still useful when deciding what weapon to pick, or how important is armour, or what skills to embark with, or even somewhat on what odds you can expect for your army). Other kinds of opponents...

- Large beasts such as Rocs: Spear+shield has been best in my testing.
- Undead siege: Spears are mediocre but they'll do, based on fragfish's.
- Iron men: Need to be sliced apart, nothing else will work, based on fragfish's.
- So tiny they're immortal undead: Not sure if they're still a problem, I'd guess spears do better than axes here (didn't test, can't find my full notes). I recall whips doing best here while morningstars also being acceptable.
Title: Re: Some more military xXScienceXx
Post by: o_O[WTFace] on August 08, 2019, 10:18:58 pm
Picks perform really well, with no particularly bad matchups.  And thats before even considering that you can train mining skill so much easier then a real weapon.  (they do still use that as their weapon skill right?).

Strike the everything!

Oh right, what about whips?  Last time I played they were lightsabers. 
Title: Re: Some more military xXScienceXx
Post by: Bjiip on August 12, 2019, 01:15:45 am
Quote from: Ulfarr on August 03, 2019, 03:18:11 am
@Bjiip, would it be possible to test the effect of the different types of attacks that each weapon has?

I don't recall for sure, but I recall some science saying that the game prefers edged attacks, so the blunt ones (hitting with the flat of the sword, etc.,) only account for about 3% of attacks.

Quote from: o_O[WTFace] on August 08, 2019, 10:18:58 pm
Oh right, what about whips?  Last time I played they were lightsabers. 

Yes- admittedly, this is a strength of fragfish's work over mine.  He tailored his enemies to stuff that you tend to face in sieges, where mine is more like dwarf versus dwarf, which isn't a situation you actually see.  Future work.

Quote from: NordicNooob on August 01, 2019, 11:28:36 am
Also, what weapon material/s did you use?

Except for the tests that are explicitly about material, I generally use iron.  It's typically plentiful for dwarves, and it's representative of what goblins will use against you too.

Now, for the new results:

I'm working on an idea that anything that gives you a bonus in combat can be measured in skill-level-equivalents (SLEs).  This is my first try with that concept.

(https://i.imgur.com/aJZ5krY.png)

I had two sides of goblins; both had battle axes with shields, and all skills at the same level (e.g., just to be thorough, they had biter, misc. object user, and even hammer user).  One side had full iron armor; the other side had no armor.  How many skill levels higher did the non-armored goblin have to be, to have a 50-50 chance of beating the less-skilled but armored goblin?

The results varied according to the rough skill range.  If the armored goblin has 0 skill levels, then the armor counts as about 0.9 skill levels (in everything).  If the armored goblin has Skilled (IV) skills, then the armor counts as about 1.53 skill levels.  If the armored goblin has Adept (VII) skills, then the armor again counts as 1.53 skill levels.

I think the concept of SLEs probably needs some work- I think a strict additive scale is probably the wrong operator.  For goblins at higher base skill levels, a single level in all their skills doesn't increase their win rate as much.  For goblins with low skill levels, a single level increases their win rate by more.  But I haven't figured out a model for that yet.

One thing I noticed: I picked goblins because they don't get a martial trance.  But I saw that one of the goblins was enraged; is that going to have an effect on the results?
Title: Re: Some more military xXScienceXx
Post by: Sarmatian123 on August 12, 2019, 02:29:04 am
Weapons get stuck in wounds, so combat depends then on raw physical strength of combatants. This is why Dwarves are winning with Goblins so easily. However, skilling wrestling and unarmed combat sucks for some reason, when using weapons.

The wisdom is so far, to get Dwarves into chain mail and train wrestling. When they master it, give them weapons to train with. When they master armor give them shields.

However if you send these training Dwarves or even some civilians into any real combat, always, give them shields, even of lousy quality and icky wooden.

BTW, nice Science!' xXgraphicsXx. ;)
Title: Re: Some more military xXScienceXx
Post by: Fleeting Frames on August 12, 2019, 10:58:58 am
The concept of measuring armor value in combat related to non-armor user skills is a novel one afaik. I'm thinking whether there may be possible something like this: multiply/add together all factors of a combatant for their combat score. divide scores with each other to get win odds. Obviously, this doesn't play too well with the unarmored/sufficiently armored distinction, though.

Enraged does have an effect on result (impacts dodging, accuracy and strength of swing), but seeing as I find it happening with even not particularly angry horses, I'm not sure it's something you can remove.

@Sarmatian123: I'd very much recommend adding a second weapon over a shield for a dwarf unskilled in shield use but skilled with weapons for anything but firebreathers. For siege-type enemies, dual is already superior even when exp is split equally between shield and weapon, and for beast-types, the difference between shield and second weapon in equal skill condition is small enough in favour of shield I bet having something you actually know how to use beats dabbling block chance.

Also, dwarves and goblins have same baseline strength (1250).
Title: Re: Some more military xXScienceXx
Post by: Imic on August 12, 2019, 12:15:35 pm
Would it be possible to equip Miner Dwarves with their Civilian uniform and their Combat uniform at the same time, so that they could simply switch from their Mining pickaxe to their Conbat pickaxe instead of running back and getting a new one? I’m getting ideas for an army of pick-wielders now...
Title: Re: Some more military xXScienceXx
Post by: anewaname on August 12, 2019, 02:02:35 pm
Quote from: Imic on August 12, 2019, 12:15:35 pm
Would it be possible to equip Miner Dwarves with their Civilian uniform and their Combat uniform at the same time, so that they could simply switch from their Mining pickaxe to their Conbat pickaxe instead of running back and getting a new one? I’m getting ideas for an army of pick-wielders now...
Sadly, no. Each dwarf has two uniforms, and each uniform gets a separate pick (Civilian and Military uniforms, where the Civilian uniform is modified by the dwarf's Mining labor). In an emergency, you can choose a dwarf, disable Mining, forbid all picks except the one the dwarf is still holding, put the dwarf in the military, and assign an un-specified pick to the dwarf's uniform. When you unpause, he should drop the pick and immediately pick it up, but this may not work if he is already in combat.

Training a bunch of dwarfs with picks by Mining, then disabling Mining and putting them in the military, is highly enjoyed by some players. My process is to get them to legendary Mining, then give them a uniform with armor but no weapon, train them until they have 5+ in Dodging (if I can), then give them a pick and shield
Title: Re: Some more military xXScienceXx
Post by: Fleeting Frames on August 12, 2019, 04:28:26 pm
For the emergency case, I prefer to disable mining, pass 1 step, then draft them with that pick in particular, to avoid forbidding all picks.

At the same time...I suppose the internal uniforms are accessible to dfhack. No idea what happens if you set them to use same items, but I suspect it wouldn't last and would desync as soon as they'd change anything in their equipment.
Title: Re: Some more military xXScienceXx
Post by: Sarmatian123 on August 13, 2019, 05:31:02 am
Quote from: Fleeting Frames on August 12, 2019, 10:58:58 am
@Sarmatian123: I'd very much recommend adding a second weapon over a shield for a dwarf unskilled in shield use but skilled with weapons for anything but firebreathers. For siege-type enemies, dual is already superior even when exp is split equally between shield and weapon, and for beast-types, the difference between shield and second weapon in equal skill condition is small enough in favour of shield I bet having something you actually know how to use beats dabbling block chance.

Also, dwarves and goblins have same baseline strength (1250).

Dwarves always looked to me more buffy then goblins. Even in leather armor against Goblins in copper armor. Maybe wooden shields is the reason? Yes, Dwarves can cary 2 shields indeed beside an ax or war hammer.
Title: Re: Some more military xXScienceXx
Post by: Fleeting Frames on August 15, 2019, 12:31:39 am
I didn't mean two shields, I meant two weapons. Two wooden shields is somewhat niche in usefulness (mostly armor-user training on wildlife or perhaps to meatshield the super-fragile made of fire fbs).

Dwarves have spatial sense, focus and trances which might be better bonuses than goblins willpower, endurance and average agility.
Title: Re: Some more military xXScienceXx
Post by: Adequate Swimmer on August 15, 2019, 05:16:13 am
Can a character with more than 2 arms benefit from multiple bucklers?
Title: Re: Some more military xXScienceXx
Post by: Fleeting Frames on August 15, 2019, 12:27:20 pm
Given that character with 2 can in adv mode, I'd say yes. There are diminishing returns for each additional shield beyond the first one, though - there's a thread in this forum somewhere.
Title: Re: Some more military xXScienceXx
Post by: seht on August 17, 2019, 12:31:33 am
If the mood takes you...know that any more work of this kind would be dearly loved by some. Comparisons to common foreign weapons, for instance. This is definitely good stuff either way, and a lot of work I'm guessing. I tested body size's influence on highly skilled and equipped matchups, and it was a huge pain in the ass to try to control for everything.