Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2

Author Topic: Some more military xXScienceXx  (Read 5587 times)

Bjiip

  • Bay Watcher
  • Competent Worldsmith
    • View Profile
Some more military xXScienceXx
« on: August 01, 2019, 12:28:34 am »

Last year, in a previous post, I did some military science in the arena, using moderately high numbers of 1v1 matches and seeing who wins.

This year, I did some more science.  Some of it was inspired by replies to the previous post- thank you to all who had replied there.

First, a simple bit of recording.  I took a totally unskilled dwarf in the arena, equipped him with basic equipment, and had him fight unarmed goblins one at a time.  As he fought, he gained skills- but not at the same rates.



Fighter skill advanced the fastest, followed by the weapon skill, as you see.  I did some additional experiments (not so formal, not recorded), but it also seemed that the skills that advanced the fastest were the most valuable in combat (dwarves with skill in Fighter tended to beat dwarves with skill in Axedwarf, and both beat those skilled in Shield user).

Second: how important is skill?  Obviously, more is better: how much better?



The x-axis shows the skill differential, and the y-axis is the win rate.  So if you are 2 skill levels below your opponent, your skill differential is -2.  If you are 3 skill levels above your opponent, your skill differential is +3.  As you might expect, the result is looks like a Sigmoid Function- you don't get to a point where you have no chance to win, but it gets smaller and smaller.

In last year's science, I tended to use dwarves.  As the commenters mentioned, dwarves can get a martial trance, which can really skew the results.  You see that in the orange points on the win rate graph; even with a high skill differential, with dwarves, the win rate can't really get above 90%.  That's why I later switched to goblins, who can't trance; there, you see the win rates trend toward 0% and 100%, as you would expect.

The skill differential battles involved one goblin with all Proficient skills, and one goblin with a different skill level (ranging from None to Expert); I calculate the skill differential from that.  It's possible that these curves look slightly different depending on your reference point (for instance, if I were to re-run it with one goblin always having Adept skill, the curve might look different).

Finally, some commenters suggested that I test picks.  I have added those to the win rate by weapon type, for armored dwarves and unarmored dwarves separately; unfortunately, for these tests I had not yet started using goblins.




As many commenters expected, the pick performs well in both armored and unarmored cases.  I calculated an overall average win rate (against all enemies in the tests), and the results are:

Armored:
------------------
Hammer: 60.5%
Pick: 53.6%
Axe: 47.7%
Sword: 43.5%
Spear: 42.4%

Unarmored:
-------------
Sword: 58.2%
Pick: 54.0%
Axe: 52.2%
Spear: 49.4%
Hammer: 33.3%

Combined:
------------
Pick: 53.8%
Sword: 50.8%
Axe: 49.9%
Hammer: 46.9%
Spear: 45.9%

Edits:
Additional results:
Effectiveness of Armor, measured in skill levels

Recommended reading:
From Iron to Steel

Spoiler: Methodology (click to show/hide)
« Last Edit: August 12, 2019, 01:16:58 am by Bjiip »
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: Some more military xXScienceXx
« Reply #1 on: August 01, 2019, 01:23:47 am »

 - Your skill progression measurement would be much smoother when using individual exp values, while also getting the benefit of not having to consider how each skill level needs 100 more exp. That requires an utility to get those values, though.

 - It's probably wise to mention methodology; for example, if you didn't mention trances, I wouldn't know to think dwarf faced more than 1 enemy at a time.

That aside, the win rate by skill differential is interesting. Mid-battle powerup can give nearly two to one edge, huh? It's harsher curve than I expected.

I wonder why sword has so good results in unarmored category. Spear stabs better, axe lops off better, pick mines with stronger strikes...I'd expect it to have in-between performance, but it shows an edge over everything else there.
« Last Edit: August 01, 2019, 01:29:29 am by Fleeting Frames »
Logged

Bjiip

  • Bay Watcher
  • Competent Worldsmith
    • View Profile
Re: Some more military xXScienceXx
« Reply #2 on: August 01, 2019, 10:42:15 am »

Your skill progression measurement would be much smoother when using individual exp values, while also getting the benefit of not having to consider how each skill level needs 100 more exp. That requires an utility to get those values, though.
Good point.  Unfortunately, I don't think Dwarf Therapist can connect to arena mode.  That's the only tool I'm familiar with that shows individual exp values.

It's probably wise to mention methodology; for example, if you didn't mention trances, I wouldn't know to think dwarf faced more than 1 enemy at a time.
As you wish; I'll add it in a spoiler to the original post.  It's mostly the same as in my 2018 thread.

Each dwarf faced only 1 enemy at a time.  Nonetheless, a lucky trance can skew the result.  I think that's why, when testing skill differential with dwarves, even those with high skill differentials can't break 90% or so.  I think some of the low-skill dwarves are getting trances and beating combats against higher-skilled enemies.

Incidentally, this sort of snowball effect is why I focus on 1v1 combats, rather than 10v10 as some do.  The latter would be affected too much by who gets the first kill.

That aside, the win rate by skill differential is interesting. Mid-battle powerup can give nearly two to one edge, huh? It's harsher curve than I expected.
Harsher than I expected, as well.  A mid-battle powerup may not give quite a two to one edge, depending on how many intermediate injuries happen before the powerup.  If someone is half dead by the time they level up...

I wonder why sword has so good results in unarmored category. Spear stabs better, axe lops off better, pick mines with stronger strikes...I'd expect it to have in-between performance, but it shows an edge over everything else there.
Nice pun!
I guess people had written swords off as a competitive choice; in From Iron to Steel, I don't see that he even tested them.
« Last Edit: August 01, 2019, 10:53:56 am by Bjiip »
Logged

NordicNooob

  • Bay Watcher
  • *Not actually Nordic
    • View Profile
Re: Some more military xXScienceXx
« Reply #3 on: August 01, 2019, 11:28:36 am »

Wonderful !!SCIENCE!!, fellow Urist!

I expect swords are doing really well due to their large contact area, so while the wounds won't be as deep on an unarmored target, they'll bleed much more.

Also, what weapon material/s did you use?
Logged

Josher

  • Bay Watcher
    • View Profile
Re: Some more military xXScienceXx
« Reply #4 on: August 03, 2019, 01:56:06 am »

I'm wondering about how to explain some of the results as well... I wonder if maybe dwarfs have a shorter interval between attacks when using a sword? I also wonder how often getting "first blood" results in victory in the unarmored category. I imagine it's most of the time.
Logged

Ulfarr

  • Bay Watcher
  • Going on a pilgrimage to Mars
    • View Profile
Re: Some more military xXScienceXx
« Reply #5 on: August 03, 2019, 03:18:11 am »

I wonder if maybe dwarfs have a shorter interval between attacks when using a sword?

They shouldn't. Skill levels might affect the interval between attacks but by default all weapons, except whips, have the same values for the [ATTACK_PREPARE_AND_RECOVER] token in the raws.


@Bjiip, would it be possible to test the effect of the different types of attacks that each weapon has? As is in vanilla DF, sharp weapons have both edge and blunt attacks with the blunt type of attacks being arguably suboptimal in most situations* due to the worse area + penetration combo compared to dedicated blunt weapons. My suggestion is to mod in duplicate items (say battle axe and battle axe 2) where the modded ones are identical to their base weapon but without the blunt attacks and then test them against each other.

*at extreme strength levels even these attacks are powerful enough to cause their target to explode in gore.

anewaname

  • Bay Watcher
  • The mattock... My choice for problem solving.
    • View Profile
Re: Some more military xXScienceXx
« Reply #6 on: August 03, 2019, 01:23:59 pm »

The sword's Size and a Velocity rating is probably having an effect during the "defensive response" phase. It would have a higher acceleration, giving higher odds of bypassing the block and dodge attempts.

After that, it is all pain.
Logged
How did I manage to successfully apply the lessons of The Screwtape Letters to my perceptions of big grocery stores?

Superdorf

  • Bay Watcher
  • Soothly we live in mighty years!
    • View Profile
Re: Some more military xXScienceXx
« Reply #7 on: August 03, 2019, 02:25:11 pm »

I hear getting cut by a really really sharp blade actually doesn't hurt so much when it first happens. Too sudden, too subtle. Not like breaking a bone, say.
Logged
Remember kids: a good fortress is a bunker, a great fortress is a tomb!
tormenting the player is important
Sigtext

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: Some more military xXScienceXx
« Reply #8 on: August 03, 2019, 11:36:05 pm »

@NordicNooob: Axes have even larger contact area than swords.

@anewaname: Expect swords have same velocities as spears and axes (it's pick that has higher acceleration).

Though, hm, they are the smallest weapons tested. Expect spear performs worse in unarmed than axe and pick, both of which are bigger, so it shouldn't be overpowering.

TheEqualsE

  • Bay Watcher
    • View Profile
Re: Some more military xXScienceXx
« Reply #9 on: August 04, 2019, 01:09:03 am »

The take away I'm getting from this is no matter what kind the target, spears are only a few percent worse than axes.  I like spears.
Logged
Likes cats for their aloofness.

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: Some more military xXScienceXx
« Reply #10 on: August 04, 2019, 08:33:01 am »

I believe the OP only posted about same-size opponents(it'd call that really for fighting a siege; for that situation in particular I'd recommend dual-wielding, but the testing is still useful when deciding what weapon to pick, or how important is armour, or what skills to embark with, or even somewhat on what odds you can expect for your army). Other kinds of opponents...

- Large beasts such as Rocs: Spear+shield has been best in my testing.
- Undead siege: Spears are mediocre but they'll do, based on fragfish's.
- Iron men: Need to be sliced apart, nothing else will work, based on fragfish's.
- So tiny they're immortal undead: Not sure if they're still a problem, I'd guess spears do better than axes here (didn't test, can't find my full notes). I recall whips doing best here while morningstars also being acceptable.
« Last Edit: August 04, 2019, 08:38:25 am by Fleeting Frames »
Logged

o_O[WTFace]

  • Bay Watcher
    • View Profile
Re: Some more military xXScienceXx
« Reply #11 on: August 08, 2019, 10:18:58 pm »

Picks perform really well, with no particularly bad matchups.  And thats before even considering that you can train mining skill so much easier then a real weapon.  (they do still use that as their weapon skill right?).

Strike the everything!

Oh right, what about whips?  Last time I played they were lightsabers. 
Logged
...likes Dwarf Fortresses for their terrifying features...

Bjiip

  • Bay Watcher
  • Competent Worldsmith
    • View Profile
Re: Some more military xXScienceXx
« Reply #12 on: August 12, 2019, 01:15:45 am »

@Bjiip, would it be possible to test the effect of the different types of attacks that each weapon has?

I don't recall for sure, but I recall some science saying that the game prefers edged attacks, so the blunt ones (hitting with the flat of the sword, etc.,) only account for about 3% of attacks.

Oh right, what about whips?  Last time I played they were lightsabers. 

Yes- admittedly, this is a strength of fragfish's work over mine.  He tailored his enemies to stuff that you tend to face in sieges, where mine is more like dwarf versus dwarf, which isn't a situation you actually see.  Future work.

Also, what weapon material/s did you use?

Except for the tests that are explicitly about material, I generally use iron.  It's typically plentiful for dwarves, and it's representative of what goblins will use against you too.

Now, for the new results:

I'm working on an idea that anything that gives you a bonus in combat can be measured in skill-level-equivalents (SLEs).  This is my first try with that concept.



I had two sides of goblins; both had battle axes with shields, and all skills at the same level (e.g., just to be thorough, they had biter, misc. object user, and even hammer user).  One side had full iron armor; the other side had no armor.  How many skill levels higher did the non-armored goblin have to be, to have a 50-50 chance of beating the less-skilled but armored goblin?

The results varied according to the rough skill range.  If the armored goblin has 0 skill levels, then the armor counts as about 0.9 skill levels (in everything).  If the armored goblin has Skilled (IV) skills, then the armor counts as about 1.53 skill levels.  If the armored goblin has Adept (VII) skills, then the armor again counts as 1.53 skill levels.

I think the concept of SLEs probably needs some work- I think a strict additive scale is probably the wrong operator.  For goblins at higher base skill levels, a single level in all their skills doesn't increase their win rate as much.  For goblins with low skill levels, a single level increases their win rate by more.  But I haven't figured out a model for that yet.

One thing I noticed: I picked goblins because they don't get a martial trance.  But I saw that one of the goblins was enraged; is that going to have an effect on the results?
Logged

Sarmatian123

  • Bay Watcher
    • View Profile
Re: Some more military xXScienceXx
« Reply #13 on: August 12, 2019, 02:29:04 am »

Weapons get stuck in wounds, so combat depends then on raw physical strength of combatants. This is why Dwarves are winning with Goblins so easily. However, skilling wrestling and unarmed combat sucks for some reason, when using weapons.

The wisdom is so far, to get Dwarves into chain mail and train wrestling. When they master it, give them weapons to train with. When they master armor give them shields.

However if you send these training Dwarves or even some civilians into any real combat, always, give them shields, even of lousy quality and icky wooden.

BTW, nice Science!' xXgraphicsXx. ;)
« Last Edit: August 12, 2019, 02:33:15 am by Sarmatian123 »
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: Some more military xXScienceXx
« Reply #14 on: August 12, 2019, 10:58:58 am »

The concept of measuring armor value in combat related to non-armor user skills is a novel one afaik. I'm thinking whether there may be possible something like this: multiply/add together all factors of a combatant for their combat score. divide scores with each other to get win odds. Obviously, this doesn't play too well with the unarmored/sufficiently armored distinction, though.

Enraged does have an effect on result (impacts dodging, accuracy and strength of swing), but seeing as I find it happening with even not particularly angry horses, I'm not sure it's something you can remove.

@Sarmatian123: I'd very much recommend adding a second weapon over a shield for a dwarf unskilled in shield use but skilled with weapons for anything but firebreathers. For siege-type enemies, dual is already superior even when exp is split equally between shield and weapon, and for beast-types, the difference between shield and second weapon in equal skill condition is small enough in favour of shield I bet having something you actually know how to use beats dabbling block chance.

Also, dwarves and goblins have same baseline strength (1250).
« Last Edit: August 12, 2019, 11:02:27 am by Fleeting Frames »
Logged
Pages: [1] 2