Bay 12 Games Forum

Dwarf Fortress => DF Dwarf Mode Discussion => Topic started by: RocheLimit on April 07, 2017, 10:14:38 pm

Title: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: RocheLimit on April 07, 2017, 10:14:38 pm
This script does the follow:


(5-8-2017 Edit: The missing i's have been found.  It should now work as copied.  Many thanks to Lethosar)
Spoiler (click to show/hide)

It should look something like this:
(5-18-2017 Edit: Note that this code will only look at the first 6 or 7 units of the fort; I will swap it out later with one showing code that checks all the active units Updated image)
Spoiler (click to show/hide)

Backstory:
My generational fort has had a constant issue with siege tags being stuck, forgotten beasts saying they have arrived when they really did not, and years of 1 or 2 herds of wildlife despite having a surface and three caverns.  Using a recent siege whose tag still remained, a recent forgotten beast that stood me up, and PatrikLundell's mention of df.global.world.units.active, I found that not only were the 'missing' units on the active unit list, but their flags could be altered using DFHack. 
Spoiler (click to show/hide)
It turns out that the 'missing' units universally had move_state=false, dead=true, and incoming=true, invaders and forgotten beast alike.  When reversing the aforementioned three flags, and ensuring can_swap is true which *most of the other units had, I was able to manually 'spawn' the unit where it should have been and add it to the Unit List.

I then used PatrikLundell's script for Dismissmerchants (http://www.bay12forums.com/smf/index.php?topic=159297.45) as a base and adapted it to search the unit list for anything with dead=true and incoming=true, then making flag changes to those that fit the parameters. 

Results:
Spoiler (click to show/hide)
Spoiler (click to show/hide)
Spoiler (click to show/hide)
Spoiler (click to show/hide)
Spoiler (click to show/hide)

Kill the retrieved invaders and the siege tag will disappear.

I imagine that, so long as an overseer is prompt in dealing with squatters in the void, no group spawned with this will be very large.

That said, I am unsure of this script's usefulness for my own fort.  I think my computer will die if I unpause with 6000+ Other Units.

(5-18-2017 Edit: For now, it is unlikely it works in the most recent version of DF, 43.05.  I will make a current fort and see what can be done with the script, if anything.)
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: PatrikLundell on April 08, 2017, 05:14:45 am
Interesting info.

You might try to run your modified script with a counter to e.g. release 300 critters at a time, deal with those, and then repeat until the whole lot has been dealt with.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: RocheLimit on April 08, 2017, 08:04:17 am
I hadn't thought of that.  And it works; I changed the iterative formula to only go after the last 200 active units, and pretty much just the last siege and a relative handful of wildlife appeared.

Spoiler (click to show/hide)

Spoiler (click to show/hide)

It will take a while, but I am sure I can work my way through my own fort's backlog now.  Thanks!
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: PatrikLundell on April 08, 2017, 12:44:19 pm
You're welcome!

If your findings help the community it's a decent win in my book.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: RocheLimit on April 10, 2017, 08:32:50 pm
I just tried this script out on some stuck merchants; there were only a handful of merchants and guards at the depot and when I could eventually got a dwarf there to trade a few weeks after they arrived, they were perpetually unloading.

Ran the script for the past 100 active units or so and the rest of the merchant convoy popped out of the void and began making their way to the depot.  Once there, my dwarf was allowed to trade with them. 

So now this script can unclog sieges to allow their complete destruction and remove the siege tag, recover packs of wildlife for fun and profit, and retrieve the rest of a merchant caravan to stop them constantly 'unloading' and allow trade.

It is turning out to be quite versatile.

Spoiler (click to show/hide)

Spoiler (click to show/hide)

Spoiler (click to show/hide)

Spoiler (click to show/hide)

Spoiler (click to show/hide)

Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: nuget102 on April 11, 2017, 09:03:06 am
Would it be possible to set this up to run automatically, like every season or year even?

This script will no doubt help countless people, and I know I'll happily use it.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: lethosor on May 07, 2017, 10:21:45 pm
It looks like the occurrences of "[i]" in your first post are being interpreted as tags, so copying the script won't work as intended. I'd recommend putting it inside a [code] block too:

Code: [Select]
[i]
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: RocheLimit on May 08, 2017, 05:22:56 am
Wow, that was a dumb mistake on my part.  Thank you for pointing that out, Lethosor.

I have edited the code in my first post to be inside a code-tag and the rather important instances of
Code: [Select]
[i] now show.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: azrael4h on May 17, 2017, 10:06:53 pm
It doesn't seem to work in 43.05 (64bit). DFHack doesn't display any errors like there isn't a script, or the text the script is supposed to have, and the siege tag remains active, units list only shows 10 gazelles, and nothing happens. 
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: TheFlame52 on May 17, 2017, 10:54:07 pm
This is awesome. I'm going to use it to fix Bastiongate.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: Ryga_ on May 17, 2017, 11:21:31 pm
PTW

Thanks, been looking for something like this for a while
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: lethosor on May 17, 2017, 11:24:20 pm
It doesn't seem to work in 43.05 (64bit). DFHack doesn't display any errors like there isn't a script, or the text the script is supposed to have, and the siege tag remains active, units list only shows 10 gazelles, and nothing happens.
Is there any output from the script? Did you make sure you copied the entire text, including the last "unitretrieval()" line?
You could try PatrikLundell's script too, I suppose.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: azrael4h on May 18, 2017, 12:46:37 am
No output. I typed in unitretrieval in the console, and it just goes to another line. I just copy-pasted the text in the spoiler tag above:
Spoiler (click to show/hide)

After it didn't work, I edited it to look like the following screenshot. Same thing, no output, no error messages, nothing. (http://i.imgur.com/y870uQs.png)

Dismissmerchants doesn't work either, though it did work on a stuck caravan when I first tested it, so that's not it. I tried fix-unit-occupancy as well. It didn't detect any occupancy issues. Last I walked a squad around the borders of the map, in an attempt to draw them from off the map edge. No one came out.

It's as if the invaders were never created at all.

It's been 8 months straight, siege continues with no end and no actual invaders.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: PatrikLundell on May 18, 2017, 04:22:30 am
I wouldn't exactly be surprised if the scripts we've come up with so far doesn't cover every bugged case.
If you can provide a link to the save I can take a look at it, but I wouldn't hold my hopes up too high.

The modified script version (in the screen shot) isn't good, as it will look at the first 7 units in the list (every time).
If you wanted to just unstick say 10 at a time, you should iterate over the whole range (as the original), but introduce
"local released = 0" directly after "function unitretrieval ()",
"released = released + 1" after "... hidden in ambush = false" and
"if released == 10 then break end" immediately after that (looks nicer if split over 3 lines, though).

However, if trying to process all units failed to produce anything, there is no reason to believe processing a subset would.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: RocheLimit on May 18, 2017, 05:55:38 am
The code Azrael4h pasted was from the original post without the code tag and missing the i's.  And if the code was copied exactly from the screenshot posted after that, then it would only look at the first 6 or so units of the embark.  The first will not work and the second will not look at the besieging units; both my fault, as I did not originally post with the code tag and the screenshot was for a test... I will have to go back and correct the screenshot as well.

(Edit: Well, scratch that.  I notice that the code pasted by Azrael4h was not in a code tag itself; the i's are there.  So while the 2nd code attempted would still only look at the first 6 or 7 active units, the first should have worked.  Statement above: Retracted.)

That said, Azrael4h's case makes me wonder if the script really needs to be looking at the dead tag.  Would it still work if it only looked for the incoming flag? It would make a certain amount of sense if only the incoming flag was relevant.

Spoiler (click to show/hide)

I will try it out later on my own fort, which still occasionally gets missing forgotten beasts/sieges.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: PatrikLundell on May 18, 2017, 06:38:34 am
A good question, RocheLimit... That's one of the parameters I'd probably look at if azrael4h is desperate enough to provide a save link.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: azrael4h on May 18, 2017, 10:25:33 am
One thing that may be happening, if they're spawning in dead, is that the bodies are being deteriorated by deterioratecorpses, which I run every time I play to get rid of vultures and keas. So the bodies may be destroyed at this point, making them unavailable to retrieve. I know that dead unnamed units disappear from the units list with that active.

It's an already buggy save, with phantom trees collapsing on the surface constantly several z levels above ground. I think they're trying to spawn where there's no longer a hill. Or maybe it's because the entire map was on fire at one point. There are empty spaces scattered all over with solid soil below them, and floating corpses and logs in similar spaces that I can't reach. So I just want to retire it at this point.

Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: PatrikLundell on May 18, 2017, 12:25:21 pm
Trees spawning and collapsing is a known issue. As you've seen it can happen under some (unknown?) circumstances that DF doesn't realize that trees and the soil beneath them is gone. Usually this involved cave-ins initially, but probably not always. I think DF believes the trees are due to grow, grows the non existent trees, and then the trees make a Coyote & Roadrunner realization that they're not standing on anything.
It ought to be possible to detect such things and "disable" the bugged trees in the list of plants on the embark, although I'm not aware of anyone having tried that.

I doubt corpse removal has anything to do with it, as corpses are not the creatures, but more the castoff body of a former creature, so the body can be raised as an undead while, at the same time the (citizens only, possibly including residents) creature (the "soul") can rise as a ghost.

Edit: I've looked into the tree cave-in issue:
I managed to create the situation by caving in a large part of the surface (about 40*40) by digging away the soil beneath, build a support, channel away the connecting surface tiles, and cutting the adjacent trees that kept supporting the whole thing with their twigs.
After the cave-in there were a fair bit of collapsing happening, and another indication of something being off was that the roots of some trees close to the edges of the cut area remained (roots magically disappear and are replaced by soil when trees are cut).

I checked the difference between a normal tree and that same tree after it has been cut down (it isn't removed from the plants.all array), and the difference was that the pointer to "tree_info" was null (it's null for saplings as well, by the way).
I saved the game at this stage.
I tried setting the tree_info pointers to null on all trees over the right half of my collapsed area, and all collapses thereafter were over the left half of the area. In addition to that, the left half sported some leaves and fruit on the ground in the same manner as when you dig out the roots under a trunk.
I then went back to the save and set the pointers to null of the left half, and collapses then happened on the right one only.
It can be noted that just setting the pointers to null did not remove the roots that shouldn't have been left in the soil.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: Melting Sky on May 19, 2017, 12:44:05 pm
I think multi-tiled trees are a huge part of the puzzle with the various pathing, missing units and phantom siege bugs that currently plague the game. This is a very nice work around.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: lethosor on May 19, 2017, 01:12:34 pm
I really doubt trees have anything to do with units that aren't on the map, and it should be obvious if they're blocking units that are on the map.

knedl (http://www.bay12forums.com/smf/index.php?topic=139553.msg7452363#msg7452363) thinks that this can be caused by saving/loading right after a caravan or siege shows up, although I don't know if that's the only cause or not.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: lethosor on July 01, 2017, 06:06:24 pm
A bit late here (sorry), but I'm looking at adapting this script for DFHack 0.43.05-r2.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: RocheLimit on July 01, 2017, 09:55:59 pm
A bit late here (sorry), but I'm looking at adapting this script for DFHack 0.43.05-r2.

No need to apologize, Lethosar.  If you could get this script to work for 43.05, I would be extremely grateful.  Albeit with limited testing and scripting knowledge, I have been unable to make it work in 43.05, and had begun viewing this script as only useful for older forts.
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: lethosor on July 01, 2017, 11:17:48 pm
It seems to do what it advertises for caravans in 0.43.05, although it doesn't seem to handle caravan items properly, and the merchants don't appear to be doing much (although that could be due to the caravan being fairly old in this fort). I haven't been able to find broken sieges/wildlife/forgotten beasts to test, but I figure a script that might work is better than no script at all (and I think it's worked for some people that I suggested it to recently).
Title: Re: Standing Siege tags, lost forgotten beasts, and missing wildlife
Post by: PatrikLundell on July 02, 2017, 04:34:26 am
I've made a single test suite of knedl's suggestion by having a save at the beginning of a season and then save immediately when a siege shows up. I tried using both regular and quicksave (which shouldn't make a difference), but neither caused any problems. Obviously, that doesn't mean saving can't cause trouble, only that it doesn't automatically result in broken sieges.