Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: 1 ... 5 6 [7] 8 9 ... 24

Author Topic: DFHack plugin embark-assistant  (Read 92111 times)

vjek

  • Bay Watcher
  • If it didn't work, change the world so it does.
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #90 on: June 19, 2019, 09:18:43 am »

Some feature requests for this amazing, time saving tool..
  • Ability to logically OR economic layers (so, for example, calcite or chalk or dolomite or limestone)
    If OR'ing can be done, One more fourth economic choice (to allow for adding all non-marble fluxes to the OR list)
If the above it not possible:
  • Ability to find any/all flux stone except marble (currently, marble is completely inconsistent in reporting and finding both in DF and thus e-a, the other fluxes work fine)
And some very specific things, because some embarks you just want a particular way..
  • Ability to select 'Extremely Flat' as in zero elevation throughout the entire embark square, rather than flat as in 0<->1 or 0<->2. (as it seems to be now?)
  • Ability to find dry caverns (something like only BLOOD_THORN, no other cavern trees)
Not sure if this is possible, pre-embark:
  • Ability to determine if water is in the caverns or not (non-aquifer/non-river water below the bottom-most soil levels, or not)
  • Ability to select specific cavern plants/shrubs and/or cavern trees, or the lack of, as a requirement. (I've found non-dry caverns with only dimple cups, pig tails and cave wheat, which was pretty cool.)

Thanks again, you've saved me hundreds of hours at least, so far.  The save/load feature, in particular, is just.. a thing of beauty.  :o 8)

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #91 on: June 19, 2019, 02:40:03 pm »

I'm not aware of any problem with finding marble. What's the problem, and what are the symptoms?

Various forms of OR choice selections have been discussed before. The issue is primarily a UI one, i.e. how to design the interface such that you can define lists without getting an cluttered mess with no overview.

It's technically possible to determine only if the "dominant" biome of a mid level tile matches that of the other tiles of the embark, as I'm not aware of any way to know if the biome of any surrounding tiles will be present on the tile or not. Thus, to absolutely ensure an embark is perfectly flat, you need to ensure an a rectangle one tile larger in all directions (i.e 3*4 -> 5*7) has all tiles of the same elevation. That also would exclude all embarks hugging the border of the world tile. I usually perform this check manually using the vanilla DF elevation difference display to select an embark tile that has a proper surrounding.

It IS possible to determine if caverns have the Underground or Chasm (i.e. "muddy cavern") biome, but as far as I know the things controlling whether a lake is present or not in a "normal" cavern is determined by a "water" parameter that provides a coverage percentage to be used by the RNG (the "water" parameter is, in turned, generated from the range defined by a world gen parameter pair, available via advanced world gen: I set my caverns to have 0-20% coverage, if I remember correctly, and this means most caverns don't have water, and muddy ones are common).

The cavern biomes seem to work the same as the surface ones, i.e. there's a chance legal things are available or not, which is available in the subsurface region data. Thus, I don't think it would be a technical problem to perform a match check (although the two cavern biomes are currently so boring the tool doesn't look at it), but a UI design one.

The Save/Load feature was implemented as a result of a suggestion/request, by the way.
Logged

vjek

  • Bay Watcher
  • If it didn't work, change the world so it does.
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #92 on: June 19, 2019, 03:43:41 pm »

In most worlds I generate, if the vanilla DF embark screen says flux, there's no flux.
In the cases where that's true, embark-assistant will say there's Marble as the source of flux.  There's no marble.  In the past week alone, I've seen dozens of examples of this, to the point where I don't even count marble as valid, and just skip those worlds where only marbe is found as flux, because it's not actually there.

I have a sample world, and embark, for just such a bug hunt.. (but I can find/gen as many as you want, it's that consistent)
Spoiler (click to show/hide)
The entire south half of the world has the marble-no-marble issue, while a few in the north do as well.
It's not the entire world, though.  Some embarks around the human/NW area that claim to have flux/marble actually do, but the overwhelming majority of embarks in this world don't.
I don't think this is an embark-assistant problem, though, which is why I was asking for a feature request in the way I did, as I suspect this is a DF issue, and embark-assistant is simply passing on what it thinks is valid information. 
In particular, those highlighted embarks in the embark areas above, they all say "Flux stone layer" present, but... there's not, of any kind.  Even without embark-assistant, it's not working as intended, from what I can see.

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #93 on: June 19, 2019, 05:34:31 pm »

Thanks for the info, vjek. I suspect it's true it's some kind of DF issue, but it's one I haven't heard about, so I think it needs an investigation to figure out what it is.

One possibility would be the known DF issue of the magma sea cutting off the geo biome above the layer of interest (marble/iron/...) layer, and some bug in the Embark Assistant's handling of that bug/deficiency. However this is just a guess until I've had time to try to figure out what's going on.
Logged

vjek

  • Bay Watcher
  • If it didn't work, change the world so it does.
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #94 on: June 20, 2019, 12:37:31 am »

I agree.  I think it's related to a single cavern, as well.  Almost all of my worlds only have a single cavern. (instead of the default 3)

I've seen quite a few worlds (with the same adv. worldgen parameters) where there are places that have marble, and only a single layer is visible above the SMR sea, or within the magma sea.  Next to it, or in the next world tile, an adjacent biome won't have any flux/marble, but claim there is some.
In particular, those places tend to be where an uneven embark or different biome crosses, intersects, or abuts.  Under those conditions, with a single cavern, it will often say there is flux/marble but there is none.

I guess it could be as simple as:  The SMR sea is where the marble technically is (or would be, if there were 2 or 3 caverns), but you can't use it or see it, because.. it's the SMR sea.  This might be the conditions required to duplicate the issue?
As a workaround, (an additional option) would it be possible to have embark-assistant, under the Flux menu, find all non-marble flux?  ( N/A, Present, Absent, Non-Marble )
I've never seen Dolomite, Calcite, Limestone, or Chalk show up (or not) in this same way.

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #95 on: June 20, 2019, 08:42:39 am »

I've found the cause of the bug. The logic taken from Prospector somehow missed a crucial line. I'm a bit puzzled, as I'm fairly sure I've tested that the transferred logic worked, so maybe some later change did remove it by mistake.  Regardless of the cause, however, adding/restoring the line caused the logic to work as intended (i.e. not report things [metals, minerals, ...] below the cutoff as present). This means 33 world tiles with flux in your world, rather than most of them.

I'm in the process of creating a pull request for this bug fix, but don't expect a new DFHack to be produced until a fair while after the Villains release (I expect it will take time to map/verify the DFHack mapping of DF structures, as usual).
Logged

vjek

  • Bay Watcher
  • If it didn't work, change the world so it does.
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #96 on: June 20, 2019, 10:49:41 am »

Well look at you being my worldgen hero for the day.  8)  Nice bug hunting!

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #97 on: July 08, 2019, 10:14:23 am »

I've done some DFHack structure exploration, and think I've finally gotten a handle on how the mid level tile "edges" work, i.e. how the biome of tiles flow into adjacent ones. In practical terms, that means it may be possible to actually determine if a mid level tile will get small pieces of different biomes, but there are significant caveats when it comes to actually using that info.

How (I think) it works [Technical, can be skipped]:
- For each mid level tile side, information exists whether the main biome of the tile should be used, or if the neighboring tile's one should be used, i.e. if biomes should flow in or out. In most cases it doesn't matter, as it's the same on both sides, but the code has to handle the situation.
- Similarly, for each corner there's info on which of the 4 tiles that meet there that should be used to provide the corner information for the corresponding corner of all of the 4 tiles.
- There's info on how the sides of a tile is split into one side section and 2 corner section (the corners need their other side's info to be completely defined).
- The split of sides into sections for a starting point for DF's (RNG seed driven) algorithm for fighting it out between the "intruder" and "defending" biome, as well as between different "intruders". This can result in the boundary between sections ending up in different places from the one defined, and it can also have the "intruder" pushed back completely along part of the line. This means the "foreign" biomes can vary much in size, with no external info on how large they'll be.
- I GUESS an "intruder" biome can only be pushed back to the border, but not spilled over backwards (i.e. that the intrusion direction will always hold, so you can reliably predict that no foreign biome will appear along a section).
- I furthermore GUESS that is should be possible, although probably extremely rare, that an "intruder" biome is completely expelled, resulting in no foreign tile where the initial info stated there should be one. If it happened, there would be a completely straight border line for that section, assuming the previous guess holds.
- The term "biome" isn't completely accurate, as apart from the biome and geo biome, the information also covers Elevation, which DF defined on a mid level tile level, and so can vary even if the actual biome is the same.

Complications:
- The mid level tiles at the border of each world tile would need to potentially import information from mid level tiles belonging to another world tile. While this isn't a problem after embark, it is pre embark, because then this information exists only for a single world tiles at that time (3*3 world tiles post embark). This is a rather significant issue, because taking all information into consideration would require either a very significant cashing of information bloating the memory footprint, or scanning the world twice on the first search (the relevant border information can probably be cashed during the first scan, as it shouldn't be too much data), because regardless of which order you scan the world in, world tiles bordering the unscanned tiles rely on information that DF hasn't generated yet for those tiles. A compromise of sorts is to not consider the border tiles for embarks during the first scan.
- When do we want to take "foreign" biome incursions into mid level tiles into consideration? That's actually not a straight forward issue: For getting a flat embark we definitely want to take it into account, but if you're looking for iron, you won't be very happy to find the "iron" consists of a dozen in-game tiles of the geo biome, and they may not even have any vein in it...

- Savagery/Evilness parameters: Most of the time you probably want incursions to be taken into account.
- Aquifer: You probably want to take it into account all the time.
- Flat: All the time.
- Clay and Sand only need a single tile to access the resource, but incursions tend to be (dangerously) close to the border, but all the time would probably be good enough.
- Coal/Flux: Incursions should probably be ignored, as you want decent volumes.
- Soil: Don't know...
- Evil/Syndrome rain/reanimation: Probably situation dependent...
- Min/Max Biome count, region type, biome: You'd probably want to take incursions into account, but will be unhappy if you were aiming for something that isn't present in that small section...
- Metal/Economic/Mineral: Same as Coal/Flux: Incursions are typically too small to be of interest.

Note that adding switches for whether to take incursions into consideration on a selection item per selection item basis would result in a selection list bloat which would make the UI unwieldier than necessary.
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #98 on: July 08, 2019, 11:01:17 am »

Regarding checking for iron, it's probably unnecessary to say but JSYK veins/clusters/blobs will always depend on the 'main' geo biome of the 48x48 tile square they sit in. As far as I've seen, it is not possible to have a tile of aluminium and blob of magnetite share that block. Coals, same story.

As for flux, incursions can contain significant amount of it; ex over 2k in my 1x1 - because its height is 7 z-levels. Then again, it depends on what one considers decent; that's not enough to build even half of a proper tower.

Soil, I think you and I will think incursion, and some will be happy with single tree for bootstrapping. Others may want larger area for sustainably forestry.

I'd say partial reanimation is lot like partial aquifers, but that's just me - it's going to be a source of undead wildlife, at least.


The technical part is why the mountain elevation as it descends can look like U-shaped spikes, I assume.

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #99 on: July 08, 2019, 11:24:30 am »

I didn't know incursions couldn't contain veins.

Incursions CAN contain significant amounts, but they can also contain almost nothing: you could have gotten 50 tiles (12.5 boulders) total instead. Also, if I remember correctly, some kinds of flux come as veins.

It should be noted that according to the DF structure XML file, there are special rules concerning mountains and lakes/oceans: lakes and oceans are supposed to yield to anything else regardless of what the instructions say, and mountains are supposed to yield to everything except lakes and oceans. I.e. land tiles can intrude on lake/ocean/mountain tiles, but they can never intrude on normal tiles.
Logged

Fleeting Frames

  • Bay Watcher
  • Spooky cart at distance
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #100 on: July 08, 2019, 11:42:04 am »

Yeah, that's "almost none" amount. Calcite is the flux that comes as vein-type - but it only happens as small clusters within limestone and marble (both layer fluxes), so....Well, I suppose one could mod other flux veins, though I haven't heard of anybody doing so.

Hm. Good to know about l/o/m; it does seem to match my experience.

Telear

  • Escaped Lunatic
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #101 on: August 22, 2019, 06:37:15 am »

One feature I'd love to see in added is the option to do 'embark-assistant search' or some such, which would start embark assistant searching using the saved set of desired features and return a count of matching tiles. This would be enormously helpful when I'm building a whole bunch of worlds and looking for the mythical 3x3 with volcano, by a waterfall with flux and iron. (Yes, I know about region manipulator and its ilk, but I'm running very short histories with the idea of regenerating the hits with a longer history, and RM doesn't really help with that).
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #102 on: August 22, 2019, 07:54:53 am »

For that purpose I'd probably write a script that just looks for the desired features around the few volcanoes in the world. A Lua script would likely be faster than a search using the compiled plugin because you'd only look a relatively few potential embarks.
A script could speed things up further by checking if there's a river in the same world tile as the volcano, and skip those that don't have any.

Regardless of the approach, however, you'd have the task of starting the save in pre embark mode, run the script/plugin, and then wait for it to complete (which takes time in large worlds). Using the plugin as it stands currently you'd have to load your saved parameters, enter search mode, and start a search, which isn't many additional key presses.
Logged

Telear

  • Escaped Lunatic
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #103 on: August 22, 2019, 10:09:41 am »

I'm aiming at eliminating the keypresses entirely. With a script that emits the number of matching tiles, I can write an AHK or Mac Automator thing that will:

Code: [Select]
START:dwarfort -gen 99 RANDOM PROFILE
      Launch dfhack
      Open region99
      dfhack-run search-script
      unless result matches 'found 0 tiles' exit
      dfhack-run die
      rm -rf data/save/region1
      run this workflow again

and just leave it running all night. Right now, I don't know how to write that script because I don't know how to spot that embark-assistant stopped running or how to grab the number of matching tiles from the df window (though I suppose running in ASCII mode might make that a bit more tractable - wait until there are no more yellow Xs on the screen and check the matching tile count).
Logged

PatrikLundell

  • Bay Watcher
    • View Profile
Re: DFHack plugin embark-assistant
« Reply #104 on: August 23, 2019, 02:42:19 am »

- I'm not familiar with either of the script thingies, and guess the code is just a mock up, but region99 doesn't match up with removal of region1 in case it isn't.
- There won't be any yellow 'X' before the plugin has generated any, so you'd need to at least wait sufficiently long for the top level survey to be completed before checking.

I'll think about the issue, but won't promise anything.
Logged
Pages: 1 ... 5 6 [7] 8 9 ... 24