Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2 3

Author Topic: DF Modloader mockup (and discussion on how to load mods in Steam DF)  (Read 7370 times)

voliol

  • Bay Watcher
    • View Profile
    • Website

I don't recall if it was in a recent interview, FotF reply, or maybe a DF talk, but somewhere Toady was asked about Steam workshop, and what it meant for mod managing. He said he'd take a look at it, and that he was surprised that some kind of utility for managing mods hadn't popped up from the community. Turns out there have been a few attempts at it, but none established themselves firmly as DFhack or Therapist did, and are by now outdated. I've also acquainted myself more closely with how the raws work lately as I developed DF Diagnosipack, So I figured, why don't I give it a try.
Below is my attempt, which I should already disclaim I will not continue working on. My hope is that it is still workable enough to inspire something better, or to build of off to make a long-standing utility. A third wishful outcome to is that the discussion this spawns, regarding how a mod loader should best be laid out, is helpful to the brothers Adams as they are to implement one into Dwarf Fortress proper.

In any case, here is the github repo: https://github.com/voliol/DF-Modloader, and DFFD mirror/direct download: https://dffd.bay12games.com/file.php?id=15633
Oh, and a screenshot:
Spoiler (click to show/hide)

You need Python 3 to run it, nothing more (hopefully).

The design was inspired primarily by the Minecraft resource pack screen. The selected mods are compiled from top going down. Other than just mashing the mods together, there is some functionality in there to (mass) edit raw objects defined "higher up" in the mod loading order, as well as a few new tokens also for the modloader to parse. You can do some pretty powerful stuff, see the example mods in the download.

New "object definitions":
[OBJECT_TYPE:OBJECT_ID]:
The "vanilla" way to define a new object, e.g. [CREATURE:TIGER]. If you define an object with the same OBJECT_TYPE and OBJECT_ID as an already defined one, the old object is overwritten.

[OBJECT_TYPE:EDIT:selection criteria]:
Selects one or multiple already defined objects to be edited. Subsequent tokens are added to the end by default. Valid selection criteria are BY_ID:OBJECT_ID, selecting individual objects by id, BY_CLASS:OBJECT_CLASS, selecting all objects (of the object type) with the given object class (or creature class), BY_TOKEN:TOKEN, selecting all objects with the given token, and BY_TOKEN_PRECISE:TOKEN, selecting all objects with the given token and token values.
E.g. [CREATURE:EDIT:BY_ID:TIGER], [CREATURE:EDIT:BY_CLASS:MAMMAL], [CREATURE:EDIT:BY_TOKEN:LAIR], [CREATURE:EDIT:BY_TOKEN_SPECIFIC:LAIR:SHRINE:100]

[OBJECT_TYPE:REMOVE:selection criteria]:
Removes the selected objects. Takes the same selection criteria as EDIT. e.g. [ENTITY:REMOVE:BY_ID:FOREST]

[OBJECT_CLASS:class name] is a new token. It grants an arbitrary "object class". For creatures, it is synonymous with [CREATURE_CLASS]

The special tokens COPY_TAGS_FROM, GO_TO_END, GO_TO_START, GO_TO_TAG have also been made to work for all objects (not just creatures).

Finally, creature variations have been generalized into "object variations". These work just the same as creature variations do, which is too say they are a little tricky. See the wiki article. The only difference is that instead of "CREATURE_VARIATION" or "CV" object variation-related tokens have "OBJECT_VARIATION" or "OV" in their names. You can also nest an object variation within a creature variation. Creature variations can still be used on creatures.
The full set of object variation-related tokens is:
Spoiler (click to show/hide)


I hope you excuse me for the sloppiness of my writing, I finished up the mockup to a point where I could feel satisfied to let it go pretty late in the evening, and wanted to make this post befor going to bed so I'm done with it. It's affecting the quality for the worse, as my brain gets all tangled. At the time of writing, it is 1 at night for me.

Eric Blank

  • Bay Watcher
  • *Remain calm*
    • View Profile
Re: DF Modloader mockup
« Reply #1 on: August 10, 2021, 01:49:28 pm »

This sounds useful, I'll have to try it out.

So if I wanted to add interactions to all intelligent creatures in the raws I would use:

Object_type:edit:by_token:intelligent
[Can_do_interaction:grant_powers]
   [Cdi:...
Object_type:edit:by_token:can_learn
[Can_do_interaction:grant_powers]
   [Cdi:...
Can I string them together to edit several tokens at once, something like:
Object_type:edit:by_token:intelligent
Object_type_additional:edit:by_token:can_learn
[Can_do_interaction:grant_powers]
   [Cdi:...

How can I make sure my mod loads early in the order (if there is one) and then adds these additions after all other mods that add creatures have been loaded? Will I have to have two different parts, one that adds the Spellcrafts files proper wherever they belong (its only dependency right now is vanilla df raws) and then a second mod that goes in and adds links to Spellcrafts content to all the files loaded before it?
« Last Edit: August 10, 2021, 01:52:45 pm by Eric Blank »
Logged
I make Spellcrafts!
I have no idea where anything is. I have no idea what anything does. This is not merely a madhouse designed by a madman, but a madhouse designed by many madmen, each with an intense hatred for the previous madman's unique flavour of madness.

voliol

  • Bay Watcher
    • View Profile
    • Website
Re: DF Modloader mockup
« Reply #2 on: August 10, 2021, 03:33:20 pm »

You can string them together using PLUS_SELECT, which I notice now I forgot to mention. It only works like an logical or though, not an and, so you can select all creatures that are mammals or poisonous in one swoop, but not ones that are necessarily both at once. Also, ”object_type” should be whatever object type you’re currently working with, so CREATURE if it’s creatures, ENTITY if it’s entities etc.

In your example it would be
[CREATURE:EDIT:BY_TOKEN:INTELLIGENT]
 [CREATURE:PLUS_SELECT:BY_TOKEN:CAN_LEARN]
    [CAN_DO_INTERACTION:...]
    ...

The order is decided by the user, they can arrange the mods in any order (using the up/down/top/bottom buttons). That’s another reason for the dependencies being written out, so the user can do so correctly. And yeah, you would have to split it up into parts. I’m sure there is some better way of presenting that than having the parts be in separate mod folders, but that’s the solution right now, and to some degree the splitting can’t be avoided.

Edit: by suggestion, there's now a github repo: https://github.com/voliol/DF-Modloader

Eric Blank

  • Bay Watcher
  • *Remain calm*
    • View Profile
Re: DF Modloader mockup
« Reply #3 on: August 10, 2021, 05:08:47 pm »

Alright, I'm definitely going to have to try this. Thanks!
Logged
I make Spellcrafts!
I have no idea where anything is. I have no idea what anything does. This is not merely a madhouse designed by a madman, but a madhouse designed by many madmen, each with an intense hatred for the previous madman's unique flavour of madness.

Mr Crabman

  • Bay Watcher
  • A person with the head and pincers of a crab.
    • View Profile
Re: DF Modloader mockup
« Reply #4 on: August 20, 2021, 08:46:20 am »

A third wishful outcome to is that the discussion this spawns, regarding how a mod loader should best be laid out, is helpful to the brothers Adams as they are to implement one into Dwarf Fortress proper.

I've got some things to say about this.

I think there are a couple of missing pieces of functionality that one could add in principle (for the actual Steam workshop, I'm not asking this be added to your program necessarily); as mentioned earlier, PLUS_SELECT doesn't work like an "and", so we could do with some way to do that that does.

Maybe a syntax like this would be good?
Code: [Select]
[CREATURE:EDIT:BY_TOKEN:CARNIVORE:AMPHIBIOUS:BY_CLASS:MAMMAL]
This probably wouldn't need to work with BY_ID for obvious reasons (because that can only ever target one specific object). A "not" could also be useful:
Code: [Select]
[CREATURE:EDIT:BY_TOKEN:CARNIVORE:AMPHIBIOUS:BY_CLASS:MAMMAL]
  [EXCLUDE:BY_TOKEN:ADOPTS_OWNER]

Maybe this is getting kinda into overkill though.

OV_CONVERT_TAG and the original CV_CONVERT_TAG have a big limitation that should probably be accounted for:
Code: [Select]
[CV_CONVERT_TAG]
[CVCT_MASTER:STATE_NAME]
[CVCT_TARGET:gila monster]
[CVCT_REPLACEMENT:gila monster man]

If multiple mods try to convert the same specific tokens, it will be a problem; the mod lowest in priority (loaded first) will end up overriding the later mods (because they won't recognise the target) which is the opposite of how it should be!

If you want your arachnid people to use a custom creature variation, and you put "Custom Spiderperson Pack" on top of the mod stack, you shouldn't have to worry whether the vanilla ANIMAL_PERSON variation is used or if someone else's mod made them into BUG_PERSON, you want to make it to SPIDER_PERSON!

Having a way to specify the target argument by its number (in this case, 1) instead of its value would solve this problem I think. Maybe also the MASTER as well (in case there are multiple)

Finally, one big thing I feel would be config files for mods; having to download one of many different variants of files (does steam workshop even let you host multiple versions of a mod on the same page?) is kind of awkward, and a better solution would be a config file that would be exposed to the GUI ingame (in whatever mod management thing there is).

Then, these values you define in the config file could be used to conditionally enable or disable certain parts of the mod; like so:
Code: [Select]
[IF:DELETE_DWARVES:1]
    [CREATURE:REMOVE:BY_ID:DWARF]
[END_IF:DELETE_DWARVES:1]

And so anything inside here would be skipped if the corresponding config checkbox were disabled. The reason I specify the config in END_IF is so that you could nest different config options inside each other.

You could also potentially allow any argument to be replaced with a !CONFIG_ARG, with an arbitrary number of CONFIG_ARG's, and they would have custom names; what this would allow is for example letting a user enter the value that will be used by POPULATION_NUMBER for a given creature; this could either be a numerical value (which the modder could specify lower and upper limits for), or a dropdown menu (with values definable by the modder in the config file).

voliol

  • Bay Watcher
    • View Profile
    • Website
Re: DF Modloader mockup
« Reply #5 on: August 20, 2021, 01:15:54 pm »

The "and" and "not" seemed easy-ish to implement, so I did. SUBSELECT and UNSELECT both work similarly to PLUS_SELECT, as separate tokens you put after the main selection. See example #4 in the github repo for how it works.

Now, some aspects to how these tokens look/work are not how I'd like them for the Steam version.
I'd actually prefer
Code: [Select]
[EDIT:CREATURE:BY_CLASS:MAMMAL]
 [SUBSELECT:BY_TOKEN:COMMON_DOMESTIC]
 [BIOME:ANY_LAND]
or
Code: [Select]
[EDIT:CREATURE:BY_CLASS:MAMMAL:BY_TOKEN:COMMON_DOMESTIC]
 [BIOME:ANY_LAND]
to
Code: [Select]
[CREATURE:EDIT:BY_CLASS:MAMMAL]
 [CREATURE:SUBSELECT:BY_TOKEN:COMMON_DOMESTIC]
 [BIOME:ANY_LAND]
The last one is just easier/faster to code (or was using the snippets I already had available).

@Mr_Crabman
As far as I know Steam workshop doesn't allow for multiple versions to share the same page, but there's some support to have links to other mods (or collections of mods) in the description.
Your other points are all good/interesting, but I don't have much input to give at this moment.

Mr Crabman

  • Bay Watcher
  • A person with the head and pincers of a crab.
    • View Profile
Re: DF Modloader mockup
« Reply #6 on: August 25, 2021, 08:52:37 am »

As far as I know Steam workshop doesn't allow for multiple versions to share the same page, but there's some support to have links to other mods (or collections of mods) in the description.

That seems a bit too unwieldy to me I think, but this gives me an idea.

You and Erik mentioned before about having to split some mods in 2 to put at different load orders might be solvable by allowing 1 single mod folder to actually be a "modpack", which can contain multiple different mods. Basically, instead of each mod having to be in the topmost mods/ folder, you could do that OR place them in mods/my_modpack_with_any_name/, and be treated by the modloader as separate mods (that can have a different load order set), just bundled all in one folder by Steam.

This technically could also duplicate the function of my [IF] blocks mentioned in the last post, since each "mod" in the "modpack" could be enabled or disabled separately, but having the [IF] may still be more maintainable.

voliol

  • Bay Watcher
    • View Profile
    • Website
Re: DF Modloader mockup
« Reply #7 on: August 25, 2021, 11:33:04 am »

That's clever! I also think you would want to group the mods in the same mod pack while not selected, something like this:
Spoiler (click to show/hide)
That way, it is easy to see which parts of a mod(pack) are left out, either because you have not selected them yet, or because they are optional and you don't plan to.

As for whether having [IF] blocks in addition to this is a good idea, I think it depends on the degree mod splitting is utilized. What I worry is that while [IF] blocks (and in-gui checkboxes) are possibly easier to understand for those without modding knowledge, they are also redundant in regards to functionality, and add another layer for the user to keep track of.
On the other hand, if you have more than a couple of (split) mod parts, selecting them and making sure they're in the right spots will be a hassle. Especially if a few of them are optional and are supposed to go right next to other parts. For those cases, [IF] blocks would really come in handy.

Mr Crabman

  • Bay Watcher
  • A person with the head and pincers of a crab.
    • View Profile
Re: DF Modloader mockup
« Reply #8 on: August 25, 2021, 12:17:32 pm »

That's clever! I also think you would want to group the mods in the same mod pack while not selected, something like this:
Spoiler (click to show/hide)
That way, it is easy to see which parts of a mod(pack) are left out, either because you have not selected them yet, or because they are optional and you don't plan to.

As for whether having [IF] blocks in addition to this is a good idea, I think it depends on the degree mod splitting is utilized. What I worry is that while [IF] blocks (and in-gui checkboxes) are possibly easier to understand for those without modding knowledge, they are also redundant in regards to functionality, and add another layer for the user to keep track of.
On the other hand, if you have more than a couple of (split) mod parts, selecting them and making sure they're in the right spots will be a hassle. Especially if a few of them are optional and are supposed to go right next to other parts. For those cases, [IF] blocks would really come in handy.

The grouping when not selected is probably a good idea; I can imagine having a long list of mods and going crazy trying to figure out which one you got from where, and being annoyed when you accidentally delete a mod you wanted to keep when you unsubscribe from a modpack.

My thoughts on the pros and cons of [IF] are pretty much as you point out; it's sort of redundant, but it's useful to keep everything in one place to as large an extent as possible, so you can just have small optional bits and pieces inside your "main" edit blocks (or brand new object blocks).  Plus, alongside the [IF] could be the more specific value replacements/insertions, and if you're gonna have one type of GUI-configurable value in the raws anyway....

Honestly, while the modpacks do cause some duplication of functionality, the main purpose for them I had in mind was for pieces of the mod that need to have a different load order; maybe they could also be used for entirely separate objects (you could imagine a modpack with separate orc and minotaur civ mods), but I'd expect most optional features within an object (or in the editing of a single object/group of objects) would be better suited to [IF] blocks.

Actually, if you didn't use [IF] blocks, you'd have to rely on the end-user picking the correct load order for your "optional edits" wouldn't you? That's a bit of an unneeded burden I think.

Speaking of load order, how does your solution handle loading changes/how do you imagine it working in vanilla? I was just thinking that there are a lot of weird quirks that could happen; for example, if vanilla goes all first, then you try to edit a vanilla creature variation, it would do nothing because the variation has already been applied to all the creatures.

And for even normal edits, there's a similar potential problem with the likes of COPY_TAGS_FROM and APPLY_CREATURE_VARIATION and so on, because even if you do all creature variations (including in mods) before creatures to solve the previous problem, still when the creature itself in vanilla is processed, the (modded) ANIMAL_PERSON variation is applied, and your later [EDIT] can't change it to [APPLY_CREATURE_VARIATION:NEW_ANIMAL_PERSON] because it's loaded later, and to get the desired result would need a lot of messy cleanup from the aftermath of [APPLY_CREATURE_VARIATION:ANIMAL_PERSON].

How is this dealt with? It seems like paradoxically edits need to be applied before vanilla (and this would extend to other mods that create objects).

voliol

  • Bay Watcher
    • View Profile
    • Website
Re: DF Modloader mockup
« Reply #9 on: August 25, 2021, 01:03:13 pm »

The idea is that Vanilla would work as any other mod, being moveable up and down and even unselected. For some mods, you would want a ”split” vanilla, but this can be something provided by the community. As it currently works in my mockup, later objects overwrite earlier ones. I think it’s a pretty good model; Say you want to want a mod to edit the vanilla animal people creature variations, then this order works:
Code: [Select]
1. Vanilla (including the animal people)
2. Your mod
3. Vanilla animal people (again)
As the animal people are loaded again, they overwrite the previous definition, this time using the changed creature variation. This way, un-split Vanilla can still be used as a base, and updated alongside the game (remember, Steam updates are automatic), and the parts that need to be loaded later being kept in check/provided by the modding community. They will probably know best which parts these are to begin with. This method of overwriting parts of Vanilla with more Vanilla means there will be redundant raws, but the loading times are neglectable so I don’t see it being any issue.

Mr Crabman

  • Bay Watcher
  • A person with the head and pincers of a crab.
    • View Profile
Re: DF Modloader mockup
« Reply #10 on: August 25, 2021, 01:41:01 pm »

The idea is that Vanilla would work as any other mod, being moveable up and down and even unselected. For some mods, you would want a ”split” vanilla, but this can be something provided by the community. As it currently works in my mockup, later objects overwrite earlier ones. I think it’s a pretty good model; Say you want to want a mod to edit the vanilla animal people creature variations, then this order works:
Code: [Select]
1. Vanilla (including the animal people)
2. Your mod
3. Vanilla animal people (again)
As the animal people are loaded again, they overwrite the previous definition, this time using the changed creature variation. This way, un-split Vanilla can still be used as a base, and updated alongside the game (remember, Steam updates are automatic), and the parts that need to be loaded later being kept in check/provided by the modding community. They will probably know best which parts these are to begin with. This method of overwriting parts of Vanilla with more Vanilla means there will be redundant raws, but the loading times are neglectable so I don’t see it being any issue.

I can't really see that working to solve this issue; if you load the vanilla animal people again, then that would solve edits to ANIMAL_PERSON itself maybe, but it wouldn't solve editing the actual creature itself directly.

Code: [Select]
[CREATURE:GIANT_TOAD]
[COPY_TAGS_FROM:TOAD]
[APPLY_CREATURE_VARIATION:GIANT]
[CV_REMOVE_TAG:CHANGE_BODY_SIZE_PERC]
[APPLY_CURRENT_CREATURE_VARIATION]
[GO_TO_END]
[SELECT_CASTE:ALL]
[CHANGE_BODY_SIZE_PERC:100700]
[GO_TO_START]
[NAME:giant toad:giant toads:giant toad]
[CASTE_NAME:giant toad:giant toads:giant toad]
[DESCRIPTION:A huge monster in the shape of a toad.]
[POPULATION_NUMBER:10:20]
[CLUSTER_NUMBER:1:1]
[CREATURE_TILE:'T']
[COLOR:2:0:0]
[PET_EXOTIC]
[PETVALUE:500]
[MOUNT_EXOTIC]
[GO_TO_END]
[PREFSTRING:beauty]
[APPLY_CREATURE_VARIATION:STANDARD_WALK_CRAWL_GAITS:3512:2634:1756:878:4900:6900] 10 kph
[APPLY_CREATURE_VARIATION:STANDARD_CRAWLING_GAITS:9000:8900:8825:8775:9500:9900] 1 kph
[APPLY_CREATURE_VARIATION:STANDARD_SWIMMING_GAITS:9000:8900:8825:8775:9500:9900] 1 kph

Say I wanted to change this:
Code: [Select]
[EDIT:CREATURE:BY_ID:GIANT_TOAD]
[CV_REMOVE_TAG:APPLY_CREATURE_VARIATION:STANDARD_SWIMMING_GAITS]

But by the time my mod is loaded, vanilla already processed everything, and applied STANDARD_SWIMMING_GAITS with the original values; sure, I could add in this extra one now, but the damage has been done, I can't remove it, I'd have to remove each of the newly added `[GAIT:SWIM]` tags directly.

Or:
Code: [Select]
[EDIT:CREATURE:BY_ID:GIANT_TOAD]
[CV_REMOVE_TAG:CV_REMOVE_TAG:CHANGE_BODY_SIZE_PERC]]

Or:
Code: [Select]
[EDIT:CREATURE:BY_ID:TOAD_MAN]
        [CV_REMOVE_TAG:APPLY_CREATURE_VARIATION:ANIMAL_PERSON]
[APPLY_CREATURE_VARIATION:ANIMAL_PERSON_LEGLESS]

You get what I'm going for here? The idea is to change what creature variations are applied in the first place (without simply overwriting the whole object).

Maybe this is the wrong syntax to be trying that with, but can you see the intention? Without the modloader you have no choice but to overwrite, but you can make a tiny little change from ANIMAL_PERSON to ANIMAL_PERSON_LEGLESS, and then the game applies those changes. No need to code stuff into ANIMAL_PERSON_LEGLESS that specifically picks out every change that ANIMAL_PERSON could have made to tweak it, you just change that one tag and that has a knock-on effect.

Your idea there of splitting vanilla, aside from being unwieldy and kind of awkward, doesn't seem like it would even work because reloading the vanilla animal people will just give you vanilla animal people + any knock-on changes caused by editing the ANIMAL_PERSON object in the mod. It doesn't allow editing the TOAD_MAN directly.

voliol

  • Bay Watcher
    • View Profile
    • Website
Re: DF Modloader mockup
« Reply #11 on: August 25, 2021, 02:01:26 pm »

To be honest I misunderstood you, so I answered a separate question. Yeah, changing which creature (or object) variations are used, or changing any other ”special” tokens like COPY_TAGS_FROM is tricky, for the reasons you mentioned. I’m not sure what the way around that would be, maybe reading through the raws twice, one time only looking for changes to special tokens?
Worst come to worst you can redefine the creature with the only difference being the changed creature variation, but that’s not really something you want to do for various reasons (such as it being manual).

Mr Crabman

  • Bay Watcher
  • A person with the head and pincers of a crab.
    • View Profile
Re: DF Modloader mockup
« Reply #12 on: August 25, 2021, 04:01:27 pm »

I’m not sure what the way around that would be, maybe reading through the raws twice, one time only looking for changes to special tokens?

Hm, that sounds like an idea. Maybe there should be something different about editing objects in terms of loading, so that ultimately, the game only loads raw objects once, and the "modloading" process is really just fiddling around with the raw text itself in preparation of actually going through the usual loading process.

So the loading would go like this:

1. Collect all creature variation IDs in the first mod/vanilla (no processing of any tokens yet).
2. Collect all creature variation IDs in the next mod, overwrite the previous objects with the same IDs, and/or apply changes to the creature variations. Repeat 1 and 2 for every mod.
3. Collect all creature IDs in the first mod/vanilla; no processing of any tokens yet, including and especially the special ones like COPY_TAGS_FROM; TOAD_MAN will still just be what you literally read in the raws.
4. Collect all creature IDs in the next mod, overwrite the previous object versions, and/or apply changes to the raw text of the creature objects. Repeat for every mod.
5. Do this same thing for any other objects that come after creatures in raw loading order (I'm pretty sure creatures aren't last).
6. After all mods objects have been collected/added, and all edits applied, now process the objects for real, and now apply all creature variations and fancy stuff.

I think the problem here was assuming that EDIT should work like actual creature variations, such that the object/creature gets processed line by line, and then we do the edits as this happens. But instead, edits should be treated as coming from "the outside" of the object kind of, cutting out lines from vanilla raws, and modifying lines, and inserting them in place.

It wouldn't be possible probably to apply an EDIT to a previous mod's EDIT, instead you would apply the EDIT to what the previous mod left behind; this might feel like we're back where we started, since the original problem was that we would be editing what the CREATURE_VARIATION left behind, but I now think CREATURE_VARIATION has a role to play and a way it functions, and EDIT shouldn't be trying to duplicate it.

Basically, originally, it seems like EDIT was just secretly doing COPY_TAGS_FROM for the original, and then using the tools of a creature variation to make further changes, whereas EDIT should be a tool for mods to make changes to an object smaller than a full overwrite; a crude way of replicating how we right now directly change the raws. Specifying manual changes in a piecemeal and semi-automated way.

Am I making sense? I feel like there should be a clearer way to express this.

voliol

  • Bay Watcher
    • View Profile
    • Website
Re: DF Modloader mockup
« Reply #13 on: August 26, 2021, 08:07:42 am »


I think this might be it! One system for altering the raws at run-time (the creature variations and special tokens), and one system which can interact with those run-time instructions (EDIT). And then some overlap that allows EDIT to do the most common thing the run-time system would do, adding tokens, because that works at both levels, and having to nest all token additions like
Code: [Select]
[CREATURE:EDIT:BY_ID:DWARF]
[CV_ADD_TAG:EXTRAVISION]
[CV_ADD_TAG:FIREIMMUNE_SUPER]
[APPLY_CURRENT_CREATURE_VARIATION]
is a hassle.

I have some questions about the details, but first I should mention that I think creature variations should be generalized for all objects, otherwise token removal and token converts can't be done to them. In the mockup I call these object variations, but the name doesn't really matter. In the rest of this post, everything I say about creature variations will be equally about the generalized form. Creature variations by name (and all their tokens) can still remain as a synonym for the sake of backwards compatibility.

For the first question, why are both points 1&2 and 3&4 needed? They seem to me like they're doing the same thing, and it doesn't need to be read twice if all the creature variations etc. are processed at the end, right?

The second question is about the syntax for changing creature variations (and special tokens like APPLY_CREATURE_VARIATION). Is it optimal to use CV_CONVERT_TOKEN and CV_REMOVE_TOKEN? I don't think so. If the systems are applied separately, they should be delineated when it comes to the tokens used as well. Basically, add copies of all the CV_ tokens to act at the EDIT-level.
Code: [Select]
[EDIT:CREATURE:BY_ID:GIANT_TOAD]
[REMOVE_SPEC_TAG:CV_REMOVE_TAG:CHANGE_BODY_SIZE_PERC]
These may seem redundant, as they could in theory be used for doing all creature variation action at EDIT-level. This is not something we want people to do, because then we've lost the point of EDIT not mimicking what can be done through creature variations. I think the natural solution would be to lock these tokens to only work with CV_ tokens and special tokens, so this example above is possible, but this isn't:
Code: [Select]
[EDIT:CREATURE:BY_ID:GIANT_TOAD]
[REMOVE_SPEC_TAG:PET_EXOTIC]

The third question is how to deal with REMOVE. Should objects be removed during steps 2/4, step 6, or as a last 7:th step? One alternative could be to demote it from what it is in the mockup, something akin to EDIT, to a "special" token which is applied at step 6., and until then just marks it for removal. Then an object could also be "un-removed", say if mod A removes all vanilla entities (in addition to many other features), and mod B is an addon to it for those who still want to play as dwarves. Mod A would at some point include:
Code: [Select]
[ENTITY:EDIT:BY_ID:MOUNTAIN]
[REMOVE_OBJECT]
And mod B would consist of:
Code: [Select]
[ENTITY:EDIT:BY_ID:MOUNTAIN]
[REMOVE_SPEC_TAG:REMOVE_OBJECT]


Edit: I realized that if EDIT goes first, then the EDIT selection won't be able to target depending on tokens added/removed/converted by special tokens. I'm not sure how big an issue this is, though. For one, COPY_TAGS_FROM would mean that even if you're only able to target the base creature (by e.g. a creature class), the changes made by EDIT would be copied over to the animal man and giant versions of the creature.
And you could select by the creature variation tokens as well. Say you want to select all intelligent creatures, then you wouldn't be able to just use [CREATURE:EDIT:BY_TOKEN:INTELLIGENT] because animal people don't have the INTELLIGENT token before their creature variation is applied. But you could do
Code: [Select]
[CREATURE:EDIT:BY_TOKEN:INTELLIGENT]
[CREATURE:PLUS_SELECT:BY_TOKEN_PRECISE:APPLY_CREATURE_VARIATION:ANIMAL_PERSON]
[CREATURE:PLUS_SELECT:BY_TOKEN_PRECISE:APPLY_CREATURE_VARIATION:ANIMAL_PERSON_LEGLESS]
Still, this depends on the creature variations adding said tokens being known, so it might be grounds for rethinking the order.

Mr Crabman

  • Bay Watcher
  • A person with the head and pincers of a crab.
    • View Profile
Re: DF Modloader mockup
« Reply #14 on: August 26, 2021, 01:11:14 pm »

I have some questions about the details, but first I should mention that I think creature variations should be generalized for all objects, otherwise token removal and token converts can't be done to them. In the mockup I call these object variations, but the name doesn't really matter. In the rest of this post, everything I say about creature variations will be equally about the generalized form. Creature variations by name (and all their tokens) can still remain as a synonym for the sake of backwards compatibility.

Maybe so; I'm not 100% sure that creature variations truly need to be generalized to everything; I mean, the functionality of removing and converting tokens of course should be generalized naturally, but for instance, descriptor color patterns don't really require the complexity of object variations (same goes for many other objects).

For the first question, why are both points 1&2 and 3&4 needed? They seem to me like they're doing the same thing, and it doesn't need to be read twice if all the creature variations etc. are processed at the end, right?

The point of having the separate steps was to illustrate how it works with different mods; step 1 is basically the same thing as step 2, but step 2 makes additional overwrites/changes to the data collected from step 1.

In other words, step 1 is gathering the vanilla data, and step 2 is gathering the mods contents and editing what was gathered in step 1. Technically "step 2" would be repeated for every mod.

The second question is about the syntax for changing creature variations (and special tokens like APPLY_CREATURE_VARIATION). Is it optimal to use CV_CONVERT_TOKEN and CV_REMOVE_TOKEN? I don't think so. If the systems are applied separately, they should be delineated when it comes to the tokens used as well. Basically, add copies of all the CV_ tokens to act at the EDIT-level.
Code: [Select]
[EDIT:CREATURE:BY_ID:GIANT_TOAD]
[REMOVE_SPEC_TAG:CV_REMOVE_TAG:CHANGE_BODY_SIZE_PERC]
These may seem redundant, as they could in theory be used for doing all creature variation action at EDIT-level. This is not something we want people to do, because then we've lost the point of EDIT not mimicking what can be done through creature variations. I think the natural solution would be to lock these tokens to only work with CV_ tokens and special tokens, so this example above is possible, but this isn't:
Code: [Select]
[EDIT:CREATURE:BY_ID:GIANT_TOAD]
[REMOVE_SPEC_TAG:PET_EXOTIC]

Hm, you're right in a sense, but I'm not 100% sure on the tag being restricted to special tokens. The idea I had in mind for EDIT is to be like putting code directly in the actual raw text, exactly as if you were typing it in manually, but only in small pieces (so other mods can make their own changes).

So going with your example, to remove PET_EXOTIC, you would have to use [CV_REMOVE_TAG:PET_EXOTIC], which would be inserted into the code, and then after all edits/mods have been processed, only then would PET_EXOTIC truly be removed, correct? You can only directly and immediately remove special tags on the spot (REMOVE_SPEC_TAG never gets added to the real object, it only exists inside the EDIT), for others you can only insert a command to the true object to delete PET_EXOTIC eventually, rather than deleting it immediately.

But I'm not sure this is the right approach; I mean, maybe it is good enough, but I feel uneasy about it for some reason. Could be complications with mods further on in the load order making their own changes maybe.

The third question is how to deal with REMOVE. Should objects be removed during steps 2/4, step 6, or as a last 7:th step? One alternative could be to demote it from what it is in the mockup, something akin to EDIT, to a "special" token which is applied at step 6., and until then just marks it for removal. Then an object could also be "un-removed", say if mod A removes all vanilla entities (in addition to many other features), and mod B is an addon to it for those who still want to play as dwarves. Mod A would at some point include:
Code: [Select]
[ENTITY:EDIT:BY_ID:MOUNTAIN]
[REMOVE_OBJECT]
And mod B would consist of:
Code: [Select]
[ENTITY:EDIT:BY_ID:MOUNTAIN]
[REMOVE_SPEC_TAG:REMOVE_OBJECT]

That's an interesting idea; I originally thought objects should be removed in steps 2/4; when mod A removes an object, mod B could then be able to recreate it; A may remove dwarves, but B adds them back (though they may be different to before, since B has to fully redefine them), whereas if you put B first and A second in the load order, B would "overwrite" the vanilla dwarves, but they would then be deleted by A.

I'm trying to think if there are any potential negative side effects to this approach; my first instinct was that someone could just offer re-adding dwarves using an [IF] block around the removal, but your way has the advantage of letting other mod authors easily undo the removal, as well as not having to manually write/copy in the whole vanilla definition of dwarves again, since this will be more robust to vanilla updates (and also what if you had another mod C installed before A? Then B would need a version that makes the same changes as C).

Edit: I realized that if EDIT goes first, then the EDIT selection won't be able to target depending on tokens added/removed/converted by special tokens. I'm not sure how big an issue this is, though. For one, COPY_TAGS_FROM would mean that even if you're only able to target the base creature (by e.g. a creature class), the changes made by EDIT would be copied over to the animal man and giant versions of the creature.
And you could select by the creature variation tokens as well. Say you want to select all intelligent creatures, then you wouldn't be able to just use [CREATURE:EDIT:BY_TOKEN:INTELLIGENT] because animal people don't have the INTELLIGENT token before their creature variation is applied. But you could do
Code: [Select]
[CREATURE:EDIT:BY_TOKEN:INTELLIGENT]
[CREATURE:PLUS_SELECT:BY_TOKEN_PRECISE:APPLY_CREATURE_VARIATION:ANIMAL_PERSON]
[CREATURE:PLUS_SELECT:BY_TOKEN_PRECISE:APPLY_CREATURE_VARIATION:ANIMAL_PERSON_LEGLESS]
Still, this depends on the creature variations adding said tokens being known, so it might be grounds for rethinking the order.

That's a good point; I mean, if you were to look around the raws with CTRL+F you wouldn't be able to find INTELLIGENT inside animal people, that's something you just have to know and expect with animal people.

Maybe in the same vein that if you want to edit CRAB and CRAB_MAN and GIANT_CRAB all together, you can just select the original CRAB and rely on COPY_TAGS_FROM to do the rest for you, you could edit just the ANIMAL_PERSON creature variation, and use [APPLY_CREATURE_VARIATION:ANIMAL_PERSON] to carry the changes over?

That would definitely be safer than your example with PLUS_SELECT, because what if another mod were to change ANIMAL_PERSON_LEGLESS to not be intelligent anymore?

It's tricky; most tokens are pure inert data (ie declarative), but the creature variation tokens are non-data commands that make changes to the data, and the game ultimately loads in the pure data left over after these commands (this thread complains about the raws mixture of declarative and non-declarative code: http://www.bay12forums.com/smf/index.php?topic=63200.0).

Your example here definitely illustrates an advantage of being able to apply edits to the end result of "pure data", but at the same time my original point was that it's useful to be able to edit the commands, because they ultimately change what the end result/pure data is quite drastically.
« Last Edit: August 26, 2021, 01:12:45 pm by Mr_Crabman »
Logged
Pages: [1] 2 3