After thinking about managing/merging mods over the holidays and reading various threads around here (of special note is probably http://www.bay12forums.com/smf/index.php?topic=142295.0 and the ones concerning PyLNP, PyDwarf, BAMM, Rubble and all the ones I have forgotten), I've had a random idea, that I like quite a bit and that could solve some problems that exist. And now I am stuck waiting for my train home, so I decided to create this thread. Whatever comes out of this will probably be some random side-project, but I'd like to avoid to work on something completely pointless.
Please note that (at least for the context of this post) consider the process of simple (just replacing/adding/deleting a line/tag) as solved, although there are still open questions.
My proposition:
Generate a structure that, let's say represents dwarves and how they work in the raws. (if raws were in xml, those structures would be a XML-Schema (https://en.wikipedia.org/wiki/XML_Schema_(W3C)).)
So basically a file that says what is "allowed" or "compatible" and what's not.
By parsing both the base raws and the raws of the mod file, two structures are created. By comparing those structures I hope to be able to create a strategy for merging those mods or at least detecting their incompatibility.
Such a structure would have to have the following basic properties:
- If two mods have the same structure, they should be very easy to merge
- also: mods that can not be trivially merged should not have the same structure (ideally not even similar)
- If two mods have a similar structure should be reasonably easy to merge
- there is some way of creating a sort of "translation" to allow mods with different structures to be merged correctly
A more specific example:
Please note that this is just an example to get the general point across, I do not know how the structure will look like or what it will do exactly. Those raws will probably not even work.
let's take a look at elves (sorry):
[MANNERISM_EYELIDS]
[SPOUSE_CONVERSION_TARGET]
[CASTE:FEMALE]
[FEMALE]
[MULTIPLE_LITTER_RARE]
[CASTE:MALE]
[MALE]
[SET_BP_GROUP:BY_TYPE:LOWERBODY][BP_ADD_TYPE:GELDABLE]
[BODY_DETAIL_PLAN:FACIAL_HAIR_TISSUE_LAYERS]
[SELECT_CASTE:ALL]
[SET_TL_GROUP:BY_CATEGORY:HEAD:HAIR]
[PLUS_TL_GROUP:BY_CATEGORY:HEAD:CHEEK_WHISKERS]
Here we have two castes, so our structure could look like
<castes>
<female/>
<male>
<geldable/>
<beards/>
</male>
</castes>
<geldable> and <beards> could be structures themselves
Now we introduce a mod called "Elven realism", that does something like this
[MANNERISM_EYELIDS]
[SPOUSE_CONVERSION_TARGET]
[CASTE:FEMALE]
[FEMALE]
[MULTIPLE_LITTER_RARE]
[BODY_DETAIL_PLAN:FACIAL_HAIR_TISSUE_LAYERS]
[CASTE:MALE]
[MALE]
[SELECT_CASTE:ALL]
[SET_TL_GROUP:BY_CATEGORY:HEAD:HAIR]
[PLUS_TL_GROUP:BY_CATEGORY:HEAD:CHEEK_WHISKERS]
Our structure could now look something like this:
<castes>
<female>
<beards/>
</female>
<male/>
</castes>An algorithm could now determine: okay, this is fine, proceed with merging
Along comes mod "Elven mermaids"
[MANNERISM_EYELIDS]
[SPOUSE_CONVERSION_TARGET]
[CASTE:FEMALE_NORMALELF]
[FEMALE]
[MULTIPLE_LITTER_RARE]
[CASTE:FEMALE_ELVENMEREMAID]
[FEMALE]
[MULTIPLE_LITTER_RARE]
[AQUATIC]
[CASTE:MALE]
[MALE]
[SET_BP_GROUP:BY_TYPE:LOWERBODY][BP_ADD_TYPE:GELDABLE]
[BODY_DETAIL_PLAN:FACIAL_HAIR_TISSUE_LAYERS]
[SELECT_CASTE:ALL]
[SET_TL_GROUP:BY_CATEGORY:HEAD:HAIR]
[PLUS_TL_GROUP:BY_CATEGORY:HEAD:CHEEK_WHISKERS]
Boom! Now Stuff breaks. There is simply no way of "correctly" solving a merge between the two mods in a clean, automated way (Do we want bearded ELVENMERMAIDS or not?)
Now, before we get into weird stuff, let's take a look at the respective structure:
<castes>
<femalenormalelf/>
<elvenmermaid/>
<male>
<geldable/>
<beards/>
</male>
</castes>
We have a new structure that does not look too bad when comparing this to vanilla. And indeed, applying just the second mods should not cause many problems, apart from general confusion.
But if we applied the first mod before the second mod we now run into problems. But because we are aware of the changed structure, we can react by for example alerting the user or even just ignoring this and merging anyway.
But the real beauty is when one of the mod authors decide to create a compatibility patch. This patch could just be some sort of "translation" between the two respective structures.
[MANNERISM_EYELIDS]
[SPOUSE_CONVERSION_TARGET]
[CASTE:FEMALE_NORMALELF] {apply changes to [FEMALE] here aswell}
[FEMALE]
[MULTIPLE_LITTER_RARE]
[CASTE:FEMALE_ELVENMEREMAID] {}
[FEMALE]
[MULTIPLE_LITTER_RARE]
[AQUATIC]
[CASTE:MALE]
[MALE]
[SET_BP_GROUP:BY_TYPE:LOWERBODY][BP_ADD_TYPE:GELDABLE]
[BODY_DETAIL_PLAN:FACIAL_HAIR_TISSUE_LAYERS]
[SELECT_CASTE:ALL]
[SET_TL_GROUP:BY_CATEGORY:HEAD:HAIR]
[PLUS_TL_GROUP:BY_CATEGORY:HEAD:CHEEK_WHISKERS]
and the generated structure:
<castes>
<femalenormalelf {apply changes from <female>}/>
<elvenmermaid {apply changes from <female>}/>
<male>
<geldable/>
<beards/>
</male>
</castes>
Great, now we can go ahead and merge, we know where to put our changes to female elves! We can even cascade this, and create a chain of structures to support all sorts of mods simultaneously.
Why do I think this is good?
- It supports "classic mods" because it does not require mods to have a certain structure/language
- worst case: the raws are so different/complex that the created structure is not usable. In this case we can just fall back to other merging methods.
- it can be used to divide mods into smaller parts that can be merged with ease - divide and conquer!
- the merging system can be optimized for small merges and still work great with big mods
- many mods should work out of the box
- a annotation system can ease the creation of compatibility patches, feature just active when a certain mod is present. Also easy graphic packs.
- facilitates modding standards
If all this does not feel very structured, I apologize. It's little more than a random idea at this stage and I have tried not to make it too technical or concrete.
As a modder I'd probably be culled during wold gen, so before I dive into programming and excessive experimentation I'd like to make sure this is not an obviously bad idea from the start. Also someone might have an infinitely better idea.
To this end I'd like to address my biggest concerns:
In order to require as few compatibility patches as possible, the structures have to be as broad as possible but as narrow as necessary to avoid conflicts. I do not know if such a balance can be found, I could be underestimating the complexity of many mods.
Deciding what structures are compatible might not be reliable enough. This can be avoided by not allowing changed structures without patches, but that kind of defeats half of the purpose of this.
I do not want modders to not implement a feature because it would not be compatible with some arbitrary standard created by some script that I wrote.
What do you think?
Would it even help with anything / make any sense or is this a stupid idea?