Bay 12 Games Forum
Dwarf Fortress => DF Suggestions => Topic started by: Evans on November 14, 2020, 02:39:32 pm
-
Rather than just copy-paste into game raws, there should be:
Mod folder
Any subfolder there should be treated as separate mod. Files from there could be read as separate entities.
mod.txt for mod loading order.
'Override', for modified vanilla files and entries.
'Remove', for deleted vanilla files and entries.
This would make life easier for everybody.
Also, logging what mods load what and in what order, also eventual crashes.
-
Rather than just copy-paste into game raws, there should be:
Mod folder
Any subfolder there should be treated as separate mod. Files from there could be read as separate entities.
mod.txt for mod loading order.
'Override', for modified vanilla files and entries.
'Remove', for deleted vanilla files and entries.
This would make life easier for everybody.
Also, logging what mods load what and in what order, also eventual crashes.
Everything is changing for mods to make it compatible with Steam Workshop. Current system won't work at all and pretty much what you're suggesting will be implemented.
-
Rather than just copy-paste into game raws, there should be:
Mod folder
Any subfolder there should be treated as separate mod. Files from there could be read as separate entities.
mod.txt for mod loading order.
'Override', for modified vanilla files and entries.
'Remove', for deleted vanilla files and entries.
This would make life easier for everybody.
Also, logging what mods load what and in what order, also eventual crashes.
Everything is changing for mods to make it compatible with Steam Workshop. Current system won't work at all and pretty much what you're suggesting will be implemented.
Including logging, override and remove?
-
Rather than just copy-paste into game raws, there should be:
Mod folder
Any subfolder there should be treated as separate mod. Files from there could be read as separate entities.
mod.txt for mod loading order.
'Override', for modified vanilla files and entries.
'Remove', for deleted vanilla files and entries.
This would make life easier for everybody.
Also, logging what mods load what and in what order, also eventual crashes.
Everything is changing for mods to make it compatible with Steam Workshop. Current system won't work at all and pretty much what you're suggesting will be implemented.
Including logging, override and remove?
Seems like Workshop won't work without it (no idea of the exact requirements), so it sounds like it would. Hasn't started yet though, of course so mostly just musing in the various interviews on the subject.
-
Including logging, override and remove?
Seems like Workshop won't work without it (no idea of the exact requirements), so it sounds like it would. Hasn't started yet though, of course so mostly just musing in the various interviews on the subject.
I'm not talking about workshop logging and functionality, but in game script handling :)
-
Darkhog made a similar thread a few months back (http://www.bay12forums.com/smf/index.php?topic=174724) (and so did Thansin in 2010 (!) (http://www.bay12forums.com/smf/index.php?topic=49030)), but it's still worth discussing as the work on it is getting closer and closer.
Reworking the mod structure in classic as well to match with whatever Workshop needs, would be the most sound option.
My guess is that all Workshop does is provide the mod files when you click the "subscribe" button, and which one should be removed when you click "unsubscribe", and then it's up to the game's handler to actually put the files somewhere sensible/install them. I doubt there is a standardized structure for mods on Steam. That said, both versions could have a folder with subfolders for each mod, and a logging file. Using Workshop as an interface, you could install mods to this folder, as well as remove them. If you don't have premium, you can either manually put mods in that folder, or do it through whatever in-game interface allows you to enable/disable/order the installed mods. Darkhog made the comparison to how Minecraft handles resource packs, and I think it's a pretty good one.
I wonder how such a system would work for applying mods to already generated worlds though. It's important that saves are allowed to use different sets of mods, especially as downloading whole versions of DF will no longer be an "easy" way to try out mods, as getting that modded version into Steam won't be easy, and the mods themselves can't contain Mephday/the new music. Removing a mod because you don't want it in your new world can't break older saves either.
Another issue is that Steam Worshop has you "subscribe" to the mods, instead of simply installing them. This means modders can update their mods and the subscriber automatically gets the updated version, without any further downloads. This is generally a good thing, but if you allow active saves to have their mods updated, there is a risk of them breaking due to this, and if you don't people will miss out on minor updates that would be compatible with their saves, which is a shame.
-
Including logging, override and remove?
Seems like Workshop won't work without it (no idea of the exact requirements), so it sounds like it would. Hasn't started yet though, of course so mostly just musing in the various interviews on the subject.
I'm not talking about workshop logging and functionality, but in game script handling :)
The steam workshop cannot work with an overhaul of in-game mod handling. Mods cannot be combined as of now, but they must be combined for the steam workshop. That's what's being said here. The way mods are loaded needs to be changed in and of itself.
-
ptw
-
Darkhog made a similar thread a few months back (http://www.bay12forums.com/smf/index.php?topic=174724) (and so did Thansin in 2010 (!) (http://www.bay12forums.com/smf/index.php?topic=49030)), but it's still worth discussing as the work on it is getting closer and closer.
Reworking the mod structure in classic as well to match with whatever Workshop needs, would be the most sound option.
My guess is that all Workshop does is provide the mod files when you click the "subscribe" button, and which one should be removed when you click "unsubscribe",
This is exactly what happens, but:
and then it's up to the game's handler to actually put the files somewhere sensible/install them. I doubt there is a standardized structure for mods on Steam.
There is a steam workshop folder for a given app, where mods are downloaded to by default, and:
That said, both versions could have a folder with subfolders for each mod, and a logging file.
Steam workshop has build in logging, which is separate from any game logging as it is handled by the client, and:
Using Workshop as an interface, you could install mods to this folder, as well as remove them. If you don't have premium, you can either manually put mods in that folder, or do it through whatever in-game interface allows you to enable/disable/order the installed mods. Darkhog made the comparison to how Minecraft handles resource packs, and I think it's a pretty good one.
Workshop is basically a centralized internal file download system that has very few rules (game id, mod id, basic location, having a game etc).
How the game handles modding, how the game handles scripts depends on how modding is implemented in a given game.
For example: in my mod EvansDF my 'CAT' entity can be different from VoliolDF 'CAT' entity due to the virtue of belonging to separate pack.
Or they could be the same entity conflicting. Or they could be the same entity loaded in the order depending on how mods are loaded.
Or they could use 'override' system, and your would override default cat, mine would be a separate entity.
Considering that modders creating mods rarely keep their files neatly organized and their entities properly named, while we will name ours EVANSDF_CAT and VOLIOLDF_CAT to make them stand out, I assure you the next hundred guys from steam will go with just a 'CAT'.
This is a rather serious issue to be considered, as there are tons of DF mods out there already. Setting up the stage for massive mod conflicts is bad idea.
I wonder how such a system would work for applying mods to already generated worlds though. It's important that saves are allowed to use different sets of mods, especially as downloading whole versions of DF will no longer be an "easy" way to try out mods, as getting that modded version into Steam won't be easy, and the mods themselves can't contain Mephday/the new music. Removing a mod because you don't want it in your new world can't break older saves either.
That is another issue here. Saves could use mod list to load rather than copy-paste of raws (that was dumb idea to begin with, even tho I understand why it made sense back then). Some sort of default item/creature/recipe for replacing missing definition could be present. It's not that hard really, to prevent game from crashing because some CAT definition is missing, but it could make game act funny later.
Another issue is that Steam Worshop has you "subscribe" to the mods, instead of simply installing them. This means modders can update their mods and the subscriber automatically gets the updated version, without any further downloads. This is generally a good thing, but if you allow active saves to have their mods updated, there is a risk of them breaking due to this, and if you don't people will miss out on minor updates that would be compatible with their saves, which is a shame.
Aye, and DF saves are not for 15 minute long games either. So copying raws makes sense here. Or not.
The thing is, the way game treats definitions NOW is as if they were hardcoded once world is created. Its bad.
It was bad to begin with (we can still hear a cry of thousand modder souls for being unable to add in one recipe into an existing world).
This will have to be changed and handled somehow.
I think leaving references to '1001 sun berry dishes' in the save game, while disabling the recipes when mod is uninstalled, instead of outright deleting these references and/or crashing (so that when mod is re downloaded later it just re-enables them) would be the right way to go.
I'm not talking about workshop logging and functionality, but in game script handling :)
The steam workshop cannot work with an overhaul of in-game mod handling. Mods cannot be combined as of now, but they must be combined for the steam workshop. That's what's being said here. The way mods are loaded needs to be changed in and of itself.
Aye, exactly.
If Putnam adds ten new cat types, they need to be loaded along the base game files. If I want to give these Putnam cats poison spitting attack later, I don't want to copy-paste his whole file (as he may want to add few new cats later on) I'd rather use 'append' option and simply insert that attack into their creature definition. That way Putnam may work on his cats as he sees fit, and my mod only adds extra feature to his work. Or If I want to make a mod pack but one of Putnam's cat does not fit into the setting, I just set my mod later in loading order and disable that particular cat with append [DISABLED] tag.
Sounds easy enough, but all this has to be implemented for long term mod support.
And yes, before someone asks, there are games that do all that, even more. See Rimworld with harmony dll extension.
-
I'd imagine we'll need something like the Mod Manager mod (https://steamcommunity.com/sharedfiles/filedetails/?id=1507748539) someone made for Rimworld to handle mixing of mods. Lots of Rimworld mods conflict and break each other (or require another mod loaded first) so it's common so have in the description that conflict info: This mod needs to loaded before The X mod to prevent errors.
I know a lot of work will still need to be done on the developers side, but I think the community will need to figure a lot out on our end. I hope it winds up working as well as Rimworld's mods, I've had a lot of trouble in the past trying to mix together mods and I almost always go back to not using them.
-
I'd imagine we'll need something like the Mod Manager mod (https://steamcommunity.com/sharedfiles/filedetails/?id=1507748539) someone made for Rimworld to handle mixing of mods. Lots of Rimworld mods conflict and break each other (or require another mod loaded first) so it's common so have in the description that conflict info: This mod needs to loaded before The X mod to prevent errors.
A mod of that kind wouldn't be possible, due to the limited scope of DF modding. Luckily, that also means there aren't too many ways for DF mods to break each other/the game. Basically the only ways they can do so is by either duplication errors, or by changing stuff in the same .txt file thereby overwriting each other.
I know a lot of work will still need to be done on the developers side, but I think the community will need to figure a lot out on our end. I hope it winds up working as well as Rimworld's mods, I've had a lot of trouble in the past trying to mix together mods and I almost always go back to not using them.
The community has been getting better at using prefixes/suffixes for the object names, the past few years. If you look back at old mods, even big ones, almost all of them have duplication friendly object names. Lots of new players/modders will be a challenge for sure, but not an insurmountable one.
-
I'd imagine we'll need something like the Mod Manager mod (https://steamcommunity.com/sharedfiles/filedetails/?id=1507748539) someone made for Rimworld to handle mixing of mods. Lots of Rimworld mods conflict and break each other (or require another mod loaded first) so it's common so have in the description that conflict info: This mod needs to loaded before The X mod to prevent errors.
A mod of that kind wouldn't be possible, due to the limited scope of DF modding. Luckily, that also means there aren't too many ways for DF mods to break each other/the game. Basically the only ways they can do so is by either duplication errors, or by changing stuff in the same .txt file thereby overwriting each other.
Which is why I am making this suggestion.
I know a lot of work will still need to be done on the developers side, but I think the community will need to figure a lot out on our end. I hope it winds up working as well as Rimworld's mods, I've had a lot of trouble in the past trying to mix together mods and I almost always go back to not using them.
The community has been getting better at using prefixes/suffixes for the object names, the past few years. If you look back at old mods, even big ones, almost all of them have duplication friendly object names. Lots of new players/modders will be a challenge for sure, but not an insurmountable one.
Its perhaps the right moment to realize, everybody, that Dwarf Fortress is no longer a niche project for nerds, full of bad jokes made of bugs and exploits.
DF is on it's way to sail the high seas and gain much, much wider audience.
Robust modding system, just like robust UI system, is something you basically do once, you do it right, and maybe tweak a bit later. It is not dependent on your artefacts and civilisation generation routines, not affected by your AI, by your soundtrack, by the number of plants and animals, by rhythm of your procedurally generated poetry you put in your game.
Once you have these two out of the way, you can put in pretty much everything you may want in you game afterwards.
But without these, and with people already having experience of other games (like mentioned RimWorld), players will be drawn in by DF infamy, hit the wall of hostile UI, bounce back, and leave negative reviews. Or swamp forum with questions why changes done to raws are not reflected in existing worlds and where they can find logs for mysterious crashes.
-
Admittedly, I assumed some ”obligatory” changes in my answer to Pillbo, such as a way for the game to protest if you have two mods creating objects with the same identifier. Like a red icon next to said mods in the in-game mod selector saying ”hey, you have a duplication error here”. But I did not communicate these thoughts, which was stupid, really. Perhaps I thought you could read my mind, perhaps I thought the discussion had already made it clear that mod conflictions was the #1 thing to be dealt with. But I digress.
As a general/separate thought, it would be nice if CREATURE_CLASS and REACTION_CLASS could be expanded to include PLANT_CLASS, ENTITY_CLASS, MATERIAL_CLASS etc.
Arbritary labels are just really useful, and they could be used to target items for mod changes as well. Say you’ve modded in a better mushroom material for your own modded shrooms, and you want all the vanilla fungi (and those modded in by other people) to use it as well. Just target all plants with PLANT_CLASS:FUNGUS and redefine their PLANT_STRUCTURAL to be based off your mushroom material template. Admittedly that would break the colors of the mushroom-trees, so maybe not the brightest example, but I hope the gist, and possible flexibility of this, is still clear.
-
I was basically trying to say having a program that could intelligently splice together text files from various mods (and point out those errors or conflicts) is what people will want, not whatever specifics you're saying is impossible about that Mod Manager. I don't see Toady creating that but leaving it to the modding community to hash it out. Even with it being standard that everything get uniquely named I'm not sure that's helpful for automated mod merging, or we're just likely to get worlds with dozens of barely differently named Drow or whatever when we try to merge of bunch of monster mods.
But what do I know, I'm not a modder.
-
As a general/separate thought, it would be nice if CREATURE_CLASS and REACTION_CLASS could be expanded to include PLANT_CLASS, ENTITY_CLASS, MATERIAL_CLASS etc.
Arbritary labels are just really useful, and they could be used to target items for mod changes as well. Say you’ve modded in a better mushroom material for your own modded shrooms, and you want all the vanilla fungi (and those modded in by other people) to use it as well. Just target all plants with PLANT_CLASS:FUNGUS and redefine their PLANT_STRUCTURAL to be based off your mushroom material template. Admittedly that would break the colors of the mushroom-trees, so maybe not the brightest example, but I hope the gist, and possible flexibility of this, is still clear.
More tags would be great. For example, I would love to see [DYED] tag for clothing items, and related - and be able to set a separate storages for dyed silks for example.
There is class MAMMAL, but there is BIRD and so on and so on. All these labels could be used later for syndromes (change bones to steel for example, why not) or special effects (target birds only) etc etc.
Possibilities are endless.
I was basically trying to say having a program that could intelligently splice together text files from various mods (and point out those errors or conflicts) is what people will want, not whatever specifics you're saying is impossible about that Mod Manager. I don't see Toady creating that but leaving it to the modding community to hash it out. Even with it being standard that everything get uniquely named I'm not sure that's helpful for automated mod merging, or we're just likely to get worlds with dozens of barely differently named Drow or whatever when we try to merge of bunch of monster mods.
But what do I know, I'm not a modder.
I doubt it can be done without any Toady's input. But something like you describe is possible - In Skyrim(?) mods had ids, and any item there was identified as MODID_ITEMID. Or maybe it was another game.
Any splicer could go the same way, if mod has some internal ID, it could just combine items without conflicts, maybe output warnings where there are items named in the same way.
-
I was basically trying to say having a program that could intelligently splice together text files from various mods (and point out those errors or conflicts) is what people will want, not whatever specifics you're saying is impossible about that Mod Manager. I don't see Toady creating that but leaving it to the modding community to hash it out. Even with it being standard that everything get uniquely named I'm not sure that's helpful for automated mod merging, or we're just likely to get worlds with dozens of barely differently named Drow or whatever when we try to merge of bunch of monster mods.
But what do I know, I'm not a modder.
I doubt it can be done without any Toady's input. But something like you describe is possible - In Skyrim(?) mods had ids, and any item there was identified as MODID_ITEMID. Or maybe it was another game.
Any splicer could go the same way, if mod has some internal ID, it could just combine items without conflicts, maybe output warnings where there are items named in the same way.
A bit needy perhaps, but I’d like for my edit to PUTNAMMOD_CAT to work, even if Putnam discontinues their mod and someone else, say Pillbo picks it up. Having the targetable ID change then, from PUTNAMMOD_CAT to PILLBOMOD_CAT, even though it’s the same object, would be pretty annoying. Another example would be if Putnam did not discontinue, but Pillbo decides to compile it and some others into a ”best mods” modpack, requiring only one subscription. The suggested automatic change to e.g. PILLBOMODPACK_CAT would also break compatibility to my edit mod.
-
A bit needy perhaps, but I’d like for my edit to PUTNAMMOD_CAT to work, even if Putnam discontinues their mod and someone else, say Pillbo picks it up. Having the targetable ID change then, from PUTNAMMOD_CAT to PILLBOMOD_CAT, even though it’s the same object, would be pretty annoying. Another example would be if Putnam did not discontinue, but Pillbo decides to compile it and some others into a ”best mods” modpack, requiring only one subscription. The suggested automatic change to e.g. PILLBOMODPACK_CAT would also break compatibility to my edit mod.
An excellent point. So naming conventions seem unavoidable anyway.
On the side note, yesterday I was fighting with bringing Mephs tileset into my updated Masterwork version. Its truly incredible for me that updating graphics can break save games.
Why entity definitions are not decoupled from tiles definition that many years after graphic packs are a thing is beyond me.
Good thing I don't have many hairs left, or I would be pulling them en masse.
There should be a stand-alone file with high priority were you would define tiles and entities making use of these (mayhaps with colour variations), and if entity is not found in raw, it is simply ignored.
Display layer should be as decoupled from simulation layer as possible. Any tile definition in object raw should be treated as fallback default with low priority.
-
Continuing the pure speculation on my end. What if Putnam modded cats in his modpack and I modded cats in my modpack and they didn't have unique ids, they were still just [CREATURE:CAT]? If a program could recognize these objects as the same but not identical you could prioritize it two ways (that I can imagine right now).
1. You could say 'mods are loaded in this order and new overrides old'. So I load PillboMod then PutnamMod and wherever there are conflicting object ids, like [CREATURE:CAT], PutnamMod is used.
2. You can have a GUI say "There are duplicate [CREATURE:CAT] objects in these mods, [click to use PillboMod's] or [click to use PutnamMod's]. Maybe even have a details option to show the diffs
I'd imagine you'd still need to use loading order in some way for mods like Modest Mod that remove objects and other mods that don't. Meaning loading PutnamMod first and it modifies [CREATURE:CAT] (and other things) then loading IHateCatsMod that removes them entirely. Otherwise your IHateCatsMod would delete [CREATURE:CAT], then the next mod would put it right back.
I'm sure there is still more I'm not aware of but this seems completely in the realm of possibility, much simpler than what DFHack currently is capable of.
Edit, or
3. some git style merge so if one mod's CAT breathes fire and the other CAT spits webs your get a web spitting fire breathing cat. It would be harder, I wonder if it's possible for a program to use git's merge ability to create files like that and recognize conflicts?
-
How about loading mods by asciibetical order of there directory (or subdirectory) names? That way numerical prefixes can be added to control loading. Also, a file called “mods.noload” with a list of module (sub)directories not to load in order to allow mods to be disabled without removing them entirely.
-
Lexicographic ordering is sorta useful, you see it "abused" (i.e. that's technically not why, there has to be a mod load order by its nature, but it's still useful) in e.g. Stellaris modding a lot, but I kinda like date modified order a la Elder Scrolls cause that allows for community-curated mod order lists that modders don't have to maintain on their own, which is neat.