Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[PeridexisErrant]

PyLNP with Mod Loader

Loading and combining mods for Dwarf Fortress

The idea is to make adding mods to Dwarf Fortress as easy as graphics packs - just select your desired load order, and the launcher merges as many as possible (giving you feedback on compatibility).  As of v0.5, it's essentially finished and ready for public testing.

The GUI is a fork of the PyLNP, hence the name, and the goal is to merge back in once this is stable enough.  The input format was extensively discussed in this thread, which also saw much early development.  It's simply a raw folder that could be installed by copying it over vanilla - and the launcher can convert an installed into this format, just like it does for graphics packs.

Usage
Run `launch.py`, and use the mods tab.

Mod installation:  place mod under `LNP/Mods/`.  Note that mods must either be in the input format described below, or installed with a working copy of DF - the script can turn the latter into the former. 

Spoiler: Development roadmap (click to show/hide)

#####v0.5  (current)
* Can install multiple non-conflicting mods. 
* Simplifies mods placed in the `LNP/Mods/` folder from a full installation or objects folder to the defined format.  Mods in wrong format should be ignored.
* Comes with a GUI - as part of the PyLNP
* Gives the user live feedback on compatibility as order is modified (and allows reordering selection)
* Writes to `/raw/installed_mods.txt` noting the load order of mods in the mix. 
* Detects special characters which break the merge function and handles by omission.
* Turn mods currently installed into an option (subtracting any others installed by the launcher if a log is present - does full analysis)
* Detects currently installed mods (from log, not analysis)

#####v0.6 - 0.8
Optional enhancements: 
* Pre-process raw files before diffing - flattening or other. 
* More improvements to mod formatter.
* Finalise the standard for `manifest.json` - a dictionary of useful or potentially useful metadata.  (author, mod name, weblink, mod version, DF version, etc)
* Read and use the manifest, and create a default if none is found.
* Handle subfolders of `/raw/` differently to reflect the content (esp. if preprocessing)
* Handle the `data/` folder to allow for graphics packs - or otherwise make mods and graphics play together

#####v0.9
* Organise a standard package of simple known-good mods to include.
* Extensive community testing.

#####v1.0
* Merge into main PyLNP distribution.  Starter Packs include optional mods by default.

Spoiler: Input Format (click to show/hide)

Each mod has folder under `LNP/Mods/`.  Within the top-level folder, which should have a meaningful name (eg `LNP/Mods/Generic_Stone`, `LNP/Mods/Gnome_Fortress`), a mod consists of:

* A `raw` folder, identically structured to the vanilla raws, containing a full copy of every file that has been changed from the vanilla raws. 
  * Absent files are presumed to be identical to vanilla, which allows very small disk size for minor mods.  Files which the mods requires to be deleted should be replaced with an empty file of the same name.  *This means the mod could be installed just by dropping it over vanilla DF.*
* An optional `readme.txt` describing the mod for human readers, and `manifest.json` for machines.

While any set of modded raws can be merged, for best results modders should make the minimum changes required for their mod, editing vanilla files in place instead of moving/renaming/replacing files or content within them.  This reduces the risk of merge conflicts, increases the customisation available to users, and decreases the chance that the merged raws will be invalid.  While the last is unlikely, it cannot be ruled out as the tool does not currently check raws for validity. 

As well as the precise format described here, the launcher will automatically simplify mods in other common formats.  These auxiliary formats are currently limited to a full install of the mod with all DF files - the preferred option - or an objects folder intended to replace the vanilla objects folder (absent files are assumed to be deletions). 

Spoiler: Pic! (click to show/hide)

Interface as of v0.3
Not seen:  green, for mods which are entirely within vanilla files and merged OK. 
Yellow mods add files, and were merged successfully.  All these would be included if you hit 'install'. 
Red is a rejected merge due to overlapping changes. 
Orange was not attempted due to a previous conflict. 

Contributions of all kinds most welcome!

[PeridexisErrant]

Reserved.

[thistleknot]

Just wanted to say congrats master collaborator

[hermes]

Yea, good work on this, PE and contributors!  Looks like a great start and I'm looking forward to seeing this progress.

[Dirst]

Ah yes, this would launch while I'm traveling.  Well played

As time permits, I will try to break it perform some field testing and let you know how it performs.

Nice job on getting this up and running!

[PeridexisErrant]

Version 0.4 includes a number of new features:

* best-effort support for another input format - objects folder which replaces vanilla raw/objects folder
* omits any characters which would break the merge function - might be ugly with accents, but it eliminates a common crash
* better feedback through GUI (unloaded mods white not orange)
* groundwork for overlapping merges later

The documentation and development roadmap have also been updated.

There are a couple of things I'd like to do next.  The manifest needs to be finalised, which means I need to read up on the .json module - and work out what the data is useful for!  Working out how to have mods and graphics play together would be great, even if it's a subset of all graphics that are based on ASCII raws (no ideas here).  And implementing preprocessing would be great - we can simply take each collection of lines and operate on them with a regular expression or something.  Suggestions for detail on any of these would be great!

[Taverius]

I'm getting a ton of errors - does this require python3?

[PeridexisErrant]

Quite possibly - I've got 3.4,and while I've tried to keep it compatible with 2.7 (and the base is) there's some Unicode stuff that could cause problems as well as random potential for bugs.

[Taverius]

Mm... even 3.3 faceplants for me, and I'm not really in a situation where I can upgrade. Guess I'll have to wait for it get folded back into mainline.

Regarding merging with graphics mods, I think your best bet is some kind of extremely light single-purpose DSL that can read something like this:

Code: [Select]

@[CREATURE_TILE:<NAME>]:[TILE:<Blah>]:[COLOR:R:G:B]
And apply it after the main mods are done. Hell, you could probably skip color to begin with.

That's a fairly simplistic parser - find root level with regex, go to next non-root-level line till you find the tile and color and replace, if you dont skip.

Trying to rig it with diff will probably lead to permanent loss of sanity.

[Meph]

Well done and good luck with it.

[milo christiansen]

Rubble comes with a semi-automated way of extracting tile/color information from non-Rubble raws and turning that into a "tileset file", currently the syntax used is suboptimal for several reasons, but I plan to change it to something like this:

Code: [Select]

{TILE;<object name>;<optional key name>;<the tile number or whatever>}
(colors use the same thing, just with COLOR instead of TILE)

The key name is used for cases where multiple tile numbers are needed for the same object.

Any way if we both use the same syntax it would save modders work and Rubble could be used to generate PyLNP tilesets automatically (as opposed to writing a new tool)

(the listed syntax is upcoming, currently the object name and key are a single field, but Rubble 5.2 should use the new way unless I forget...)

[PeridexisErrant]

Well done and good luck with it.

Thanks!  Here's hoping I can push a standard format and create space for a community of the smaller mods...

Mm... even 3.3 faceplants for me, and I'm not really in a situation where I can upgrade. Guess I'll have to wait for it get folded back into mainline.

Drat, I had hoped it wouldn't fail quite so badly. 

Rubble comes with a semi-automated way of extracting tile/color information from non-Rubble raws and turning that into a "tileset file", currently the syntax used is suboptimal for several reasons, but I plan to change it to something like <snip>

Anyway if we both use the same syntax it would save modders work and Rubble could be used to generate PyLNP tilesets automatically (as opposed to writing a new tool)

Sounds promising!  I'm keen to keep all the inputs as standard as possible, which is specifically a set of raws with vanilla files removed, but if the logic exists it can be reapplied. 

[lethosor]

It would be best to ensure compatibility with Python 2.7, since it's the default on most Linux distributions and OS X.

[fricy]

PTW

It would be best to ensure compatibility with Python 2.7, since it's the default on most Linux distributions and OS X.

+1
I spent the last day trying to figure out a bullet-proof python config that can be used to compile a working PyLNP for 10.6-10.9 and failed miserably. Python3 is out of the question, as there's no working pyinstaller for it. Python2.7 with tkinter framework crashes reliably, PIL is deprecated, and pillow is buggy on 10.9. My-little-dependency-hell. 

[lethosor]

Tkinter crashes? What OS version are you using? If you're using 10.6, you'll probably need to update Tk. (This library might be possible to distribute without installing if you point the DYLD_LIBRARY_PATH environment variable to its location, the way the "df" script does.)