Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[Clément]

FUSE is a library for making filesystem in userspace on Linux. For some time, I have been using unionfs-fuse for keeping separate game, mods and user files for games that do not do it natively (like Dwarf Fortress). But it was not perfect as I still needed to merge configuration files (init.txt, d_init.txt) manually on updates as unionfs-fuse replace whole files. So I decided to try and make a FUSE filesystem with specific behaviour for some files.

In a same way as unionfs-fuse, dfmerge-fuse uses several read-only branched and a writable one to create one mount point presenting the files from the branches and writing modification to the writable one.

The currently supported merging method are:

• Merging some specific DF file formats (currently: init.txt, d_init.txt, interface.txt).
  • (d_)init.txt tokens are identified by the first value and the other are replaced when a similar token is found in another branch.
  • Interface.txt works in a similar way for BIND tokens but other tokens (KEY or SYM) are added to the corresponding BIND section. It also support a DELETE token for removing key bindings.
  • overrides.txt merge twbt overrides, rename tilesets with already used names and overwrites overrides matching the same tiles.
• Direct use of the writable branch (for saves).
• Link save raws to the general raws
• A quick and dirty unionfs-like method for the other files

You can get the source code on my github: https://github.com/Cwningen/dfmerge-fuse
It is still WIP and not tested enough, so I would not recommend to use it on your regular game without making backups.

Notes:
Currently, files with the specific merging (init.txt, d_init.txt, interface.txt) are overwritten in the writable branch every time the file is opened in the merged filesystem and any comment is discarded.

What I am planning (or dreaming) to do:

• Support more file format merging
  • raws but that looks difficult from what I know).
  • maybe something like a simple concatenate scheme for overrides.txt (twbt).
• More configuration for the merging methods (currently hard coded)
• Updating raws for the savegames: either linking from the general raws or using specific branches for each savegames
• Other suggestions?

[Trouserman]

Very cool.

Notes:
Currently, files with the specific merging (init.txt, d_init.txt, interface.txt) are overwritten in the writable branch every time the file is opened in the merged filesystem and any comment is discarded.

Do you check the files' modification dates?

• Other suggestions?

You could possibly support a three-way merge on raw files. My raw file merges almost always proceed without any hiccups. (Though that "almost" gives me pause. I'm not sure what would be appropriate to do in the event of a conflict, for a FUSE plugin.)

[Clément]

Do you check the files' modification dates?

For checking if the file was modified ? It is easier than that, my program is implementing the file system so I see every write operation on the file. I can easily now if the file was modified and only write then (edit: it is done). That won't change the fact that comments are discarded when the file is written back in the writable branch. Anyway, I think it is best to keep your settings in read only branch, and let the writable branch for in-game modification.

You could possibly support a three-way merge on raw files. My raw file merges almost always proceed without any hiccups. (Though that "almost" gives me pause. I'm not sure what would be appropriate to do in the event of a conflict, for a FUSE plugin.)

By three-way merge, I assume you are speaking of merging an old version, a new version and modified old version into a modified new version. That is not what I meant to do here. I apply mods on a base version (maybe already modified by other mods), it is always a two-way merge. Files in a mod need not to be complete but only a patch made for the current version. The mod can be generated from a three-way diff but that outside the scope of the filesystem.

It was maybe not very clear in my first post but merged files (init.txt, d_init.txt, ...) must be minimal patches if you want to see the modification that came from updating the other branches. For example, if you make a "mod" whose only purpose is displaying the FPS, init.txt should only contains:

Code: [Select]

[FPS:YES] so that if another mod that change the tileset, is loaded before the FPS mod, it is not overwritten by the FPS mod.

I think I will make some tools based on the algorithms used in the filesystem to generate such diff files.

Edit: The filesystem no only write the file when it was modified, comments are still discarded when writing. I also added a dfdiff tool to make diff file for mods. Example

Code: [Select]

$ dfdiff init ~/games/df-test/{base,mayday}/data/init/init.txt
[FONT:mayday-no-highlight.png]
[FPS:YES]
[FULLFONT:mayday-no-highlight.png]
[GRAPHICS:YES]
[GRAPHICS_FONT:mayday.png]
[GRAPHICS_FULLFONT:mayday.png]
[G_FPS_CAP:25]
[INTRO:NO]
[MACRO_MS:0]
[PRINT_MODE:TWBT]
[SOUND:NO]
[TRUETYPE:NO]
[VSYNC:YES]
[ZOOM_SPEED:5]

[Clément]

I added the support for TWBT overrides.txt and raw directory for every save is now a link to the general raw. So now, updating the general raw, updates all the saves. The downside is that all saves have the same raws, but they can easily be changed by remounting the game with different branches.

[rmblr]

Amazing, gonna give this a try.

[Clément]

Along with some bug fixes and minor improvements. I have changed the way directories are passed to the program. Now, directories are searched according to environment variable or command line option:

• DF base files are searched in $XDG_DATA_DIRS/dfmerge-fuse/df, that means, by default, $XDG_DATA_HOME (~/.local/share), /usr/local/share and /usr/share, in this order
• Mods directory are searched in $XDG_DATA_DIRS/dfmerge-fuse/mods
• User files are stored in $XDG_CONFIG_HOME/dfmerge-fuse/userfiles (default ~/.config)

So now only the mount point and the mods name (directory names in mods directory and not full path) are passed to dfmerge-fuse. Directories can be modified with command line options (see README.md).

Here is an example of how to use it:

Code: [Select]

# Create the mount point
mkdir -p $XDG_RUNTIME_DIR/dfmerge-fuse/df
# Mount the filesystem with some mods
dfmerge-fuse $XDG_RUNTIME_DIR/dfmerge-fuse/df dfhack twbt mayday &
# Go to the mounted df directory
cd $XDG_RUNTIME/dfmerge-fuse/df
# Run the game
./dfhack
# Get out of the directory before unmounting
cd ..
# Unmount the filesystem
fusermount -u df