Bay 12 Games Forum

So... I think I may have a solution with how to apply older mods to newer versions of the game.

http://cyberelk.net/tim/patchutils/man/rn01re02.html

http://en.wikipedia.org/wiki/Quilt_%28software%29

Basicaly, merge patch files.

Vs trying to apply a 34_11 mod to 40_08 as a patch of changes that we based on 34_11 vs 40_08...

One could take a diff patch of 34_11 to 40_08,
  call this 34-11To40-08.patch
And a 34_11 to Accelerated,
  call this 34_11ToAccelerated.patch

Then, using the tool combinediff (I guess any conflicts would have to be addressed on the patch file level)

something like combinediff 34-11To40-08.patch 34_11ToAccelerated.patch > 34_11To40_08_Accelerated.patch

and apply this patch to a 34_11 version of the game to achieve a v40_08 mod of the game, then do a diff comparison from 40_08 to this mod of v40_08

In light of what that script is doing, I recommend this one now

$files = @(get-childitem -include *.txt -recurse -path $path -filter $filter)
Write-Host "files loaded";
foreach ($file in $files) {
        $out1Pass = "$file" + ".1pass"
$outFile = "$file" + "2"

        Get-Content $file | Foreach-object {
            $_ -replace "`t","" `
-replace '\]\[',"]`r`n["
        } | Set-Content $outFile
    }

I was reading on what it does, and it rewrites the replacement operations.  I'm gonna push a better script in the next day or two if I don't learn a better way to write a script.

I figured out what causes the merge conflicts...

It's when I derive a patch from:

34_11 to accelerated

then...

I try to apply it to

40_08

the conflicts arise because my patch file is confused when it expects a 34_11 line and see's a 40_08 line.  Only way to fix that is to incorporate into the patch the 34_11 to 40_08 changes as well as the accelerated patch

so...

to alleviate this...  it's almost like we need to apply a diff patch of 34_11 to 40_08 + the accelerated patch in the same patch file somehow.

good to see your plugins in 40_08

not sure if I'm in an oldie moldie thread or not...


but


I noticed I accidentally hit like ctrl-s or something and brought up a "custom reactions" menu... however... I can't seem to bring it up again.

Can you please confirm or deny this great feature's existence or not and how I may access it again?

okay, final udpate.

I worked really hard to ensure when I split token's that it didn't create [empty] newlines.

basically, tabs are removed first, then a carriage return "`r" and new line "`n" is created inbetween ]['s

works in powershell

$files = @(get-childitem -include *.txt -recurse -path $path -filter $filter)
Write-Host "files loaded";
foreach ($file in $files) {
        $out1Pass = "$file" + ".1pass"
$outFile = "$file" + "2"

        Get-Content $file | Foreach-object {
            $_ -replace "`t","" `
-replace '\]\[',"]`r`n["
        } | Set-Content $outFile
    }




output is .txt2

sample before and after output




Update
I did some tests with trying a 3 way merge.  The merging process is a lot smoother with the flattened raws, but I could still see a need for still checking if the token's in conflicts are being duplicated.  That would resolve most conflicts I think.

Update 2
I took a walk and thought about why I was having [merge] conflicts.  I was wondering if, using this parse method, it was due to not setting all tokens on their own line by themself when they were next to a commented out line... 

however, the merge conflicts I had were not due to token's being next to a comment (I think I might have to update the parse method to consider if a token is adjacent to a comment).

I think it might be because I applied two mods (patch of 34_11 to accelerated, and a patch of accelerated to modest...as one patch over a 3rd mod (40_08)), which I think may have cause some [merge] conflicts with [duplicate] grazer tags.

In other words, I'm thinking if raws are parsed using this method, and applied one at a time to a 3rd mod.

For ex.

diff of 34_11 vs 34_11 accelerated: as a patch
  applied to 40_08
diff 34_11 accelerated vs 34_11 accelerated modest: as a patch
  applied to 40_08 accelerated

one might avoid conflicts?  I certainly believe raw parsing the files in this manner before merging reduces the # of contextual conflicts.

Here's what a diff between 34_11 and 34_11 accelerate modest looks like with this parse.

http://pastebin.com/curfNcMH

Update 3:
I think if one back-update's some in house object tracking variables (using these patch files); such as file_name, and the first [object: name] token, using a scripting language like python, one should be able to fill in the blanks when it comes to patching location (even without col, row information)

Then one could parse the diff files and derive the object id, and be able to ensure the diff file is being applied to the correct object/token id (such as: creature or item or entity, or whatever) by keeping track of duplicate[ token]s!

That would resolve the grazer issue.  If castes come into play...  I guess a simple check if the current caste is having a token duplicated, then figure out what token is the master token to be used.

I was thinking about some more complex version of extended diff format.
For example:

Code: [Select]

raw>object>entity_default.txt
?[ENTITY:MOUNTAIN]
+[NOPAIN]
-[PERMITTED_REACTION:SMELT_IRON] # just sample.
@[ENTITY:DWARVES]

Where

Code: [Select]

     ? find line
     + mean insert line
     -  remove line
     @ replace line
     # comment

This can be easily connected with each other.
For example if you want replace tag ITEM_WEAPON:Sword with ITEM_WEAPON:Big Sword
You need to just write it like that

Code: [Select]

raw>object>item_weapon.txt
?[ITEM_WEAPON:Sword]
@[ITEM_WEAPON:Big Sword]

It should be much easier that regexp.

while I agree with you logging the token id with the patch file.  Peredexis was concerned with the drama involved with modifying the patch format.

I believe by flattening the raw file, it allows for contextual block matching to take over.  So, in theory, creatures should line up and token differences should be noticed more.  It might not really be the ideal solution and may require additional parsing (such as token comparisons between the same object between two mods).

I was reading through your program.

It sounds cool.  I know how I can make some mods of mine in rubble.

Then I was thinking how i'm trying to auto-merge mods and how you deal with basically cherry picked "patch files" inbetween mods more or less; however, a modder has to cherry pick, and (I'm assuming cuz I haven't seen an example yet, but) when he does, he has to expand on just the patch differences?

So I was thinking (I haven't read all the documentation... yet).

Is you seem to be very aware of the entire raw structure, it seems like if you allow a player to modify aspects of reactions (kind of like a diff change) using your rubble system, that you yourself on the backend must wipe out and recreate the object id they are editing.  I haven't gotten up to that point in the documentation.  It would seem like a pain to make a modder have to rewrite an entire object by hand if he only made a single change to an object.  Like I said though, I'm not 100% on how you have your's setup.

While your rubble system seems great.  It seems like A HUGE LEARNING CURVE to apply mods, and it asks a lot of a mod creator.  Although... I can see if someone wanted to LEARN HOW TO MOD using rubble.  That's one thing.  But to be a modder and want to convert a mod to using rubble...  idk.  There's a lot to learn I would say.  Some of the things seem like modding shortcuts, but some of the other things seem like a lot of 3rd party language learning.  The basic templates seem easy enough.  However, when I mod (generally limited to merging), usually notepad++ and tortoisegitmerge is all I need and at most I need to google a regular expression every now and then.  I'm not 100% sure if porting diff patches to rubble format would be more or less work than trying to manually merge conflicts.  Have you considered parsing diff files?  Or have you had similar issues with having to review conflicts?  With what Peredexis just pointed out, a way to flatten diff files is to replace [ and ] with newlines followed by their respective [ and ].  I put a powershell script up that does it.  I think it has the potential for knocking out diff merge conflicts.

I have to admit though, your tool seems pretty all encompassing from this vantage point.

that's a pretty great idea.

Broken token's wouldn't be too obvious though.

I would think a newline should be around every [ and ].

That way broken tokens would stand out?  Probably not worried about broken tokens?  Well idk if it would have an affect.

I say broken because I've seen mods that comment out a bracket of a token to null it out from being loaded.

Unless you autoprune all broken tokens/comments beforehand...

Update:

That is a great idea.  However, that's not the route I'm taking atm.  I was honestly thinking of a way to do what you just described using a batch file however.  I'm wondering if a batch file can use a regular expression to perform such a feat.

look at this from some rawexplorer link I was viewing

http://dftokens.gumpstudio.com:8080/ViewToken.aspx?Token=PHILOSOPHER