Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[DragonDePlatino]

Speaking of which...I think I've changed my mind about getting an executable. I'm already able to do everything I want with the current version (i.e. override soil, layer, mineral and gem raws) so I'll be interested in getting an executable for the next version of GemSet.

Do you have a target date for that?

I'm going to be finishing up all of my tile-related stuff in a day or two, but after that I need to touch up some old stuff and wait for some TWBT bugfixes from mifki. Take your time.

Also, I can't thank you enough for releasing this wonderful program. There's no way I could have manually gone through all of those gem raws without loosing my mind. 8I

[milo christiansen]

So... Ignore everything I said about looking at Rubble except for the bit about the tileset format. I took a look at BAMM once I got home, and discovered it is much more advanced than what I was using for Rubble. Of course I proceeded to rewrite Rubble's system so it does what BAMM does (more-or-less)

I do have a suggestion, I'll just let this proposal (proposals actually) speak for itself:

Code: [Select]

# Extended BAMM Rule Format Proposal

# This is an example of a proposed extension to the BAMM rule format. I will almost certainly be
# using this extended format (or something like it) for Rubble's tileset applier and the rule based
# raw modification system Rubble will be gaining soon.

# The only drawback I can see of this over the original format is it will need a more complicated parser.
# This format is not any harder to parse, but it does require more advanced techniques to split it
# into tokens. Anyone who has ever written a proper lexer should find this an easy task, and for those
# who have not, well nows your chance to learn!

# The primary benefit of this format over the original rule file format is that it lends itself well
# to defining large tag sets with minimum repeating and maximum reuse (and so less chance of errors).

# Rubble 6.9 comes with a very simple utf8 one-character-at-a-time lexer for this format that can be
# found in "other/src/rubble/rblutil/merge/lexer.go". This lexer in untested, and the parser half of
# the pair is not even started. In my lexer newlines have no special significance except that they
# terminate tokens not otherwise terminated, they do not show in the token stream. Rule boundaries
# are easy to detect by looking for a spot where a token that can end a rule is immediately followed
# by a token that can begin a rule. Since tokens inside a rule always alternate between item tokens
# and separator tokens this is an easy case to detect, just look for two non-separator tokens in a
# row.

# One proposed change is in range statements (the "?(x,y)" syntax) instead of having an empty second
# argument for "no limit" use a negative number ("?(x,-1)"). Obviously the empty version would be
# valid too, it just makes sense. A related extension would be to make a single argument range
# statement be equivalent to a statement where min and max are equal. Neither extension is particularly
# important, but why not? It doesn't really complicate anything, particularly if you have a proper
# parser. Anyway, this example file will use these extensions.

# Escaping special characters (if required) is accomplished via the standard backslash syntax in my
# lexer. Currently you can escape any character, including comment markers. Escapes are only processed
# in strings, but since they make no sense anywhere else this is not a limitation. A side effect of
# this is that any backslashes you want to add need to be doubled up.

# My lexer handles comments by treating them like whitespace. Comments go from the marker to the first
# linefeed ('\n'), carriage returns ('\r') receive no special handling.

# Define a named rule block. This allows you to insert one or more rules by name at a later time,
# which can greatly reduce repetition (and hence chance of errors).
!(MATERIAL_TAGS){
DISPLAY_COLOR:?(3)
TILE:?
ITEM_SYMBOL:?
BUILD_COLOR:?(3)
BASIC_COLOR:?(2)
}

# Here is an example of a rule block, basically all rules in the block are treated as if the rule
# at the beginning of the block is prefixing them. Rules in the block may themselves have attached blocks.
CREATURE:$|{
CREATURE_TILE:?
CREATURE_SOLDIER_TILE:?
ALTTILE:?
SOLDIER_ALTTILE:?
COLOR:?(3)
GLOWTILE:?
GLOWCOLOR:?(3)
REMAINS_COLOR:?(3)
SENSE_CREATURE_CLASS:$:?(4)

# Here we use a named rule block, as you can see this works almost exactly like a normal block.
MATERIAL:$|%(MATERIAL_TAGS)
USE_MATERIAL:$|%(MATERIAL_TAGS)
USE_MATERIAL_TEMPLATE:$:&|%(MATERIAL_TAGS)

CASTE:$|{
CASTE_TILE:?
CASTE_SOLDIER_TILE:?
CASTE_ALTTILE:?
CASTE_SOLDIER_ALTTILE:?
CASTE_COLOR:?(3)
CASTE_GLOWTILE:?
CASTE_GLOWCOLOR:?(3)
REMAINS_COLOR:?(3)
SENSE_CREATURE_CLASS:$:?(4)

MATERIAL:$|%(MATERIAL_TAGS)
USE_MATERIAL:$|%(MATERIAL_TAGS)
USE_MATERIAL_TEMPLATE:$:&|%(MATERIAL_TAGS)
}
}

# A more exotic example:
{
CREATURE:$
CREATURE:$|CASTE:$
}|{
MATERIAL:$
USE_MATERIAL:$
USE_MATERIAL_TEMPLATE:$
}|%(MATERIAL_TAGS)
# Maybe a little hard to read (and hence not that great of an idea), but a fine demo of the power
# offered by this extension.

# Of course the normal one-line, one-rule form still works, and in some cases is easier to use.
# Backwards compatibility is preserved, not that either BAMM or Rubble has a large enough user base
# to make backwards compatibility much of a problem...

# Another extension I would like to add is kill and add rules, basically if the source has a tag
# that matches a add rule and the dest does not, the tag is added to the dest, and vice versa.
EXAMPLE:$|-(KILL_RULE:$)
EXAMPLE:$|+(ADD_RULE:$)
# Obviously kill and add rules are of little interest for tileset merging, but would be invaluable for
# general rule-based raw editing (which would be a useful addition to Rubble).

# You may have noticed a syntax trend where special actions are in the form:
# <one letter action code>(<argument 1>, <argument 2>, ..., <argument n>)
# I decided that this would be a nice general extension of the existing range syntax, but if you
# (or anyone else) have a better idea I want to hear it.

Tell me what you think.

BAMM looks like a nice tool, and I would like Rubble's functionality to be as BAMM compatible as possible, but I will be implementing some kind of extended rule format, whether BAMM does or not.

[Button]

...I just hit F5 with the wrong window selected and lost a long post.

TL;DR VERSION!

1. I think that first paragraph is the greatest compliment I have ever received. Thank you.

2. I mostly like your proposed format. I'm not sure about the sanitization, since IMMUTABLE tokens don't include special characters, and only IMMUTABLE tokens should be literals in raw format definitions. Unless you're talking about including this kind of code in actual raws as part of your injection functionality, in which case that kind of handling is well outside of BAMM's scope for 2015.

3. I'm also not sure about the KILL_RULE and ADD_RULE functionality. That seems like a problematically oversimplified approach for multi-mod merging. I'm still formulating the exact approach I'm going to take for mod merging, but I want it to be subtler than this. I don't want to use a sword where a scalpel is called for, and hardcoding what tags to add or drop into the tag templates themselves? That's a claymore.

4. What value do you hope standardizing the formats between the two tools will add? The only real benefit I can see is that when new versions come out both apps can be brought up to date by one person. Which isn't nothing, but given that the two tools have different goals, I'm not sure that it's enough. Obviously I really appreciate the input, and I'd like to continue bouncing format ideas off each other even if standardization doesn't happen because your ideas are pretty rad; but unless there's some greater value to it that I'm overlooking, I don't think standardization for standardization's sake is enough of an incentive to justify the additional work of handling features that only the other of us needs.

[milo christiansen]

2) I am not sure what you are talking about here.

3) I am thinking that it would be great for making special purpose rules to do a single task to a whole raw set. In Rubble this would be used as a sort of patch system to allow modders to make changes with only a few rules, a simple way to may small changes to a specific object. Not useful for tilesets, but possibly useful for other things. This part of the suggestion is very hazy for now, and will need to be refined by actual usage.

4) To put it simply, shared rules and the ability to make a "patch" with one tool and apply it with another. For example Rubble 6.9 can make a version of the raws that contains only the tileset related tags, this "tileset file" could be applied to any raw set with BAMM with only two small problems 1) BAMM's default ruleset may not work, but since the current version of the Rubble default rules are BAMM compatible it basically means switching the rules out. 2) Rubble uses a custom system for editing the init files, adding [AUX_FILE:INIT.TXT] or [AUX_FILE:D_INIT.TXT] to the top of the init files would solve that.

EDIT: Anyway, I have been at the library for ~7 hours now, so time to go home. I will probably not be able to respond for a week or two

[Button]

2) I am not sure what you are talking about here.

This:

Code: [Select]

# Escaping special characters (if required) is accomplished via the standard backslash syntax in my
# lexer. Currently you can escape any character, including comment markers. Escapes are only processed
# in strings, but since they make no sense anywhere else this is not a limitation. A side effect of
# this is that any backslashes you want to add need to be doubled up.

3) I am thinking that it would be great for making special purpose rules to do a single task to a whole raw set. In Rubble this would be used as a sort of patch system to allow modders to make changes with only a few rules, a simple way to may small changes to a specific object. Not useful for tilesets, but possibly useful for other things. This part of the suggestion is very hazy for now, and will need to be refined by actual usage.

Yeah, I think this is one of those things which is in scope for you and out of scope for me. In BAMM the syntax for tilesets & eventually full mods isn't intended to be touched by anyone but myself. It just builds the scaffolding to read raws onto.

4) To put it simply, shared rules and the ability to make a "patch" with one tool and apply it with another. For example Rubble 6.9 can make a version of the raws that contains only the tileset related tags, this "tileset file" could be applied to any raw set with BAMM with only two small problems 1) BAMM's default ruleset may not work, but since the current version of the Rubble default rules are BAMM compatible it basically means switching the rules out. 2) Rubble uses a custom system for editing the init files, adding [AUX_FILE:INIT.TXT] or [AUX_FILE:D_INIT.TXT] to the top of the init files would solve that.

If it's legal DF raw, BAMM will function on it, with a few exceptions which I'm working on eliminating.

EDIT: Anyway, I have been at the library for ~7 hours now, so time to go home. I will probably not be able to respond for a week or two

I hope you have a good internetvacation! Or whatever!

[Button]

FYI, I've finally updated the first post to include a summary and brief tutorial.

[milo christiansen]

Well I got to a library sooner than I had thought, but I won't be able to stay long (not having internet at home sucks).

I see what you mean about certain features being useful for Rubble but not BAMM, that is not a problem for me, I'll just make an effort to make sure the common parts remain compatible. That way common uses (applying tilesets) can share data/rules/etc. Feel free to do whatever you want with BAMM, maintaining compatibility is my problem

If I make any substantial improvements to something related to BAMM in Rubble I'll post them here, just in case you want to steal them

[SalmonGod]

Trying this out with Gemset/Modest Mod, after mashing in Stahl's Armory and Fortress Defense.

Looking at the log file, it seems to run through its process just fine.  At the end, it says "Writing modified raws..."  But then my output folder is empty.  Nothing seems to be generated.  Any ideas?

Also, I have a bunch of lines that say "Found more than one matching child".  What does that mean?  (Edit:  I tried removing the files that produces this error, which was all the plant raws except grasses, and there was no difference in output)

[Button]

Trying this out with Gemset/Modest Mod, after mashing in Stahl's Armory and Fortress Defense.

Looking at the log file, it seems to run through its process just fine.  At the end, it says "Writing modified raws..."  But then my output folder is empty.  Nothing seems to be generated.  Any ideas?

If it was exiting properly, it would end with "All files written," not "Writing modified raws...". It's getting bogged down somewhere in the file system calls. What operating system are you using? And what are the values of your target, source, and output properties in run.config?

I need to log crashes better. I'll take care of that later this week.

Also, I have a bunch of lines that say "Found more than one matching child".  What does that mean?  (Edit:  I tried removing the files that produces this error, which was all the plant raws except grasses, and there was no difference in output)

That's just a debug line, there's nothing actually wrong with it. If you only want to see proper errors, I recommend you set verbose to False.

[SalmonGod]

What operating system are you using? And what are the values of your target, source, and output properties in run.config?

Win 7 x64

source=C:/Dwarf Fortress/BAMM!
target=C:/Dwarf Fortress/objects
output=C:/Dwarf Fortress/raws

[Button]

Win 7 x64

source=C:/Dwarf Fortress/BAMM!
target=C:/Dwarf Fortress/objects
output=C:/Dwarf Fortress/raws

OK, I think I see what's going on here, maybe. What's in the /BAMM! directory?

source is supposed to point at the source of the graphics, not source code. So in your case, source should be pointing at a set of raws with GemSet installed.

[SalmonGod]

What's in the /BAMM! directory?

It's the BAMM! folder included with the Gemset package.  Has a bunch of raws that look like they're written purely for the sake of running through your script.

[Button]

Oh. Hm.

Yeah I've got nothing. I'll add better crash logging later this week and that'll narrow it down somewhat, hopefully.

Thanks for giving this a try, I really appreciate it!

[SalmonGod]

No problem!  Thanks for working on such a useful tool!

[Button]

FYI, there's roughly an even possibility that this will or won't work after the release. If it doesn't, I'll make it a priority.

Adding new functionality is still on a back burner. (Lots going on at work that's got my front burners occupied for a while.)

Oh, and I added in better crash logging a couple months ago. I must have forgotten to post about it. Sorry.