Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[expwnent]

I was originally going to make this a PM to Putnam, but it will affect raw modders also, so I'm making it public. Really it's to anyone who scans through raws to get extra information like autoSyndrome does with SYN_CLASSes, but I think that's just me and Putnam.


As DFHack modder interfaces get more numerous and complicated, it will get harder for modders to keep track of the naming conventions that signal data to scripts like itemsyndrome, autoSyndrome, syndromeTrigger, etc. The natural solution is to standardize the way that modders pass this extra information to DFHack. I think we should use XML. XML is inherently extensible, so scripters and dfhackers could easily design their own structure for tags that are relevant to whatever script/plugin they're writing without a new DFHack release. The DFHack parser will expose the parsed files as a tree of XML tags with a list of key/value arguments. All you'd have to do as a scripter is walk through the tree and extract the information you need instead of walking through raw files in your script.

Would you be willing to change the way itemsyndrome works to cooperate with this? So far (I think) it's only you and me that do this sort of thing, so if this is ever going to get done, now is the time. It's too hard to convince large numbers of people to change their stuff even if it's a better way of doing things. If we switch over now then we can keep it nice and standardized. Putting weird raw tags into the game is awkward at best from a design point of view anyway. I would greatly appreciate it if you'd be willing to switch over. If there's a better way of doing things, the sooner you switch over to it, the easier it will be and the less people will have to change their stuff.

I do think the old interface should be removed entirely, but I might be persuaded to leave it for a release or two to help the transition for modders. It would break backward compatibility, but only once and for the better in the long run. DF itself can change without regard to backward compatibility, so it comes with the territory.


TL;DR:

1. I'm going to make the XML parser. Would you be willing to use it the new format once I make a nice interface?

2. I could be persuaded to use a different format, but I really think XML is the way to go.

3. I could be persuaded to maintain backward compatibility, but this really is a much better way of doing things in the long run. It seems like a bad design to have two completely different ways of doing the same thing, especially if they might conflict.

[Putnam]

I think an easier-to-read raw such as JSON would be advisable, but yeah, the SYN_CLASS stuff is starting to get a bit ridiculous. I try to keep it similar to autosyndrome (using \UNIT_ID and such for units), but in the end it's using a system that isn't really designed for that kind of thing when other systems are completely viable and designed to be used in that way.

Even more important is that using JSON or XML allows for things that aren't syndromes, creatures or materials to have special things attached to them with no fuss. Right now, those three are the only objects in the raws that allow an arbitrary amount of arbitrarily named class tokens.

On the other hand, I'm kinda obsessed with raw modding and have an attachment to it, but that's just me and it doesn't have much affect on other things.

[expwnent]

With my implementation, XML is actually LL(2) instead of LL(1) and that makes the parser a pain to write. I'll look at JSON and see how nice it is. Truth be told, I kind of hate xml, I just didn't know any good alternatives.

[Warmist]

i personaly don't like xml. Btw why do we need any xml or json specs? we could just parse the raws-like file i.e. [X:Y:Z:W] with some way to escape ":" and plugins/scripts could access stuff by X.Y.Z.W (or something like that). No need for json or xml parsers.

[expwnent]

https://gist.github.com/expwnent/134d16f4e5dc4389d9cf

See examples. Lots of cool stuff we can do with JSON. Unfortunately gist doesn't seem to let you do line wrapping, so it's kind of a pain to look at but it's there. Copy+paste it into a text file for convenience.

[Warmist]

Hmm wouldn't that look a lot simpler and readable in lua?

[Merkator]

@expwnent I think your proposal look quite good, but why you don't use just standard JSON format.
Here is my few cents.
1: https://gist.github.com/Demagogue/a39441de7f652842a725

I don't know if any good JSON parser for Lua exist but this shouldn't be that terrible to write. And you have advantage of being editable on any web editor. Another pros is that you don't need to write special cases for comments.

[Putnam]

It will probably written in C++, which has several JSON parsers available already.

[expwnent]

Basically the only thing I didn't like about standard JSON was the lack of recursive comments.

I might end up doing something different anyway. The new main idea is to process the input format into dfhack commands which would be way too long and elaborate to type in manually and modify plugins/scripts so that you have to specifically register things on the command line or in dfhack.init or with this dfhack preprocessor file in order to register autoSyndrome/itemsyndrome/etc stuff. This file would just save people a lot of typing.

Or I could go with the original idea, but this seems simpler and I'm not sure the full power of the parse tree would be necessary for any plugins in the foreseeable future.

[expwnent]

The new plan is that all information that plugins/scripts use will be given by command-line arguments in the dfhack.init file. This may include a lot of repetitive typing in the dfhack.init file, so I'm working on a macro language processor which will process a special input file into a dfhack.init file and then execute it. The format of the file will be determined by a simple text file, so changes in the format will not require a new release of dfhack. The end result is that you'll have one small file in which you declare all of the dfhack-modder interface stuff in a nice, easy format, and that format will be easily extensible to new plugins and scripts. In the short term, you'll have to replace all the awkward SYN_TAG stuff and prefixes on df identifiers with lines in the dfhack.init file. I regret the break in backwards compatibility, but in the long run, this way will be much better.

Example:

Code: [Select]

outsideOnly outside BUILDING_1 BUILDING_2 inside BUILDING_3
instead of putting silly prefixes on the identifiers.