Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  
Pages: [1] 2

Author Topic: New utility : Editor for the raw files...  (Read 11226 times)

ragnarok667

  • Bay Watcher
    • View Profile
New utility : Editor for the raw files...
« on: August 14, 2009, 07:07:34 pm »

EDIT: I shall keep a log of what has been requested and how it's going at the top of this post. This should help me keep focus and make it easier to update you guys.

Current bugs:

Current requests:
  • Open all the raw files at once instead of one at a time : required for body & attack requests.[Ragnarok667]
  • Parsing/validation of body tokens [Captain Mayday]
  • GUI creation of attack tokens [Deon]
  • Add color-coding to tokens [Deon]
  • Remove obsolete tokens in tags.ini [Deon]


Hi !

I'm rather new to DF (about 2 weeks) but thanks to the recession and the lack
of available jobs, I've been spending a lot of time playing around with this game
(both adventure and fortress) and found it quite fun.

Now, I was considering modding the game, but found a lack of actual custom
editors for the raw files (as mentioned in older threads, like : http://www.bay12games.com/forum/index.php?topic=19206.0).

So, I instead of playing some DF yesterday afternoon I decided to try and code
a little utility to handle the raw files. *This ended up taking more like 2 days than an afternoon...  ;)

Now, it doesn't have many features as I'm completely clueless about the modding
process for DF, but I've modded other games before so it can't be that different.

So, my main point is that I'd like you guys to try and see what you think,
how it can be made better. You know the drill, find bugs, interface changes
needed, support of new features and such. I'm thinking if this seems interesting
to you guys I may very well create a thread for it, but before I put any more
work on this I'd like to have some of you experienced DF modders take a look
and give me some feedback.

The software is developped in VB.NET 2k5 (.NET framework 2.0 dependancy)
and should work on most windows XP installations provided the .NET framework
is installed and on (hopefully) any Vista/7 installations.

Here is the terms I came up with for the different parts of the raw files and
how I use them. Please correct me if other more widely used terms exist.
  • tag: [identifier:data]  the data part is not required in all cases
  • object type : refers to the data part (***) of the tag [OBJECT:***]
  • entry : a block of tags starting with a specific tag (as described in groups.ini,
    based on what I've seen in the raw files)

Current features are :
  • Can open all the raw files of v0.28.181.40d without crashing (yet)
  • Can save/save as any of those opened file with (as far as I can see)
    no loss of data, order or structure (it will add tabs to the entries missing
    it though, as to make it more easily readable)
  • When you open a file, it will recognize it's type from [OBJECT:***]
    using a data file called groups.ini and group tags into entry blocks accordingly.
    The tags [identifier:data] that are selected as start of entry are all modifiable
    in groups.ini for future compatibility.
  • The list of all entries appears on the left of the window and can be filtered
    by tag identifier or data by partial or exact match
  • Any entry can be duplicated (inserted after the original one) or deleted
    (except the [OBJECT:***], obviously)
  • You can insert a new template entry before or after any other entry
    (except [OBJECT:***], nothing can go before that one (*: I will require help
    here from experienced modders for the templates... I've put up the stuff that
    seemed most common/required from what I could get of the raw files. )
  • You can also duplicate and delete tags for individual entries
  • Any tag can be added manually (insert then type the info) or through a
    list of possible tags.
  • All the possible tags (tokens ?) should be listed in tags.ini, which also
    supports a brief description to help starting modders or as a quick reference
    to experienced ones *
*: I've only added the tokens for creatures as this was time consuming :)
But I did need some to test the program with and it seemed the most useful one.
I'm planning on keeping it as updatable as possible without changing the code base, so it's very dependant on it's config files (configs.ini, groups.ini and tags.ini)

I've attached a download (scanned with Avast, clear of viruses as far as I can
tell) from DF File Depot of the lastest build and also linked to a basic screenshot
of the software.

Well, that's it for now... I hope this can be useful to people out there !


Download : http://dffd.wimbli.com/file.php?id=1339

Screenshot (click on links for bigger versions):


http://i1005.photobucket.com/albums/af172/Ragnarok700/DFRAWEDITOR-Screenshot0001.jpg


http://i1005.photobucket.com/albums/af172/Ragnarok700/DFRAWEDITOR-Screenshot0002.jpg
« Last Edit: August 16, 2009, 12:14:48 am by ragnarok667 »
Logged

Daywalkah

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #1 on: August 14, 2009, 10:29:00 pm »

Sorry if I'm being an idiot, it's actually unavoidable for me, and I only skimmed this post anyway, but what advantage does it give you? Why is it helpful? Recently I had a pack of Rhesus Macaques steal my nice, shiny things from a recent siege,(funny thing is, is that it was the first summer of the fort) and i gave them the aquatic tag to get my things back.I just minimized the game, went to the raws, and changed a few things. How does this utility make it easier? Explain a bit more. You have rights to call me an idiot if you already did explain.

Scratch that, I looked at the screenshot and thought, "this utility actually might
will make life a lot easier". It is a somewhat brilliant idea. For only two weeks of playing DF, you did a pretty good job.
« Last Edit: August 14, 2009, 10:34:18 pm by dragon1423 »
Logged
Magma cancels flow; interrupted by Ironblood.

Duke 2.0

  • Bay Watcher
  • [CONQUISTADOR:BIRD]
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #2 on: August 14, 2009, 10:32:07 pm »

 It basically makes the larger RAW files much easier to modify, and possibly modify in bulk. Instead of scrolling down in a text editor and looking for specific creature name tags, one can just scroll down a list, and a few other features I don't entirely understand.

 Still, amazing how you made this after only a few weeks of playing. This is indeed very cool.
Logged
Buck up friendo, we're all on the level here.
I would bet money Andrew has edited things retroactively, except I can't prove anything because it was edited retroactively.
MIERDO MILLAS DE VIBORAS FURIOSAS PARA ESTRANGULARTE MUERTO

ragnarok667

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #3 on: August 14, 2009, 10:41:46 pm »

Ah ! Finally some feedback !

Hmm.. thanks for the positive attitude about it... I wasn't sure if this was done already.. as I would expect from a game with thousands of players that works on linux ?  :P

I'd be very happy to hear about people who have used it a little and see if it makes bulk modifications and managing lots of creatures easier. I'm actually using it right now to make my first new creature / civ and see how it goes.

I don't know of any bugs yet, but I'm sure there has to be quite a few, so I'd like to get some feedback on that.

Also, if any of you are interested in using it / giving feedback, that'd be great. I unfortunately don't have a job right now until my classes resume in september... but LOTS of time for programming and "dwarfing", so I'll gladly explore any ideas and features you want to throw at me and try to implement everything that can help.

[Edit: about features, you should check the filters (top left corner), it allows you to list only the objects that have a specified identifier or data in one or more of their tags/tokens [identifier:tag]. so you could filter creature_standard.txt by identifier and search for mega and it would find all the creatures that have a tag that contains MEGA, or you could make it a "exact match" search and try for "megabeast", that way it would not list creatures with "semimegabeast", for example.
there is also the templates [select an object in the left list, right click and chose insert. depending on the file/content type, it will either add a basic template in directly (like the creature files), or give you a dialog to choose from (item, plant, etc.) ] which allows you to create basic objects for every type of file... they are very basic since I don't know much about modding this game yet, so this is something I'd like more input on :)
« Last Edit: August 14, 2009, 10:54:35 pm by ragnarok667 »
Logged

ragnarok667

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #4 on: August 14, 2009, 10:50:13 pm »

  Still, amazing how you made this after only a few weeks of playing. This is indeed very cool.

For only two weeks of playing DF, you did a pretty good job.

Hmm, about that... having no school for the end of the summer and no job right now... it created an opportunity for a LOT of dwarfing (as I like to call it).. something in the range of 5-10hours a day for around 10 days out of those 2 weeks :P

Plus I really was in need of a good game made out (moslty) only ascii characters :) [yes, I don't play with graphics :P]
Logged

Daywalkah

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #5 on: August 14, 2009, 10:50:45 pm »

Can you make it rain in DF? (random I know, I ask everyone this, I have a major forest fire problem during sieges)

I'll try it out eventually, I gots another Elder Cyclops siege (OMG! THEY WON'T STOP!).
My one marksdwarf (yes just one marskdwarf on steroids, already 23 cyclops kills, no lie, legendary in many millitary skills) has been protecting my BRAND NEW 7 dwarf fortress for almost a year now. If he dies, I'll......well he just won't die.
Logged
Magma cancels flow; interrupted by Ironblood.

ragnarok667

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #6 on: August 14, 2009, 10:52:25 pm »

Hmm... rain...

As far as I know, rain is based on the weather system and comes in a rather (like in real life ?) random fashion...

Do you mean forcing the game to make it rain ? That I'd have no idea really :(
Logged

Duke 2.0

  • Bay Watcher
  • [CONQUISTADOR:BIRD]
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #7 on: August 14, 2009, 10:57:01 pm »

 Know this is a RAW editor, not a data editor. The program you are thinking about would change DF while it runs to get an effect. This program merely makes it easier to modify the text documents in the RAW folder.

 Edit: And for anybody wondering: Rain is not in those files.
Logged
Buck up friendo, we're all on the level here.
I would bet money Andrew has edited things retroactively, except I can't prove anything because it was edited retroactively.
MIERDO MILLAS DE VIBORAS FURIOSAS PARA ESTRANGULARTE MUERTO

Daywalkah

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #8 on: August 14, 2009, 10:57:08 pm »

That stinks..... The forest fires my dwarves start cause many problems (long/short story).

While it runs almost always=crashes. Evil contraptions.
Logged
Magma cancels flow; interrupted by Ironblood.

Captain Mayday

  • Bay Watcher
  • A Special Kind of Terrible
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #9 on: August 15, 2009, 01:03:02 am »

Actually, this would be quite useful for people to detect errors in their body definitions.
Is it possible to add functionality to a) parse existing body files for body parts and allow them to be inserted, and b) to make sure that the body parts are added to the creature in the correct order so that body parts are loaded after the body parts they depend on, possibly marking body parts with dependencies that do not exist in the creature already.
Logged
Why not join us on IRC? irc.newnet.net #bay12games

Deon

  • Bay Watcher
  • 💀 💀 💀 💀 💀
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #10 on: August 15, 2009, 02:36:50 am »

I have no need in such tool because I'm prety used to notepad++ with special syntax highlighting I've set up for myself.
It would be useful for starting people though.

A suggestion: make various similar tags to be differentiated by color. It really helps (at least for me).
I.e. ACTIVE_SEASON, WEAPON: , ARMOR:, and the like.

P.S. Also it would be nice if you removed obsolete tokens (like that LARGE_LAVA etc. you show on your screenie), they are from 2d version of DF.

P.P.S. Also I don't think that making an attack string to take a few strings is a good idea, you could find a way to place it on a single screen in your interface to avoid hundred of errors (i.e. make it a single "attack" string with other options as parameters, i.e. damage, type and other additions like bloodsucking or latching).

Good luck with your work!
« Last Edit: August 15, 2009, 02:43:52 am by Deon »
Logged
▬(ஜ۩۞۩ஜ)▬
✫ DF Wanderer ✫ - the adventure mode crafting and tweaks
✫ Cartographer's Lounge ✫ - a custom worldgen repository

ragnarok667

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #11 on: August 15, 2009, 08:06:33 am »

a) parse existing body files for body parts and allow them to be inserted

a) Yes, this is quite possible. The code structure is mostly there already. It would be nearly the same as loading the CREATURE tokens into the insert menu.

My long term goal was to ask the user to open the raw/objects folder directly instead of one file at a time. That way, more features that are dependant on more than one file would be possible.

Question: If the user adds custom raw text files, can they have more than one "object type" in them ? like [CREATURE:***] and [ITEM:***], for example ?

b) to make sure that the body parts are added to the creature in the correct order so that body parts are loaded after the body parts they depend on, possibly marking body parts with dependencies that do not exist in the creature already.

b) I did intend to add some kind of parsing/validation to the loaded file. But yeah, a basic thing like order of body parts should be good place to start.

I have no need in such tool because I'm prety used to notepad++ with special syntax highlighting I've set up for myself.
It would be useful for starting people though.

This is exactly what I was thinking in my description. Until this utility gets a lot more features, it won't be all that useful for experienced modders. But for beginners, this can potentially reduce by a lot basic structure, typo and logic errors.

And this is also why people like you are needed to help me understand what is exactly needed, because like I said earlier, I'm barely a DF beginner myself :)

A suggestion: make various similar tags to be differentiated by color. It really helps (at least for me).
I.e. ACTIVE_SEASON, WEAPON: , ARMOR:, and the like.

This is a great idea for quick visual identification. One problem : I don't know about "and the like" that much. I do have two alternatives if you're interested though. #1. Since you already added custom syntax rules to Notepad++ to help editing the raw files, would you be interested in quickly reviewing the tags.ini file for my program and setup colors for similar tags ? I'd imagine it could be something like [IDENTIFIER:DATA] *tab* color *tab* info/description instead of the current [IDENTIFIER:DATA] *tab* info/description. Or #2 : Message me a copy fo your custom syntax rules for Notepad++ so I can get a better idea of what experienced modders like so I can add something similar color-coding wise.

P.S. Also it would be nice if you removed obsolete tokens (like that LARGE_LAVA etc. you show on your screenie), they are from 2d version of DF.

This is something that would be very useful, but again I have to rely on outside help. It would be as simple as opening tags.ini and removing the brackets around the tokens that aren't valid/are obsolete. This would disable them instantly. Obviously, it would be quite good if someone who knows about those could do that and upload a copy of the modified tags.ini here.

Example from the wiki :
marked "not used anymore": ADD_VERSION, GENPOWER
marked "obsolete": SESSILE_LAND

This is why there is a bunch of obsolete tokens in tags.ini, I have no way of knowing other than the wiki and your help.

P.P.S. Also I don't think that making an attack string to take a few strings is a good idea, you could find a way to place it on a single screen in your interface to avoid hundred of errors (i.e. make it a single "attack" string with other options as parameters, i.e. damage, type and other additions like bloodsucking or latching).

I get the idea that [ATTACK:***] is quite the complex string, but unfortunately, I'm not sure I fully understand it yet either...
[ As a side note, I just tried to mod a new creature/civ into the game, and it crashes at worldgen... so I'm guessing I'll need a lot more experimenting...  (btw, is there a log of where the worldgen crashed so I can see why ?) ]
So, yes, I would like to add more specific options for [ATTACK:***], and as Captain Mayday asked, for [BODY:***] too.
These two tokens seem to be at the limit of what the token structure can support without becoming too complex and at the same time this makes them harder to handle on beginners and a lot more error prone. So, in respect with the goal of this software, this should be high priority.

Also, I've noticed while looking at creatures / body yesterday (while *trying* to mod) that BODY can also be linked to [ATTACK:***] (by using BYTOKEN instead of BYTYPE). So I think I should work on getting BODY validated first, and then attack. Any thoughts on this anyone ?

Good luck with your work!

Taking the time to give out all those ideas was very helpful, thanks !

And my thanks to the others who posted before, too !
Logged

Deon

  • Bay Watcher
  • 💀 💀 💀 💀 💀
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #12 on: August 15, 2009, 11:24:27 am »

1) Yes, it's better to validate body first (and by validate I mean to make sure that all bodyparts are connected to something already present; do you know that the order in BODY: is an important thing? If you add a new bodypart in [BODY:***] it should connect to something already mentioned). Then you'll be able to assign attacks to specific body parts.

2) Attack string:
[ATTACK:MAIN:BYTYPE:MOUTH:bite:bites:1:6:GORE] [*ATTACKFLAG_CANLATCH*][*ATTACKFLAG_WITH*][*OPTIONAL_ABILITIES*]

Let me expain:
a) [*ATTACKFLAG_CANLATCH*] = [ATTACKFLAG_CANLATCH] , it's an optional flag which means that the creature can "latch" with an attack (it means that it can connect to an attacked bodypart with the attacking bodypart and further attacks will "shake" the opponent by that attacked bodypart).
b) [*ATTACKFLAG_WITH*] = [ATTACKFLAG_WITH] , it's an optional flag which means that there will be indication in the attack notification (combat text) that the creature attacked with THIS bodypart.
c) [*OPTIONAL_ABILITIES*] - there could be 2 different abilities:
 - [SPECIALATTACK_SUCK_BLOOD:50:100] - this means that the creature sucks blood with an attack, causing additional bleeding
 - [SPECIALATTACK_INJECT_EXTRACT:50:100] - this means that the creature injects extract; you should validate that the creature has extract and antivenin first.

======================================================

About attack parameters:
[ATTACK:a:b:c:d:e:f:g:h]
a) MAIN/SECOND - "MAIN" is always used, there may be multiple attack strings with "MAIN" and they will be used randomly' ; and "SECOND" attacks are used only when all "MAIN" are not possible (i.e. bodypart grabbed/removed).
b) BYTYPE/BYTOKEN - "BYTYPE" means that the attack will be assigned to any "type" assigned in "c)"; types are: GRASP/MOUTH/STANCE/LIMB/UPPERBODY/etc. , so they are not names of bodyparts but tokens asssigned to them; "BYTOKEN" means that "c)" will indicate an EXACT bodypart name.
c) It means what does the creature attack with: if "b)" is "bytype", it lists a  bodypart type token; if "b)" is "bytoken", it lists the exact name of the bodypart to attack with.
d) , e) are first and third person text for the attack, i.e. "bite";"bites", this is how the attack is shown in the combat log.
f). g) are minimum and maximum values of the attack roll. More: attack formula.
h) Damage type. It may be: GORE/SLASH/BLUDGEON/PIERCE/BURN/HEAT/COLD.

GORE is a general whip/mouth attack, it can cut bodyparts off with a very high attack value
SLASH is a common sword/axe/claws attack, it easily cuts bodyparts off
BLUDGEON is a common foot/hammer attack, it sends people flying
PIERCE is a common spear/stinger attack, it pierces internal organs and makes the attack to be stuck
BURN is like hot/acidic attack, it's rarely used
HEAT is a lightning/fire attack, it damages the whole bodypart
COLD is a freezing attack, again it damages the whole bodypart


I hope it helps.
« Last Edit: August 15, 2009, 11:27:54 am by Deon »
Logged
▬(ஜ۩۞۩ஜ)▬
✫ DF Wanderer ✫ - the adventure mode crafting and tweaks
✫ Cartographer's Lounge ✫ - a custom worldgen repository

ragnarok667

  • Bay Watcher
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #13 on: August 15, 2009, 03:39:05 pm »

Thanks !

This very much confirms what I thought was happening as fact and helps me understand more of what I haven't experimented with.

As stated before, I'll start with adding parsing/validation for body entries first. But to link creature and body files, I'll have to allow more than one file to be loaded at once. Luckily, using OOP properly, I made the file loading interface it's own object, so it shouldn't be to hard. I'll just have a list of "file objects" instead of a single pointer :)

I asked a question earlier but I think the post was too long for anyone to see :

Can someday add more than one type of object in the same file, example, lets say : ITEM_AMMO and BODY ?  or MATGLOSS_WOOD and CREATURE ?
Logged

Deon

  • Bay Watcher
  • 💀 💀 💀 💀 💀
    • View Profile
Re: New utility : Editor for the raw files...
« Reply #14 on: August 15, 2009, 03:46:00 pm »

The contents of the files are determined by [OBJECT:****] string which comes second (after file name which is necessary too).
I doubt that it's possible to have 2 [OBJECT:***] paragraphs in the same file, however I've never tried it.
Logged
▬(ஜ۩۞۩ஜ)▬
✫ DF Wanderer ✫ - the adventure mode crafting and tweaks
✫ Cartographer's Lounge ✫ - a custom worldgen repository
Pages: [1] 2