Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Author Topic: Raw file tag sorter  (Read 2192 times)

Rochndil

  • Bay Watcher
  • Mad Modder
    • View Profile
Raw file tag sorter
« on: March 05, 2009, 04:10:08 pm »

Greetings!

I was just looking at a modded raw file, and I noticed that the tags in this particular file were all jumbled up. Now, I'm pretty sure that they'll import fine that way, but it sure makes it challenging to READ the thing.

So, wouldn't it be possible to create a fairly simple rules-based text parsing program that would step through the raw file and sort all the tags for each type of object into a consistent order? Obviously rules would have to be drawn up, but I'm fairly sure that Toady's original entries are either consistent or close to it (to use them as a model).

I did a search or two and don't THINK such a beast already exists. I've seen far more complex applications designed by the talented members of this forum, so I'm sure it's do-able from a technical standpoint.

Is this, in your enlightened opinions:

1. Useful?
2. Writable?

I'd offer to take a stab at it, but I'm a tech writer, not a programmer. I do have an unrelated project I've been working on for half of forever, so I actually do have something to contribute - soon!

Rochndil, who has many ideas he cannot bring to fruition...
Logged
Build an Elf a fire, keep him warm for a day.
Drown an Elf in magma, keep him warm for the rest of his life!

Footkerchief

  • Bay Watcher
  • The Juffo-Wup is strong in this place.
    • View Profile
Re: Raw file tag sorter
« Reply #1 on: March 05, 2009, 04:41:15 pm »

1. It sounds extremely handy, though I'd hate to see my amazing memory for tag locations become irrelevant.
2. Very writable, I think.  It would probably need a list of permitted "start" tags (stuff like [CREATURE:X], [ENTITY:Y] and so on) so that it could find the beginnings and ends of entries without depending on whitespace.
Logged

deej

  • Bay Watcher
  • Beeswax!
    • View Profile
    • Showoff Site
Re: Raw file tag sorter
« Reply #2 on: March 05, 2009, 05:27:22 pm »

I agree that the raws need organization.  I spent more time manually alphabetizing them than actually inputting stuff for my mod.

In some cases strict alphabetizing may not be wanted though, for example I left the gems in their value categories and only alphabetized within those categories.

Footkerchief

  • Bay Watcher
  • The Juffo-Wup is strong in this place.
    • View Profile
Re: Raw file tag sorter
« Reply #3 on: March 05, 2009, 05:29:56 pm »

I agree that the raws need organization.  I spent more time manually alphabetizing them than actually inputting stuff for my mod.

In some cases strict alphabetizing may not be wanted though, for example I left the gems in their value categories and only alphabetized within those categories.

Are you talking about alphabetizing the entries themselves?  Like putting CREATURE:TIGER above CREATURE:ZEBRA?  That might be useful too, but the OP is talking about sorting the tags within each entry.
Logged

deej

  • Bay Watcher
  • Beeswax!
    • View Profile
    • Showoff Site
Re: Raw file tag sorter
« Reply #4 on: March 05, 2009, 05:35:12 pm »

Ah, that's what I get for agreeing before understanding :D

Yes, I meant the entries themselves.... the tags in the mineral, gem, and layer entries aren't varied enough to have even noticed if they weren't in order  :-[

i2amroy

  • Bay Watcher
  • Cats, ruling the world one dwarf at a time
    • View Profile
Re: Raw file tag sorter
« Reply #5 on: March 05, 2009, 05:44:25 pm »

This would be pretty cool, especially if it would work on macs as well. Though I would feel bad not having to use my amazing memory for tag placement like Footkerchief said.
@deej: Yeah, organizing the entries is hard, but once you finish organizing them then you never have to do it again as long as you make sure to start any new entries in their proper place.
Logged
Quote from: PTTG
It would be brutally difficult and probably won't work. In other words, it's absolutely dwarven!
Cataclysm: Dark Days Ahead - A fun zombie survival rougelike that I'm dev-ing for.

Sinergistic

  • Bay Watcher
    • View Profile
Re: Raw file tag sorter
« Reply #6 on: March 06, 2009, 03:00:28 am »

Most advanced text editors already have sort functions.

To do it in notepad++, just highlight the raw (minus the header tag. ex: [CREATURE:UNIQUE_NAME]) and go TextFX -> TextFX Tools -> Sort lines case insensitive (at column).

I'm sure other text editor programs that are slight more advanced than notepad (not ++) should be able to do this as well.

The downside is you need to highlight each raw you want sorted, though notepad++ will let you make a macro for the TextFX -> TextFX Tools -> Sort clicks.

Though personally, I don't see the gain in sorting them alphabetically. The raws already have a kind of order to them, and I've spent so much time messing around with them, I can find the tags I want really quick, and the readjustment period would put me off.

As for making a tool to do nothing but sort raw files: Could be very useful if people want to sort them. I would include some options, such as placing all [NAME:_:_] at the top of the raw, grouping the [NOPAIN/NOBREATH/NO____] tags all on one line.

Another option/feature would be expanding/contracting raw files. IE, the NOPAIN/BREATH/SLEEP etc tags all being on one line, using this option would place them all on their own lines, and using it again would contract them back to all being on the same line. You could expand this to work for entire entries ("Are creatures you aren't working on getting in your way? Contract the ones you aren't working on leaving the ones you are working on very visible!")
Logged

Rochndil

  • Bay Watcher
  • Mad Modder
    • View Profile
Re: Raw file tag sorter
« Reply #7 on: March 06, 2009, 06:24:26 pm »

Good afternoon!

I was originally writing about sorting tags WITHIN a given entity, but sorting the entities themselves would also be very useful. I've been mulling this idea over, and some of your comments made me think of an additional possibility.

Any program smart enough to sort the tags into their correct order, and possibly re-order the entities themselves, would be just a hair away from being able to parse out the contents of the given file to a database - heck, it might NEED to do that to do the sorting efficiently.

Think for a moment how useful it would be to modders to have all the raw data in a nicely organized database display, where you could compare entities to each other, easily make changes that spanned a group of entities at one time, import and merge compatible entities into one another, and then when it was all done, just write out the data (correctly) into a raw TXT file.

Now THAT would totally rock. And, I don't think it would be that difficult to accomplish. It would certainly make modding a LOT easier, maybe even easy enough for ME to do. :)

Rochndil, who thinks too much sometimes...
Logged
Build an Elf a fire, keep him warm for a day.
Drown an Elf in magma, keep him warm for the rest of his life!

SolarShado

  • Bay Watcher
  • Psi-Blade => Your Back
    • View Profile
Re: Raw file tag sorter
« Reply #8 on: March 06, 2009, 06:32:07 pm »

Not that i'm shooting down this idea, but something that detailed may be rendered useless with the comming update. I take that back, not useless, just outdated. If you've done that much, i suppose you could tweak it to work with the new raws.

The sorter should still work tho, sure Toady isn't changing the format of the raws that much,
Logged
Avid (rabid?) Linux user. Preferred flavor: Arch

Sinergistic

  • Bay Watcher
    • View Profile
Re: Raw file tag sorter
« Reply #9 on: March 06, 2009, 07:41:56 pm »

Not that i'm shooting down this idea, but something that detailed may be rendered useless with the comming update. I take that back, not useless, just outdated. If you've done that much, i suppose you could tweak it to work with the new raws.

The sorter should still work tho, sure Toady isn't changing the format of the raws that much,

lol, unless toady is planning on inventing an entire new language, I can't see this being a problem.
Logged

Footkerchief

  • Bay Watcher
  • The Juffo-Wup is strong in this place.
    • View Profile
Re: Raw file tag sorter
« Reply #10 on: March 06, 2009, 09:12:30 pm »

Yeah, a simple version of this program would not depend very heavily on the particulars on the tags themselves.  It would just need a rather short list of entry declaration strings, which don't change very much between releases (next version has a couple new ones plus stones/gems/metals getting merged into INORGANIC).
Logged

ZeroGravitas

  • Bay Watcher
    • View Profile
Re: Raw file tag sorter
« Reply #11 on: March 06, 2009, 09:17:20 pm »

i don't get it. i've never had  a  problem with using the Control-F search function in any text file before.
Logged

Martin

  • Bay Watcher
    • View Profile
Re: Raw file tag sorter
« Reply #12 on: March 08, 2009, 12:27:13 am »

It should be pretty easy to do. Basically you want a list of tags already sorted, a regex that can identify each entry and then break it down into tags, then walk the list, bubble up the matches, and output the result.

Rather than make it a standalone program, could any of these sites host a php page that you could just copy/paste in your file and it'll sort it and hand it back. No distribution, no platform issues, auto update.

Rochndil

  • Bay Watcher
  • Mad Modder
    • View Profile
Re: Raw file tag sorter
« Reply #13 on: March 08, 2009, 09:15:22 am »

That's an excellent suggestion, Martin. Part of what I do for a living is software distribution, and I heartily agree that a web-based app would be a great deal easier to handle than a distributed executable.

Rochndil, currently sorting through his mods for the NEXT big DF build...
Logged
Build an Elf a fire, keep him warm for a day.
Drown an Elf in magma, keep him warm for the rest of his life!