Bay 12 Games Forum

Dwarf Fortress => DF Modding => Tilesets and Graphics => Topic started by: fricy on January 28, 2016, 08:35:17 am

Title: Dwarf Fortress graphics repositories
Post by: fricy on January 28, 2016, 08:35:17 am
The purpose of this thread is to discuss the future of the DFgraphics (https://github.com/DFgraphics) organization.

The github repository was first created to help the management of the various Starter packs (LNP), and to create an archive if older versions are needed.

Packs maintained:
Afro (Spacefox fork) (http://dffd.wimbli.com/file.php?id=9137)
CLA (http://www.bay12forums.com/smf/index.php?topic=105376.0)
Duerer (http://www.bay12forums.com/smf/index.php?topic=142083.0)
Gemset (http://www.bay12forums.com/smf/index.php?topic=150753.0)
Grim Fortress (http://www.bay12forums.com/smf/index.php?topic=122421.0)
Ironhand (http://www.bay12forums.com/smf/index.php?topic=122400.0)
JollyBastion (http://www.bay12forums.com/smf/index.php?topic=104261.0)
Mayday (http://www.bay12forums.com/smf/index.php?topic=137370.0)
Obsidian (http://www.bay12forums.com/smf/index.php?topic=126934.0)
Phoebus (http://www.bay12forums.com/smf/index.php?topic=137096.0)
Shizzle (http://dffd.wimbli.com/file.php?id=7205)
Spacefox/LeoCean (http://www.bay12forums.com/smf/index.php?topic=129219.120)
Taffer (http://www.bay12forums.com/smf/index.php?topic=107924.0)
Tergel (http://www.bay12forums.com/smf/index.php?topic=145802.0)
Wanderlust (http://www.bay12forums.com/smf/index.php?topic=145362.0)

Contribution guide:
1, Register a github (https://github.com/) account, learn a bit about git repository management. If you prefer a GUI, I recommend Sourcetree. (https://www.sourcetreeapp.com/)
2, Fork an existing, or create a new repository
3, Make your edits and commit them to your (forked) repository. If you make lot's of edits, don't send them in one giant commit, but group the changes in logical chunks, so it's easier to review. The commit message should be easy to understand.
4, Send a pull request with your changes if making edits, or a request for inclusion if it's a new repo
5, If you are the manager of the repo, review and merge pull requests, create version tags.

Format:
The repositories are created to support PyLNP's reduced raw format.
Reduced raw format was designed to maximise ease of installation, compatibility across DF versions and with other mods, and to minimise file size for storage and distribution. It is quite simply a data and raw folder, identically structured to vanilla DF, with all unmodified files removed. It can thus be installed simply by overwriting a vanilla install of DF. Additionally graphic packs must contain a content manifest detailing df version, content version, contributor name and pack name.
Folders to include:
/data/init
/data/art
/raw/objects
/raw/graphics

TODO:
Create organization
Recruite contributors
Add appropriate license to all packs
Better guide
Title: Re: Dwarf Fortress graphics repositories
Post by: fricy on January 28, 2016, 08:35:39 am
-reserved-
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on January 28, 2016, 10:17:24 am
PTW
Title: Re: Dwarf Fortress graphics repositories
Post by: Abadrausar on January 28, 2016, 11:34:44 am
Hi Fricy!

Good work with the repo.

I use a lot bisasam tileset (http://dwarffortresswiki.org/index.php/File:Bisasam_20x20.png) or his yobbo variant (http://dwarffortresswiki.org/index.php/File:Bisasam_20x20_mod_T.png), both with tilesize 20x20 are included in the DF wiki tileset repo (http://dwarffortresswiki.org/index.php/Tileset_repository)

I have remarked that in your repo no 20x20 tilesets are included, the reason that makes me think that they are important is that if you play a nanofort (1x1 embark with 48x48 tiles) a 20x20 tileset is your best bet for not having to scroll at all when you have a screen resolution of 1920x1080.

As you know nanoforts are FPS aware, and have been blessed by Toady one as one of the ways of delaying the FPS death of the game. These underappreciated tilesets are sharp and clear, very defined, however if you use a more frequent and known 24x24 tileset you are forced to scroll.

Usually I copy myself those tilesets and modify the inits accordingly, but I think that it could be a good idea to offer those tilesets to all the users of the pack even with a special recommendation for people interested in doing nanoforts.

Cheers.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 28, 2016, 11:48:25 am
Hi Fricy!

This is probably a better place to write about this things.   Can you create a subfolder in the repository for 24x24 versions and placeholders for the different tilesets so that I can commit the ones I'm creating?
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 28, 2016, 12:54:44 pm
Ok the results are out of a Spacefox 16x16 to 24x24 upscale.   Here is the output of two scaling algorithms (after deconstructing each grid, applying them to each individual tile separately, and then reconstructing the file):

HDx (the very same one used by Arclance's Tile Resizer (http://www.bay12forums.com/smf/index.php?topic=87196.0)):
(https://dl.dropboxusercontent.com/u/408446/DF/Spacefox_24x24_HDx.png)


Weifu2x (the resizer is taken from https://github.com/tanakamura/waifu2x-converter-cpp):
(https://dl.dropboxusercontent.com/u/408446/DF/Spacefox_24x24_Waifu2x.png)


The 16x16 original, for comparison, is here:
(https://dl.dropboxusercontent.com/u/408446/DF/Spacefox_16x16.png)

Both are good, but I think Weifu2x is crisper and clearer (much better than my previous attempt because last time I didn't deconstruct the grid first).  What do you guys think (a good tile to compare is the tree at the bottom-right corner, and the rails and engraved walls)?

EDIT:  Note that you must first open all images in separate tabs.  I noticed that the Forum's engine is reducing their quality.
Title: Re: Dwarf Fortress graphics repositories
Post by: Clatch on January 28, 2016, 01:00:47 pm
Not to discount the Lazy Newb packs, but I can see this being a nice addition to the Homebrew (http://brew.sh) games tap for installing graphics packs into vanilla DF.

brew tap homebrew/games
brew install dwarf-fortress
brew install <graphic_set>
Title: Re: Dwarf Fortress graphics repositories
Post by: fricy on January 28, 2016, 03:09:53 pm
@Abadrausar: Good point, there's Taffer 20x and Gemset 24x, but other than that the sets are not high def friendly. It looks like Spacefox will be converted to 24x, and maybe Phoebus too. The question is, if it's necessary at all to package both 16x and 24x versions, or not. Opinions?
As for Bisasam: this repo so far only contains "graphics sets" - by that I mean they all contain some customization besides a tileset image, like a color scheme, init modifications, creature gfx, etc. For some sets these modifications are minor, like grim fortress, so it could be argued that they are "tilesets" and not "graphics sets".  There's a separate tileset collection folder in the starter packs for these. I guess those also deserve a place here, but before I do that some brainstorming is in order on how to make them easier to install/configure with Pylnp. Paging PeridexisErrant.

@Quiet-Sun: Nice. I read your post in Spacefox, please post this and the app too - if there are no copyright problems with posting some proprietary Matlab software. It expensive...
I think both algorithms produce nice results, the texture details are better with weifu, but hqx is better when the tile is uniform and contains straight lines. Best results would be to photoshop a mix-and-match. I am not going to ask you to do that. :)
Instead of a subfolder a new branch would be better. I can do it if you want me to, but you know, the secret purpose of this thread is to find people who can send me pull request I can review at my leisure. :D

@Clatch: Excellent idea, though it would be something like:
brew install dwarf-fortress - - with-gfx phoebus
Do you want to work on it? I need to study brew formulas before I can manage that.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on January 28, 2016, 03:55:27 pm
Yay, PTW and comment!

Re: licenses - my preference for this is Creative Commons Share-Alike, since graphics packs are clearly not code.

@Abadrausar: Good point, there's Taffer 20x and Gemset 24x, but other than that the sets are not high def friendly. It looks like Spacefox will be converted to 24x, and maybe Phoebus too. The question is, if it's necessary at all to package both 16x and 24x versions, or not. Opinions?
As for Bisasam: this repo so far only contains "graphics sets" - by that I mean they all contain some customization besides a tileset image, like a color scheme, init modifications, creature gfx, etc. For some sets these modifications are minor, like grim fortress, so it could be argued that they are "tilesets" and not "graphics sets".  There's a separate tileset collection folder in the starter packs for these. I guess those also deserve a place here, but before I do that some brainstorming is in order on how to make them easier to install/configure with Pylnp. Paging PeridexisErrant.

@Quiet-Sun: Nice. I read your post in Spacefox, please post this and the app too - if there are no copyright problems with posting some proprietary Matlab software. It expensive...

I'd tend to ship all the tilesets, and stick with the original sizes by default (for better small-screen and creature tile compatibility).  However we should probably take any references to tilesize out of the pack names, and maybe think about improving PyLNP support for variable-tilesize packs and a target tilesize.

If it doesn't touch /raw/graphics/, it's a tileset.  Otherwise, I think we might as well maintain it.

re: Matlab - the runtime is expensive/proprietary, but the code can still be Free/open-source.  GNU Octave supplies a mostly-drop in replacement for the runtime too.  Plus with source I can probably translate to Numpy, and then we can bundle it into PyLNP...
Title: Re: Dwarf Fortress graphics repositories
Post by: Taffer on January 28, 2016, 04:22:29 pm
TODO:
...
Add appropriate license to all packs

Quote from: Taffer
The short answer is that I consider the license to be CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/), and you can use it as you like as long as you give me credit for the tileset.
Source (https://github.com/nc-z/taffer/issues/1).

Note that seems very legally "grey". Bitmap fonts simply aren't eligible for copyright in many countries. DF's code pages are in the public domain for a reason and it wasn't corporate generosity.

Further complications:
I would rather abandon the pretense that we can sort this out, at least in regards to the tileset portions of the packs. I have attempted to resolve this in my own pack by crediting every tile that I copied from existing non-public domain sources (and I asked permission in advance for them), but I wouldn't fancy legally testing my theory. Inviting all contributers to specify a license implies that the copyright is theirs, which might upset people who perceive (fairly or unfairly) that the credit for their tiles has been stolen.

With that aside, thank you kindly for maintaining graphics packs that have lost their original maintainers.

In regards to resizing, I'm really not a fan of any resizing algorithm that I've seen, and I've tried many of them. Still, HDx, Weifu, SuperSai, etc at least look better than DF's hideous zoom feature, and many people seem to enjoy using larger sets. Perhaps they should be included.

@PeridexisErrant: Out of curiosity, is there any reason why you don't seem to ship my tileset? I keep (http://www.bay12forums.com/smf/index.php?topic=107924.msg3501564#msg3501564) noticing (http://www.bay12forums.com/smf/index.php?topic=107924.msg6661901#msg6661901) screenshots of people using my tileset with messed up raws, and my suspicion is that people install my tileset over top of the default Starter Pack installation, which doesn't use vanilla raws by default. This is derailing the point, however.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 28, 2016, 04:30:47 pm
@Quiet-Sun: Nice. I read your post in Spacefox, please post this and the app too - if there are no copyright problems with posting some proprietary Matlab software. It expensive...
I think both algorithms produce nice results, the texture details are better with weifu, but hqx is better when the tile is uniform and contains straight lines. Best results would be to photoshop a mix-and-match. I am not going to ask you to do that. :)
Instead of a subfolder a new branch would be better. I can do it if you want me to, but you know, the secret purpose of this thread is to find people who can send me pull request I can review at my leisure. :D

Of course! My intention was to truly take advantage of github, but I was thinking there should be separate 24x24 and 16x16 repositories.  My request to you is to create separate 24x24 repositories so that I can clone them, change them, and commit them while leaving the 16x16 in peace.    My intention is to create 24x24 versions of all of them following your system:  First I will create and commit 24x24 versions of the 40.24 sets, and then after you massage/approve them, submit the 42.05 versions,


re: Matlab - the runtime is expensive/proprietary, but the code can still be Free/open-source.  GNU Octave supplies a mostly-drop in replacement for the runtime too.  Plus with source I can probably translate to Numpy, and then we can bundle it into PyLNP...

My code is actually quite simple (and can be freely distributed without any complaint's from Mathworks), because is mainly a wrapper of pre-existing upscaling HQx and Waifu2x binaries.  Translating it to Python should be no problem!  I just don't know any Python yet and I preferred to have a finished product to iterate upon (instead of a project I don't know when I will be able to get around to do).   Matlab is super expensive, but the runtime component is free.   I will upload a windows binary of my matlab program (along with the source) that can be run without any cost.

A word of caution, upscaling a graphics set takes quite a bit of time (we are talking about several hours), because the best results are obtained when you upscale each tile separately.   I wonder if perhaps is better to pre-generate and pack the 16x16 and 24x24 sets with PyLNP.  Since DF doesn't change that often, perhaps it's also better to keep the upscaling application separate from PyLNP (of course translating it to Python is still totally worth it!).   This way people are very much aware of what they are getting into when they launch it  :P
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 28, 2016, 06:12:37 pm
QUICK UPDATE

I started processing the 40.24 files in the repository and only this ones warrant the creation of a 24x24 version:

- Afro-Graphics
- CLA
- Ironhand
- Mayday
- Obsidian
- Phoebus
- Spacefox
Title: Re: Dwarf Fortress graphics repositories
Post by: Pidgeot on January 28, 2016, 06:36:54 pm
A word of caution, upscaling a graphics set takes quite a bit of time (we are talking about several hours), because the best results are obtained when you upscale each tile separately.   I wonder if perhaps is better to pre-generate and pack the 16x16 and 24x24 sets with PyLNP.  Since DF doesn't change that often, perhaps it's also better to keep the upscaling application separate from PyLNP (of course translating it to Python is still totally worth it!).   This way people are very much aware of what they are getting into when they launch it  :P

Yeah, this is something that's going to be a bit of a problem for integrating upscaling directly into PyLNP. You absolutely have to scale each tile seperately, because the results are generally going to look terrible if you don't, with tiles bleeding into each other and creating odd artifacts.

In the case of waifu, it is an *absolutely awesome* scaling algorithm, but it is also crazy expensive. You wouldn't want to introduce any overhead you can avoid whatsoever... so it would probably be best to call out to C code (possibly using a wrapper so it builds as a C-based Python module?), although even then I suspect it might be too expensive without resorting to GPU acceleration...
Title: Re: Dwarf Fortress graphics repositories
Post by: Clatch on January 28, 2016, 06:40:48 pm
@Clatch: Excellent idea, though it would be something like:
brew install dwarf-fortress - - with-gfx phoebus
Do you want to work on it? I need to study brew formulas before I can manage that.

Sounds good.  I'll give that some attention.
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on January 28, 2016, 07:00:33 pm
What's the objection to including pre-cooked versions of the up-sized graphics sets in the packs? Is it just a matter of file size/avoiding duplicates?
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on January 28, 2016, 09:59:01 pm
What's the objection to including pre-cooked versions of the up-sized graphics sets in the packs? Is it just a matter of file size/avoiding duplicates?

Nothing serious, so we're likely to do this.

In we can make on-the-fly rescaling work well it would be much more flexible though, and could significantly reduce filesize.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on January 29, 2016, 01:54:35 am
fricy "Removed assorted river/lake fauna until raw conflicts are resolved (https://github.com/DFgraphics/Mayday/commit/c7212ff35b29c93f9e349b9d7aac59d44252ba42)" on the 28th, when Habbadax fixed these conflicts back on the 26th (http://www.bay12forums.com/smf/index.php?topic=137370.msg6767173#msg6767173). While fricy has never ignored problems when they are brought to his attention, I think it's important that others realize that he cannot be expected to maintain all of the definition files let alone each set in the repo.

If the game loads for him using a particular set it gets a pass and is "updated" (http://www.bay12forums.com/smf/index.php?topic=140808.msg6766469#msg6766469). I definitely don't blame his process considering the idea of maintaining all those sets seems ludicrous, but I hope that people with the know how could help out on github to avoid these issues in the future. Ideally someone could even reorganize the Frankenstein sets, but just having them defined correctly so that "updated" actually meant updated would be a good start. Heh.

Anyway, good luck.




edit: I pointed Habbadax towards this thread.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on January 29, 2016, 03:02:27 am
Todo: port or write linting tools to catch as much as possible, and hook them up to Travis CI for automated testing.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 29, 2016, 07:33:44 am
I'm all for standardizing the various DF graphics and using a common github repo that all graphics authors can contribute to and I obviously give permission to use all my stuff.
How would it be organized? Each graphics author has his own repo that gets pushed/pulled to the common DFgraphics repo? Or do we directly work on the central repo?


As for the license thing something like
Would be what I'd prefer for my stuff. No idea what a good license for that would be.

Some other things:

For graphic sets, maybe we could agree on a common organization (i.e. where which creatures are on the graphic sets and how they're separated). That would make it easier to spot errors, maintain, and update all graphics of the community with minimal effort.
I assume the easiest would be to follow the organization and order from the raws (I'm probably the only one who doesn't already separate the creature graphic sets by their raw files, but I'm not sure if the order within is the same for everyone).
Ideally we would have a template and a script that generates that template from the raws. so when new creatures are added, it's easier to update.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 29, 2016, 10:30:43 am
For graphic sets, maybe we could agree on a common organization (i.e. where which creatures are on the graphic sets and how they're separated). That would make it easier to spot errors, maintain, and update all graphics of the community with minimal effort.
I assume the easiest would be to follow the organization and order from the raws (I'm probably the only one who doesn't already separate the creature graphic sets by their raw files, but I'm not sure if the order within is the same for everyone).
Ideally we would have a template and a script that generates that template from the raws. so when new creatures are added, it's easier to update.

What great ideas!
Title: Re: Dwarf Fortress graphics repositories
Post by: Clatch on January 29, 2016, 12:19:23 pm
What's the objection to including pre-cooked versions of the up-sized graphics sets in the packs? Is it just a matter of file size/avoiding duplicates?

The multiple art assets can be present in the art directory, but the original init file can be managed by the scripts calling the git repo.  It might be nice to include some standardized notation of art sizes so that they are easily handled without duplicating code: ei, <art_name>_15x15.png, <art_name>_18x18.png, etc.
Title: Re: Dwarf Fortress graphics repositories
Post by: Clatch on January 29, 2016, 12:23:17 pm
It might be an overkill, but it would be nice to have a standardized xml file to suffice as a dictionary of available assets instead of separate repos for sizes or compatibilities (Vanilla, TWBT, etc.).
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 29, 2016, 12:51:46 pm
What's the objection to including pre-cooked versions of the up-sized graphics sets in the packs? Is it just a matter of file size/avoiding duplicates?

The multiple art assets can be present in the art directory, but the original init file can be managed by the scripts calling the git repo.  It might be nice to include some standardized notation of art sizes so that they are easily handled without duplicating code: ei, <art_name>_15x15.png, <art_name>_18x18.png, etc.

This is also a great idea!  This way we can include all resolutions in a single release and not having to have different git repositories for different resolutions.   This nomenclature already exists in the /art/ folder, we just need to implement it inside the raw/graphics folder.   It has an added benefit, if we include 16/24/32 assets in all releases, then we can later use a TWBT-like hack to use different ones at different zoom levels (just a crazy idea).

What about the following summary of these ideas:

1.  Defining a naming structure and tile positioning standard that authors must follow if they want their tileset to be included in both the repository and PyLNP (this means that each creature's position must be exactly the same for all tilesets. i.e. same file, same x,y coordinate).
2.  Updating dead, but loved, tilesets to match this standard (I'm thinking that we can start with spacefox).
3.  Append the resolution to the name of all files within the raw/graphics. Folder (i.e. dwarves.png->dwarves_16x16.png).
4.  Add upscaled assets to those sets that merit it, using the same folder structure (i.e. so now /raw/graphics/spacefox/ contains dwarves_16x16.png, dwarves_24x24.png, & dwarves_32x32.png). 
5.  Adding a resolution selection option to PyLNP, so that PyLNP can customize the .txt files that tell DF which file to use (defaulting to the original if no other resolutions are present).

Another crazy idea is creating a universal repository with all tilesets in subfolders (similar to what fricy is currently doing with the hybrids), but including all graphics assets (they must be standardized in terms of filenames and tile positioning) and write a separate application that allows a user to chose which dwarves to use, which elves, which animals, etc.   Similar to the way many Skyim mods install.  All that would be necessary is updating the .txt files so that they use the right files and resolutions

Just brainstorming


EDIT:  I have been looking at the raw/object folders and I think I could make a MATLAB script that takes a current graphics set and standardizes it (regardless of the original structure), both in terms of its png and its text files based on a given DF build.   Hmmmm! I'll give it a try and let you know how it goes.  If I can make it work then we could use it to very easily adapt to any new structure or clasification Toady may come up with.  I'll give it a try with creatures and animals first and let you know how it goes
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 29, 2016, 05:21:24 pm
Hey guys, I need some help with something.   Looking at the txts I see that most lines use square brackets:

[DEFAULT:PHOEBUS_ANIMALS:0:0:AS_IS:DEFAULT]

but sometimes there are parenthesis ():

((CREATURE_GRAPHICS:FROGMAN))
   ((CHILD:SPHR_MON:0:2:AS_IS:DEFAULT))


and sometimes exclamation marks:

!CREATURE_GRAPHICS:CAT]
   !DEFAULT:PHOEBUS_ANIMALS:0:0:AS_IS:DEFAULT]
   !CHILD:PHOEBUS_ANIMALS:1:0:AS_IS:DEFAULT]
   !ANIMATED:PHOEBUS_ANIMALS:2:0:AS_IS:DEFAULT] zombie
   !!!!!SKELETON:PHOEBUS_ANIMALS:3:0:AS_IS:DEFAULT] skeleton


What is the meaning of these symbols?  why SKELETON always have 4 exclamation marks?  Do I need to keep this formating?   Thanks a lot!

Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 29, 2016, 05:35:25 pm
I was under the impression that only square brackets work. The parentheses and exclamation marks might be fancy ways to comment out lines, i.e. make them non-functional.

Best idea would be to test if these entries actually work in game.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 29, 2016, 05:38:30 pm
I was under the impression that only square brackets work. The parentheses and exclamation marks might be fancy ways to comment out lines, i.e. make them non-functional.

Best idea would be to test if these entries actually work in game.

Thank you so much CLA!  I will for the moment only use entries with a square bracket.   One more question.  Do you know what is the difference between something like this:

[STANDARD:BIRDS:2:0:AS_IS:ANIMATED]

and something like this:

[ANIMATED:DOREN_CRITTERMEN:2:0:AS_IS:DEFAULT]

?

Title: Re: Dwarf Fortress graphics repositories
Post by: LeoCean on January 29, 2016, 06:30:08 pm
Square brackets are the only way for them to be read as graphics, if you just comment out

!CREATURE_GRAPHICS:CAT]
   [DEFAULT:PHOEBUS_ANIMALS:0:0:AS_IS:DEFAULT]

you will make the next graphic below that messed up. Had that problem with alpacas in spacefox. Though for whatever reason it won't mess up the one below the one it messes up.

They are both skelton/zombie/evil class monsters in your examples, the second example is how it is normally done, I've no idea who did the first one or if it'd work but it might. The comments are that way so they are easy to mass reverse, you wouldn't see the same comment on both sides afterall as that would be counter productive. Skeleton is no longer used I believe just the animated but it was left in, in the case that it gets reused.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 29, 2016, 06:43:09 pm
Square brackets are the only way for them to be read as graphics, if you just comment out

!CREATURE_GRAPHICS:CAT]
   [DEFAULT:PHOEBUS_ANIMALS:0:0:AS_IS:DEFAULT]

you will make the next graphic below that messed up. Had that problem with alpacas in spacefox. Though for whatever reason it won't mess up the one below the one it messes up.

They are both skelton/zombie/evil class monsters in your examples, the second example is how it is normally done, I've no idea who did the first one or if it'd work but it might. The comments are that way so they are easy to mass reverse, you wouldn't see the same comment on both sides afterall as that would be counter productive. Skeleton is no longer used I believe just the animated but it was left in, in the case that it gets reused.

Thank you Leo!  The first implementation is the one I found inside Mayday's set.  For example:

[CREATURE_GRAPHICS:TOAD]
   [DEFAULT:CREATURES:0:0:AS_IS:DEFAULT]
   [CHILD:CREATURES:1:0:AS_IS:DEFAULT]
   [STANDARD:CREATURES:2:0:AS_IS:ANIMATED]
   [STANDARD:CREATURES:3:0:AS_IS:GHOST]

I also found it in the Afro-Graphics set for Beefmo animals:

[CREATURE_GRAPHICS:BIRD_STORK_WHITE]
   [DEFAULT:BIRDS:30:0:AS_IS:DEFAULT]
   [CHILD:BIRDS:30:1:AS_IS:DEFAULT]
   STANDARD:BIRDS:30:2:AS_IS:ANIMATED

Although I assume that in this case it will ignore the line because it doesn't have a bracket.

Now, inside Ironhand I see:

[CREATURE_GRAPHICS:DOG]
  [DEFAULT:CRITTER:0:0:AS_IS:DEFAULT]
  [CHILD:CRITTER:1:0:AS_IS:DEFAULT]
  [ANIMATED:CRITTER:2:0:AS_IS:ANIMATED]

I'm a little confused  :o





Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 29, 2016, 07:38:03 pm
Quote from: http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#List_of_Professions.2C_Creatures_and_States
Note that the Tokens GHOST and ANIMATED go in the profession spot, not the texture token as you would expect - this means that you can't have separate graphics for miner zombies, axedwarf zombies, speardwarf zombies, etc.

That's how I understand it works. The others might be mistakes. Again, your best bet is to just test if these creature graphics actually work or not (and update the wiki in case something turns out to be wrong).
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on January 29, 2016, 07:40:32 pm
@Quiet-Sun This may answer a lot of your questions. (http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#Text_File_Syntax)
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 29, 2016, 07:44:23 pm
Quote from: http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#List_of_Professions.2C_Creatures_and_States
Note that the Tokens GHOST and ANIMATED go in the profession spot, not the texture token as you would expect - this means that you can't have separate graphics for miner zombies, axedwarf zombies, speardwarf zombies, etc.

@Quiet-Sun This may answer a lot of your questions. (http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#Text_File_Syntax)


Ah! this is very helpful! thank you guys!  I almost have a working version to show.  I'll post it when I have it
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 29, 2016, 11:03:08 pm
Ok guys!   Here is the result of my creature script.  It works by asking the user to select a reference DF build (I downloaded a fresh 42.05) and a graphic set (I ran it on the Afro, Mayday, Phoebus, and Spacefox taken from the Fricy's repository).  It works like this:

- Open each creature object file from the reference DF build.
- Find the creature tags inside that file.
- Look in all .txt files of the graphic set for the corresponding match.
- Open the corresponding png and extract the tile.
- Add it to a png file with the same name as the DF reference txt file.  Different creatures are placed in different columns.  Different textures and/or professions are placed in different rows.  Each combination is unique and identical regardless of the set.

Here are the outputs for the aforementioned sets for the creature_domestic.txt file:

Afro:
(https://dl.dropboxusercontent.com/u/408446/DF/creature_domestic-Afro.png)

Mayday:
(https://dl.dropboxusercontent.com/u/408446/DF/creature_domestic-Mayday.png)

Phoebus:
(https://dl.dropboxusercontent.com/u/408446/DF/creature_domestic-Phoebus.png)

Spacefox:
(https://dl.dropboxusercontent.com/u/408446/DF/creature_domestic-Spacefox.png)

Each column corresponds to an animal, the rows are:

- DEFAULT
- CHILD
- STANDARD
- ANIMATED
- GHOST
- TRAINED_HUNTER
- TRAINED_WAR

Note that some animals are missing in some of the sets (so those tiles are left blank), but now there is a uniquely defined tile for all sets.

What I'm proposing (based on CLA's idea) is that we create a standard set of text files pointing to standard tiles that are exactly the same regardless of the graphics set.   Then, using a more sophisticated version of my script, we populate the png files for each set.   The advantage is that it becomes very easy to mix and match sets and we can very easily update the whole thing if DF adds or removes creatures.  The other nice thing is that by doing this with a script, the artists can do their own thing in whichever order they want, and then we can just run some code and create standardized versions of the sets.

Anyway, forgive me for the wall of text.   Let me know what you guys think so that I know whether to move forward.

If you want to compare the different sets at your own leisure you can download them here:

https://dl.dropboxusercontent.com/u/408446/DF/Graphics%20Standardization.zip
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on January 30, 2016, 03:49:49 am
Note that some animals are missing in some of the sets (so those tiles are left blank), but now there is a uniquely defined tile for all sets.

What I'm proposing (based on CLA's idea) is that we create a standard set of text files pointing to standard tiles that are exactly the same regardless of the graphics set.   Then, using a more sophisticated version of my script, we populate the png files for each set.   The advantage is that it becomes very easy to mix and match sets and we can very easily update the whole thing if DF adds or removes creatures.

How is this standard definition file being created? I didn't see any in the download.
Is this standard definition file going to be automated somehow?
If the standard definition file is automated, does it know NOT to define a creature's tile if it's left blank?*
If it's not automated, how are you going to mix and match sets?
How are the professions pulled for the standard races and animal people?


*If you define a creature to a transparent tile that's exactly what you'll get in game.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 30, 2016, 07:12:40 am
Quote from: burned
does it know NOT to define a creature's tile if it's left blank?*
Good point. Any better options than commenting out that creature definition?
Spoiler: regarding CLA (click to show/hide)

Speaking of CLA - how do we deal with several creature definitions pointing at the same tile? I use that sometimes (Giant creatures whose base creature uses a capital letter like Lion share the same graphic), and as it stands right now, with the new standard, I would have the same graphic multiple times in several files. Is there a smart way around that without compromising the standard? In the end it's not that much of a maintenance problem, but I figured I'd mention it.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 30, 2016, 10:41:35 am
How is this standard definition file being created? I didn't see any in the download.

1. Is being created using a matlab script.  It is very much a work in progress, I was not planning to release it until we have a standard, but if you are curious here it is:

https://dl.dropboxusercontent.com/u/408446/DF/DF_Graphics_Standardization.m (https://dl.dropboxusercontent.com/u/408446/DF/DF_Graphics_Standardization.m)


Is this standard definition file going to be automated somehow?

2. Fully automatic, no burden whatsoever on the artists.  We would provide matrix templates with (names crossing textures/professions with the respective animals/races) in case artists just want to fill them up and not to bother with making the text files.  However, thanks to the automatic script, artists can still do things in whichever way they want and we can standardize things for them.


If the standard definition file is automated, does it know NOT to define a creature's tile if it's left blank?*

3. I was thinking about keeping each artist work as pure and unpolluted as possible.  This means there will blanks wherever a particular artist has not drawn art.   The creation of a functional graphic set would be done separately (automatically with a separate script) and would involve the user submitting a file in which he/she ranks artists from highest to lowest preference and then the blanks are filled in order of preference.  Alternatively one could chose to fill blanks with letters from a given set (see CLA's concern below).


How are the professions pulled for the standard races and animal people?

We need to define standards for, creatures, beastmen, and major races.  But my plan is to base the order strictly in the way they appear in DF object files.  This way if toady changes something we simply run the script and now is fully compatible.


Speaking of CLA - how do we deal with several creature definitions pointing at the same tile? I use that sometimes (Giant creatures whose base creature uses a capital letter like Lion share the same graphic), and as it stands right now, with the new standard, I would have the same graphic multiple times in several files. Is there a smart way around that without compromising the standard? In the end it's not that much of a maintenance problem, but I figured I'd mention it.

In the standard I'm proposing the tile would be repeated in the matrix.  Note that the artists don't need to repeat things to mantain the standards.  Artists would have two options:

1.  Submit just the png files if they follow the standard (in which case they would certainly need to repeat the tile).
2.  Do it the usual way: submit png files with tiles in any order they want and supporting .txt files indicating which file corresponds to each creature/profession (here is where you would indicate that the same tile would be use for several creatures).   If this is the case, we just run the artist's work though the script and voila! standard version :)

See for example the output of the creature_large_ocean file for Mayday.  They use a fallback texture to indicate a creature when there is no specific art so in the output it just gets repeated.  When it's time to create a fully functional graphic set something like this is what we would do.  Note that we can create automatic rules to fill some of the holes.  We can for example desaturate the animal (make it black and white) to indicate an animated version.

(https://dl.dropboxusercontent.com/u/408446/DF/creature_large_ocean.png)
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 30, 2016, 11:30:26 am
If this is the case, we just run the artist's work though the script
Oh yeah, of course. Didn't think of that.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on January 30, 2016, 12:31:15 pm
I didn't see anything in your script regarding a standard definition file. It reads as if it goes through the existing image(s) and existing definition file(s) and creates png(s) as far as I can tell. But, maybe I don't understand your script and I'm just missing the part that creates the definition file. But, based on your answers, I think this is really about our terminology. Let me start over to avoid confusion.

How is this standard definition TEXT file being created? I didn't see any in the download.
Is this standard definition TEXT file going to be automated somehow?
If the standard definition TEXT file is automated, does it know NOT to define a creature's tile if it's left blank?
If it's not automated, how are you going to mix and match sets?
How are the professions pulled for the standard races and animal people for the TEXT file?

Sorry if I wasn't clear before. I do understand what you have done. I wanted to know how you are going to tackle the standard definition text file. Based on this comment . . .

We would provide matrix templates with (names crossing textures/professions with the respective animals/races) in case artists just want to fill them up and not to bother with making the text files.

 . . . you would just "provide matrix templates."

How?
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 30, 2016, 01:21:35 pm
I didn't see anything in your script regarding a standard definition file. It reads as if it goes through the existing image(s) and existing definition file(s) and creates png(s) as far as I can tell. But, maybe I don't understand your script and I'm just missing the part that creates the definition file. But, based on your answers, I think this is really about our terminology. Let me start over to avoid confusion.

How is this standard definition TEXT file being created? I didn't see any in the download.
Is this standard definition TEXT file going to be automated somehow?
If the standard definition TEXT file is automated, does it know NOT to define a creature's tile if it's left blank?
If it's not automated, how are you going to mix and match sets?
How are the professions pulled for the standard races and animal people for the TEXT file?

Sorry if I wasn't clear before. I do understand what you have done. I wanted to know how you are going to tackle the standard definition text file. Based on this comment . . .

We would provide matrix templates with (names crossing textures/professions with the respective animals/races) in case artists just want to fill them up and not to bother with making the text files.

 . . . you would just "provide matrix templates."

How?

I'm on my phone so I'll give you a more detailed explanation later. Currently, there is no script to create the text files yet, but I don't expect this to be difficult to implement. My focus right now is to develop a proof of concept and that's why I started with the creature images. 

My intention for the next improvement of the script is the generation of the template so give me a couple of days and you'll see them
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 31, 2016, 03:30:53 am
Ok, here is what I mean by a matrix template.  The files you are going to look have all been generated by the automatic script (text and all).  I have now filled all rows if there is at least a standard tile like this:

- If no child, use standard
- If no animated, turn standard B&W and flip it along the left-right direction
- If no trained_hunter, or trained_war, use standard.

tiles where you see black outlines don't exist in a particular graphic set:

creature_birds_new - Phoebus:  Had only standards and childs for all animals, all other tiles filled as above
(https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/creature_birds_new_Phoebus.png)

creature_insects - CLA: Missing giants
(https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/creature_insects_CLA.png)

creature_domestic - Spacefox:  Has it all except trained_hunter and trained_war.  I don't know how many creatures can become trained.
(https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/creature_domestic_Spacefox.png)


creature_mountain_new - Afro:  Empty
(https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/creature_mountain_new_Afro.png)


Anyway, you get the idea.  All converted creature sets for this iteration can be found here:

https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/Graphics%20Standardization2.zip


The idea is that with each tile uniquely defined, artists can fill the gaps without worrying about the text files.   Even if you just want to fill one creature, incorporating such a contribution to a functional graphic set is going to be much easier than Fricy trying to plug holes everywhere in an increasingly large group of graphic sets.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 31, 2016, 07:03:42 am
- If no animated, turn standard B&W and flip it along the left-right direction
I was about to ask whether we could drop that and just leave it as it is, but the more I think about it, the more I actually like it for CLA.

Another thing though: How do we deal with AS_IS vs ADD_COLOR? tell the script which to use? I figure you could let the script check whether the tile only contains white/greyscale or other colors too, but there's always the possibility that someone wants ADD_COLOR even with other colors or AS_IS with no colors (for example those animated tiles, or maybe a polar bear). Replacing all instances of AS_IS with ADD_COLOR is easy enough, but if you have mixed sets, it's gonna get annoying.

Pulling the color out of the raws and coloring the png automatically is problematic too, since it depends on the color scheme.


In the end, I think the most sensible thing to do is to just keep it as ADD_COLOR as default, and have people replace it with AS_IS where they need it. It's the less used alternative anyway.
Title: Re: Dwarf Fortress graphics repositories
Post by: Pidgeot on January 31, 2016, 07:45:11 am
Ok, here is what I mean by a matrix template.  The files you are going to look have all been generated by the automatic script (text and all).  I have now filled all rows if there is at least a standard tile like this:

Obviously this is a work in progress, so the details are not finalized, but one thing I notice is that it's not entirely obvious where the tile boundaries are located just based on the grid lines on the edge. On the first column, the first tile includes both the "left" and the "right" line of black pixels; on subsequent ones it's only the right line that's included.

Would it perhaps make sense to include space for the grid lines, perhaps, and actually extend them into the "tile" area for the sake of overview? That would also make it easier to identify an undefined cell: it's one where every pixel is transparent, rather than one where certain edges are black and everything else s transparent.

(It would also theoretically make it possible to specify special cases, e.g. ADD_COLOR/AS_IS, by using specific RGB values for specific pixels in the border... but that's a more advanced concept anyway, so don't worry too much about that for now)
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on January 31, 2016, 12:00:36 pm
How do we deal with AS_IS vs ADD_COLOR?
I totally forgot about the AS_IS versus ADD_COLOR. Thanks.


The files you are going to look have all been generated by the automatic script (text and all).
&
The idea is that with each tile uniquely defined, artists can fill the gaps without worrying about the text files.

Standard image organization is great for the repo, but the current problem that exists in the repo are with the definition text files. Organizing the images in each set to match a standard means nothing without updated working definitions. The challenge is the definition text file that syncs with the images you are creating.

Anyway, you get the idea.

I do get the idea and I'd love to see you or someone pull this off, but all of some of my questions still remain unanswered.

edit: Well, not all of them since you said it would be automated. Heh.

I don't know how many creatures can become trained.
P.S. Here is a list. (http://dwarffortresswiki.org/index.php/DF2014:Animal_trainer#Train_a_hunting_animal) I hope that helps.

edit 2: link fixed =X!
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 31, 2016, 12:09:37 pm
Obviously this is a work in progress, so the details are not finalized, but one thing I notice is that it's not entirely obvious where the tile boundaries are located just based on the grid lines on the edge. On the first column, the first tile includes both the "left" and the "right" line of black pixels; on subsequent ones it's only the right line that's included.

Would it perhaps make sense to include space for the grid lines, perhaps, and actually extend them into the "tile" area for the sake of overview? That would also make it easier to identify an undefined cell: it's one where every pixel is transparent, rather than one where certain edges are black and everything else s transparent.

My bad, I intended to mark all borders, but forgot.  Now it's done and columns/rows alternate between black and gray

(https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/creature_small_mammals_CLA.png)    (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/creature_fanciful_CLA.png)

Another thing though: How do we deal with AS_IS vs ADD_COLOR? tell the script which to use? I figure you could let the script check whether the tile only contains white/greyscale or other colors too, but there's always the possibility that someone wants ADD_COLOR even with other colors or AS_IS with no colors (for example those animated tiles, or maybe a polar bear). Replacing all instances of AS_IS with ADD_COLOR is easy enough, but if you have mixed sets, it's gonna get annoying.

Pulling the color out of the raws and coloring the png automatically is problematic too, since it depends on the color scheme.

In the end, I think the most sensible thing to do is to just keep it as ADD_COLOR as default, and have people replace it with AS_IS where they need it. It's the less used alternative anyway.

(It would also theoretically make it possible to specify special cases, e.g. ADD_COLOR/AS_IS, by using specific RGB values for specific pixels in the border... but that's a more advanced concept anyway, so don't worry too much about that for now)

Since I'm so new to DF, I'm only familiarizing myself with the different sections so that's why I greatly appreciate your feedback.  Pidgeot, what you suggest is very easy to do I will give it a try. 

QUICK QUESTION: Can the ADD_COLOR and AS_IS entries have anything after them besides DEFAULT?


I don't know how many creatures can become trained.
P.S. Here is a list. (http://P.S. http://dwarffortresswiki.org/index.php/DF2014:Animal_trainer#Train_a_hunting_animal) I hope that helps.

Thank you!


Standard image organization is great for the repo, but the current problem that exists in the repo are with the definition text files. Organizing the images in each set to match a standard means nothing without updated working definitions. The challenge is the definition text file that syncs with the images you are creating.

Anyway, you get the idea.

I do get the idea and I'd love to see you or someone pull this off, but all of my questions still remain unanswered.

Seeing that the idea has traction, what comes next is generating the text files.  I honestly think it shouldn't be too difficult.  Let's see how it goes.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 31, 2016, 12:31:10 pm
QUICK QUESTION: Can the ADD_COLOR and AS_IS entries have anything after them besides DEFAULT?

Yes, the texture tokens (i.e. the various rows in your example image: ANIMATED, GHOST, ADVENTURER, etc).
http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#List_of_Professions.2C_Creatures_and_States

As for the border thing, I assume that's only for the example files or - if we follow pidgeots suggestion to use it for colorization indication - temporary, because those would be visible in game.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on January 31, 2016, 12:44:52 pm
I guess the flaw that I see in your current script is that it relies on preexisting definition text files. So if a HIPPO is defined twice or something is misspelled or GIANT_LION is defined as LION_GIANT or the myriad of other possibilities - how does that even work? Also, what's the point if you need a preexisting error free definition text file for the script to work in the first place, you know?

When you say it shouldn't be too difficult I feel as if you're not considering things like professions that exist in game that you can define tiles for that are not in the raws . . . how are you going to pull that off?

No pressure Quiet-Sun. ;]
Title: Re: Dwarf Fortress graphics repositories
Post by: Pidgeot on January 31, 2016, 12:51:10 pm
As for the border thing, I assume that's only for the example files or - if we follow pidgeots suggestion to use it for colorization indication - temporary, because those would be visible in game.

The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.

The border is useful because it makes the separation between each tile more clear - and it also just happens that we can exploit the fact that it's there anyway to carry hidden information. For example, we wouldn't necessarily need to store the tile size explicitly - the top left (or bottom right, etc.) pixel of the image could be colored as RGB(x_size, y_size, whatever) and we could just check that when building the raws to know what we were working with. The final component could be used to mark a file format version, or you could go to RGBA to specify how *many* tiles were in each direction, etc. It's really just a way you might capture the additional metadata that needs to be applied to the raws (of course, you need to figure out *what* that metadata might be, and document how that gets mapped between the images and the raws, but the basic idea is that if you just add a single image, you can just copy the border of a cell that works similarly - or make your graphical change, update the raws after applying it, and then regenerate the grid format.

*The reason for 17x17 instead of 18x18 is that you otherwise get the appearance of a 2 pixel border between cells, but only a single pixel border on the edges. It makes things seem a bit more ordered if the borders appear uniform.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 31, 2016, 02:22:30 pm
The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.

The border is useful because it makes the separation between each tile more clear - and it also just happens that we can exploit the fact that it's there anyway to carry hidden information. For example, we wouldn't necessarily need to store the tile size explicitly - the top left (or bottom right, etc.) pixel of the image could be colored as RGB(x_size, y_size, whatever) and we could just check that when building the raws to know what we were working with. The final component could be used to mark a file format version, or you could go to RGBA to specify how *many* tiles were in each direction, etc. It's really just a way you might capture the additional metadata that needs to be applied to the raws (of course, you need to figure out *what* that metadata might be, and document how that gets mapped between the images and the raws, but the basic idea is that if you just add a single image, you can just copy the border of a cell that works similarly - or make your graphical change, update the raws after applying it, and then regenerate the grid format.

*The reason for 17x17 instead of 18x18 is that you otherwise get the appearance of a 2 pixel border between cells, but only a single pixel border on the edges. It makes things seem a bit more ordered if the borders appear uniform.

I was thinking that it would be best to keep the tile size the same.  The reason is that we have the walls and other items that need to be continuous.  We could take the outer pixels from the creature tile, but I have the impression that some artists use the entire tile.   What I was thinking as an alternative is to use only the corners of the tile.  4corners*4channels(RGBA)*255 gives us plenty of room to encode information.  What do you think?



I guess the flaw that I see in your current script is that it relies on preexisting definition text files. So if a HIPPO is defined twice or something is misspelled or GIANT_LION is defined as LION_GIANT or the myriad of other possibilities - how does that even work? Also, what's the point if you need a preexisting error free definition text file for the script to work in the first place, you know?

When you say it shouldn't be too difficult I feel as if you're not considering things like professions that exist in game that you can define tiles for that are not in the raws . . . how are you going to pull that off?

No pressure Quiet-Sun. ;]

I should correct myself.  It shouldn't be too difficult to do it for creatures.  Humanoids are a different beast, but I'm not going to worry about them now or I won't make any progress.   As you have said before this is not an easy task, but one that I consider tractable.  I know I'm not going to be able to pull it off in the first try, but that's not a big deal for me.

One thing that you should keep in mind is that we are both defining a standard AND standardizing existing disparaging sets.  At some point or other, something will have to be fixed by a human, but I am convinced that we can do most of the work automatically.   

What I need right now is proactive help and not sarcasm.  Identifying problems is useful and necessary, but your input would be immensely more valuable if you also try to help us find solutions.  Your help (and by you I mean everyone that understands how DF works) is critical for my success.

HERE ARE SOME THINGS THAT I NEED:

- Does anyone know where in DF's /objects/*.txt files are all the professions (or other variables that may affect tile appearance)?  I want my standardizing scripts for humanoids to be solely based on the files found in DF so that we can update things easily as the game evolve. 

- I see that some races have tons of professions (dwarves), but some seem to have a handful (worm-men).  Where in DF's /objects/*.txt can I find such a difference defined?  Is it just those that are creatures?  How many professions am I dealing with?
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 31, 2016, 02:31:39 pm
I guess the flaw that I see in your current script is that it relies on preexisting definition text files. So if a HIPPO is defined twice or something is misspelled or GIANT_LION is defined as LION_GIANT or the myriad of other possibilities - how does that even work? Also, what's the point if you need a preexisting error free definition text file for the script to work in the first place, you know?

The way I understood Quiet-Sun's efforts, there would be two things the script does eventually:


So the script would take existing creature graphics and their existing creature definition files and generate a dataset. Tiles stored separately with their definition taken from the text file.
You would get a set of tiles with a known meaning (tile 1 is creature X, tile 2 is creature Y, tile 3 is creature Z:profession A, tile 4 is creature Z:profession B, etc).

On the other end, the script would take the raws and - based on hardcoded rules - generate another dataset from it (number of creatures and their names, number of texture tokens/professions and their names, resulting dimensions of image files).
You would get a set of creatures with a known coordinate (creature X sits at 2:5, creature Y at 0:0, etc)

Finally, it would create standard creature definition text files and the new, reordered image files.


The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.
The border is useful because it makes the separation between each tile more clear -
More clear for whom? It's trivial for a computer to just count the 16 pixels, it doesn't need to 'see'. And eventually the whole process from start to finish would be automated, so nobody else would need to distinguish the tiles either.

Quote
metadata embedding
I don't see the reasoning with that. Essentially, it seems that between the first input and the final output, you suggest to store all the information in the image - I'm wondering why not store all the information in a plain text database instead? That seems easier, more compatible and more transparent.
Taking the existing tilesets, separating them, adding a border with embedded data, then taking that and removing it again to use in DF seems a bit roundabout. Why not transfer the metadata directly in the final definition file?


EDIT: Just realized, the "datasets" I'm talking about, are the creature definition files.
Title: Re: Dwarf Fortress graphics repositories
Post by: Pidgeot on January 31, 2016, 03:13:52 pm
I was thinking that it would be best to keep the tile size the same.  The reason is that we have the walls and other items that need to be continuous.  We could take the outer pixels from the creature tile, but I have the impression that some artists use the entire tile.   What I was thinking as an alternative is to use only the corners of the tile.  4corners*4channels(RGBA)*255 gives us plenty of room to encode information.  What do you think?

I believe it is a bad idea because it becomes impossible to use those corners for actual graphics (how would you know what is supposed to be there?) - which is already going to cause issues with some of the examples you've already shown (e.g. reindeer in Spacefox).

Even if you cheated and said you only reserved the alpha channel for this information... that's still going to make the graphics visibly different, which is going to make it difficult to properly imagine how it's going to look.

I imagine that very few people will want to manually add the metadata we're talking about - they'd much rather draw the graphics, generate some raws, modify those, then run the process in reverse. They have the *option*, sure, since it's all just pixels with defined colors, but it's a bit abstract and not something I imagine most people are up for (except perhaps if they're just modifying an existing sprite for a one-off, in which case they can copy the border as well).

For the case where things have to join up... those also have to be able to join up with *themselves*, and you can't really check that anyway in a grid format like this.

The idea is that if you have, say, a 16x16 set of tiles, the generated image from all of this uses a grid of 17x17* - 1 pixel in each dimension is used for the border, and then the remaining 16 pixels are the actual tile graphics.

If the 16x16 tile graphics are completely blank - i.e., all pixels transparent - then we ignore that tile, because it's not specified, otherwise we cut out that 16x16 tile, assemble them all into whatever image layout the game wants.
The border is useful because it makes the separation between each tile more clear -
More clear for whom? It's trivial for a computer to just count the 16 pixels, it doesn't need to 'see'. And eventually the whole process from start to finish would be automated, so nobody else would need to distinguish the tiles either.

For a person who is using the grid to figure out what they need to create. It's just as trivial for the computer to count 17 (or 18) pixels, but it's much easier for a human to make sure they're "staying within the lines" if they have lines to follow for those cases where you're doing one-off edits; otherwise it might be difficult to see you're accidentally going one pixel into another tile (which just happened to be a transparent area, so you couldn't tell easily).

It's also a tiny bit easier for the computer when it needs to figure out whether to skip a given tile (because you're looking for a tile where all pixels are transparent, as opposed to pixels where some are and some aren't).

Quote
Quote
metadata embedding
I don't see the reasoning with that. Essentially, it seems that between the first input and the final output, you suggest to store all the information in the image - I'm wondering why not store all the information in a plain text database instead? That seems easier, more compatible and more transparent.
Taking the existing tilesets, separating them, adding a border with embedded data, then taking that and removing it again to use in DF seems a bit roundabout. Why not transfer the metadata directly in the final definition file?


EDIT: Just realized, the "datasets" I'm talking about, are the creature definition files.

You can certainly use a secondary file for the metadata if you'd prefer, and it'd probably be more efficient since you'd potentially be able to avoid stitching the image together - but it's my impression that for graphics sets, the amount of customization needed is going to be quite limited post-standardization anyway. You have to analyze the image anyway to tell whether or not a given tile is present (and therefore, whether it should be pointed to in the generated raws)... so it's not much extra effort to analyze a couple extra pixels to get the metadata you need at the same time.

You could also just choose to leave all metadata in the RAWs and only use this to unify the positions for a given creature. Depends what you want to be able to do, of course... but if that's your entire goal, then it seems overkill to add all of the grid labels.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on January 31, 2016, 03:42:47 pm
I guess the flaw that I see in your current script is that it relies on preexisting definition text files. So if a HIPPO is defined twice or something is misspelled or GIANT_LION is defined as LION_GIANT or the myriad of other possibilities - how does that even work? Also, what's the point if you need a preexisting error free definition text file for the script to work in the first place, you know?

When you say it shouldn't be too difficult I feel as if you're not considering things like professions that exist in game that you can define tiles for that are not in the raws . . . how are you going to pull that off?

No pressure Quiet-Sun. ;]

I should correct myself.  It shouldn't be too difficult to do it for creatures.  Humanoids are a different beast, but I'm not going to worry about them now or I won't make any progress.   As you have said before this is not an easy task, but one that I consider tractable.  I know I'm not going to be able to pull it off in the first try, but that's not a big deal for me.

One thing that you should keep in mind is that we are both defining a standard AND standardizing existing disparaging sets.  At some point or other, something will have to be fixed by a human, but I am convinced that we can do most of the work automatically.   

What I need right now is proactive help and not sarcasm.  Identifying problems is useful and necessary, but your input would be immensely more valuable if you also try to help us find solutions.  Your help (and by you I mean everyone that understands how DF works) is critical for my success.

Quiet-Sun, I was not being sarcastic. I sincerely don't see the point if you need a preexisting error free definition text file.

You're right. Solutions are more helpful than just the problems. But, I do believe sharing the problems I am identifying is worth it so that solutions are sought rather then the problem overlooked. I have no idea at this point what the solution(s) would be. I am sharing it via this thread so that you and others are made aware of what needs to be addressed. Sincerely, I would love for you or someone to pull this off as previously stated. To be absolutely clear, I am rooting for you!


HERE ARE SOME THINGS THAT I NEED:

- Does anyone know where in DF's /objects/*.txt files are all the professions (or other variables that may affect tile appearance)?  I want my standardizing scripts for humanoids to be solely based on the files found in DF so that we can update things easily as the game evolve. 

No, they are not all the professions. That was one of the things I was concerned about since there are a lot of professions that are not in ANY of the /objects/*.txt files, hence my sincere question of "how are you going to pull that off?"

I do not see a way that you can cover all professions based on the /objects/*.txt files found in DF.

- I see that some races have tons of professions (dwarves), but some seem to have a handful (worm-men).  Where in DF's /objects/*.txt can I find such a difference defined?  Is it just those that are creatures?  How many professions am I dealing with?

I have a complete list I can send you, but it has everything in it - meaning, not just currently used professions, but outdated professions (in case they are implemented again), professions that are mere guesses, and professions I have not been been able to successfully verify, but they are referenced on the wiki and other definition files consistently.


On the other end, the script would take the raws and - based on hardcoded rules - generate another dataset from it (number of creatures and their names, number of texture tokens/professions and their names, resulting dimensions of image files).
You would get a set of creatures with a known coordinate (creature X sits at 2:5, creature Y at 0:0, etc)

Finally, it would create standard creature definition text files and the new, reordered image files.

Thanks CLA, but the above part is where I am lost. So instead of a preexisting definition text you would have to set up the tiles in an image to match the predefined defintion text? Also, do you mean hardcoded rules would be the solution to the professions not found in the any of the raw files? In other words, the script hardcodes all the professions somehow?
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 31, 2016, 04:30:10 pm
More clear for whom? It's trivial for a computer to just count the 16 pixels, it doesn't need to 'see'. And eventually the whole process from start to finish would be automated, so nobody else would need to distinguish the tiles either.
For a person who is using the grid to figure out what they need to create.
Oh I see now what you mean.
Photoshop (and I assume Krita or GIMP) have features to display a grid of specified size, which solves that problem elegantly. If we want to embed information in the file, I would only do it on empty tiles, and checkerboard those with two different colors. In addition, add the tile coordinate on the empty tile.

So instead of a preexisting definition text you would have to set up the tiles in an image to match the predefined defintion text?
Yes. For existing graphics, this would happen automatically based on the old definition text. For entirely new graphics, you'd have to learn the coordinates from the standard definition file.
Right now, every graphic set artist has to draw all the tiles in whatever organization they deem best, and then type out all the creature definitions manually.
With standardized creature definitions, you have to type out nothing and just draw. Downside is that you have to draw specific creatures on specific tiles. I'd still consider that a major improvement though.

This is, how I understand it, the main advantage of this whole endeavor: reduced workload for artists because the definition files are autogenerated. It can only be automated if it's standardized. And if we can take all the information we have from the raws, we never (or rarely) need to update the script.

As for professions, with "hardcoded rules" you quoted I was simply referring to the actual raw parser code. I.e. 'look for string "[CREATURE:<anything>]" and discard everything before and after' - that kind of stuff.
Regardless, you're right: If there's no way to get a correct and complete list from either the raws or DFHack trickery, I see no other solution than to hardcode it (In a separate file, ideally).


For reference, this is the ideal workflow I hope to see for graphic set authors once we have a working setup:
Spoiler (click to show/hide)

Current problems I see:
Do we have to hardcode professions?
What do we do with empty tiles? When the script can differentiate between an empty tile and a non-empty tile, it should be trivial to comment out that line. That seems to be the easiest solution.
We could instead fill the empty tiles with their default letter of the default tileset.
This would reduce the difference in the definition files, but as every graphic set needs its own definition file anyway (AS_IS vs ADD_COLOR), it would just complicate things, I think.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 31, 2016, 06:19:33 pm
I have a complete list I can send you, but it has everything in it - meaning, not just currently used professions, but outdated professions (in case they are implemented again), professions that are mere guesses, and professions I have not been been able to successfully verify, but they are referenced on the wiki and other definition files consistently.

Thank you Burned! this would be very helpful.   Can you please send me one without outdated professions?  We can assume your list is complete and improve it if need be.



For reference, this is the ideal workflow I hope to see for graphic set authors once we have a working setup:
Spoiler (click to show/hide)

Do we have to hardcode professions?
What do we do with empty tiles? When the script can differentiate between an empty tile and a non-empty tile, it should be trivial to comment out that line. That seems to be the easiest solution.
We could instead fill the empty tiles with their default letter of the default tileset.
This would reduce the difference in the definition files, but as every graphic set needs its own definition file anyway (AS_IS vs ADD_COLOR), it would just complicate things, I think.

That's exactly it CLA!!  That is exactly what I would like to implement.  I think I have a solution for all the problems you mention:


Do we have to hardcode professions?

I'm going to create three separate reference texture/profession files: one for creatures, one for beastmen, and one for major races (see Burned comment above).  We can then expand them or trim them as our knowledge improves (and/or DF changes).  These will determine the entries on the y axis of the matrix (what you reference as A through Z).   The creatures will come directly from the raws.  These will determine the entries of the x axis of the matrix (what you reference as 1 throuh N).


What do we do with empty tiles? When the script can differentiate between an empty tile and a non-empty tile, it should be trivial to comment out that line. That seems to be the easiest solution.

We can try both approaches.  They should not be too difficult to implement and see what works best.  Alternatively we could use a default creature template like Mayday for all empty creatures.


This would reduce the difference in the definition files, but as every graphic set needs its own definition file anyway (AS_IS vs ADD_COLOR), it would just complicate things, I think.

Actually I think I have the solution fro the AS_IS vs ADD_COLOR.  Looking at CLA's set I noticed that all his tiles are pure white.   We can use ADD_COLOR if the tile has only white, or AS_IS if not (we can make the rule more complicated if necessary, for example using histograms of the tiles).
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on January 31, 2016, 08:20:39 pm
Ok Guys!  I have the standard text files for creatures now.  You can see one of them here:

Spoiler (click to show/hide)

Here is its corresponding image:

(https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/creature_amphibians_CLA.png)

Both were generated automatically.   A quick cross-check will show you that they match.


All the rest can be downloaded here:

https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/Creature_TXTs.zip (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/Creature_TXTs.zip)

You will notice that for the moment the txt file has the "AS_IS" tag, also that the tile sizes are mising.   Now my next step is to open this files and modify them (with the correct AS_IS or ADD_COLOR), as I create the standard images, and we will be ready for our first ingame experiment!
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on January 31, 2016, 09:02:23 pm
Actually I think I have the solution fro the AS_IS vs ADD_COLOR.  Looking at CLA's set I noticed that all his tiles are pure white.   We can use ADD_COLOR if the tile has only white, or AS_IS if not (we can make the rule more complicated if necessary, for example using histograms of the tiles).
See my earlier comment:
Another thing though: How do we deal with AS_IS vs ADD_COLOR? tell the script which to use? I figure you could let the script check whether the tile only contains white/greyscale or other colors too, but there's always the possibility that someone wants ADD_COLOR even with other colors or AS_IS with no colors (for example those animated tiles, or maybe a polar bear). Replacing all instances of AS_IS with ADD_COLOR is easy enough, but if you have mixed sets, it's gonna get annoying.

Pulling the color out of the raws and coloring the png automatically is problematic too, since it depends on the color scheme.


In the end, I think the most sensible thing to do is to just keep it as ADD_COLOR as default, and have people replace it with AS_IS where they need it. It's the less used alternative anyway.
*note that I meant "keep AS_IS as default", of course.

I really think that graphic sets with ADD_COLOR are a very rare exception, and letting the script guess introduces another layer of complexity with plenty of possible issues whereas just using a text editor and replacing all instances of one for another is trivial for the few artists that want to use ADD_COLOR.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 01, 2016, 04:17:40 am
Ok Guys,

Here is my first attempt at creating a functional standard graphics set.  As expected, it didn't work  :'(  The worse thing is that I'm not sure why.  Could you please look at these sets to see if there are obvious problems to you?

In this file I'm including the CLA and Spacefox sets.   I chose them because one is 16x AS_IS, and the other is 18x ADD_COLOR.  (not to mention that CLA is following this thread so hopefully he/she can find the problems easily)

https://dl.dropboxusercontent.com/u/408446/DF/Graphics%20Standardization_1st_Attempt.zip (https://dl.dropboxusercontent.com/u/408446/DF/Graphics%20Standardization_1st_Attempt.zip)

If you have some time to check them for problems it would be great!  Thanks a lot!

I haven't touched the /raw/objects/ folders at all, only the /raw/graphics/  Could the problem be there?  other possible culprits are reversing x,y tile coordinates, having typos, messing up the headers  :-[

Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 07:04:19 am
Could you please look at these sets to see if there are obvious problems to you?

Based on the structure and text files I can tell you what I see offhand.

1. The definition text file is named creature_whatever_graphics.txt, but it's defined as creature_whatever.
2. The old definition files are present which means if you correct 1. you will then have everything from Adder to Yak defined twice, each calling different image files since both old and new images are present.
3. If you get rid of the old definition files and old images, you have defined everything that is not in the set as a red tile (This includes the wagon since it's techinically a creature).



Thank you Burned! this would be very helpful. Can you please send me one without outdated professions?  We can assume your list is complete and improve it if need be.


The complete list that I've cobbled together is available right now if you want it (see below spoiler), but they are organized by professions not by used, outdated, etc. I did make some quick notes for your reference if you want to organize them differently.


Notes on notes:


"Not sure" means I feel I've seen them in game, but can't recall at the moment.
"Never verified" means I've actively tried to verify these and have never seen them in game.
I believe CHIEF_MEDICAL_DWARF works for any race, but not all professions listed work for every race.
I've noted the ones that I know are specific to a particular race, but I know there are more than I noted.
Some are defined as one thing, but appear in game as another (see elf queen and princess for example).
Law_Enforce is the fortress guard. Tax_Escort was either turned off with the tax collector or is used as the "Royal Guard" - I'm not sure and it's been awhile since I've had a king arrive.

Verification for creatures is easy, since you can spawn them in the arena. For professions you have to play adventurer or dwarf fortress to verify the tiles.

Wait . . . I'm an idiot.

*sends email*

Maybe Toady could just add professions inside the arena?

In any case, the list is below . . .


Spoiler (click to show/hide)





Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 01, 2016, 11:36:01 am

Based on the structure and text files I can tell you what I see offhand.

1. The definition text file is named creature_whatever_graphics.txt, but it's defined as creature_whatever.
2. The old definition files are present which means if you correct 1. you will then have everything from Adder to Yak defined twice, each calling different image files since both old and new images are present.
3. If you get rid of the old definition files and old images, you have defined everything that is not in the set as a red tile (This includes the wagon since it's techinically a creature).


Thank you Burned!  I found the problem!  Graphics deffinition txts need to start with the string "graphics_"  so instead of "creature_suberranean_graphics.txt",  I had to use "graphics_creature_subterranean.txt"  ::)

You see some of the old files, but that is because I need to keep beastmen and major races.  I made absolutely sure there were no duplicate entries.   Now I have a bug where dogs look like cows, but that is much easier to fix.  I'll upload the correct files later today, but I'm stoked to see it work!  I think we will have the standard ready very soon!
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 12:24:30 pm
Thank you Burned!  I found the problem!  Graphics deffinition txts need to start with the string "graphics_"  so instead of "creature_suberranean_graphics.txt",  I had to use "graphics_creature_subterranean.txt"  ::)

You're right. I failed to catch that.


You see some of the old files, but that is because I need to keep beastmen and major races.  I made absolutely sure there were no duplicate entries.   

I see. I originally didn't understand why they were mixed; I made the mistake of thinking all the old files were present.
Why not test them without the old files to avoid errors that are not coming from your script?



Now I have a bug where dogs look like cows, but that is much easier to fix. I'll upload the correct files later today, but I'm stoked to see it work!  I think we will have the standard ready very soon!

I don't see the dog/cow error, but it's possible you mean in the files post first attempt.

I look forward to your progress.

Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 01, 2016, 01:56:16 pm
Actually I think I have the solution fro the AS_IS vs ADD_COLOR.  Looking at CLA's set I noticed that all his tiles are pure white.   We can use ADD_COLOR if the tile has only white, or AS_IS if not (we can make the rule more complicated if necessary, for example using histograms of the tiles).
A lot of ADD_COLOR graphics are grayscale rather than pure white.  I know that my Rally Ho tileset uses grayscale ADD_COLOR for the DEFAULT tile, but the rest of the the tiles are colored AS_IS

Horizontally mirroring tiles may have unintended effect for ASCII tilesets - turning b into d, p into q, etc.  This will probably be something that just needs those tiles changed by the artist, but it's something to be aware of during testing so you don't mistake it for the game grabbing the wrong tile.

STANDARD <<< similar to default. I forget which one takes precedence or why there are two defaults.
STANDARD is used when a creature has no profession.  These are the peasants, haulers, etc.
DEFAULT is used when a creature's profession doesn't have a graphic defined.

Something else to note is that a lot of tile definitions have changed rather recently and some older tilesets will use old methods which, ideally, your script should catch and correct.
For example:
[DEFAULT:MyCreature:0:0:AS_IS:ANIMATED] should now be [ANIMATED:MyCreature:0:0:AS_IS:DEFAULT]
[CREATURE:TIGER_MAN] should be [CREATURE:TIGERMAN]
I'm pretty sure Phoebus still defines zombies the old way, and any tile set that hasn't updated in the last month will use the wrong name for tiger men.  For a more complete list for the latter, I'd need to diff the raws for each version
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 01, 2016, 03:15:39 pm
Ok Guys!  I think I have the first functional graphics sets with all creatures standardized.  Since I have three artists reading this thread, I figured I would start with theirs (I used the 42.05 versions).   I also did Spacefox 40.24, in case someone can help me look for errors.    Here are the files, please let me know of any problems (note that red tiles are supposed to be there to indicate missing tiles):

42.05:
-  CLA  (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/42.05_CLA%20-%20Standard.zip)
-  Rally-Ho (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/42.05-Rally%20Ho%20-%20Standard.zip)

40.24:
-  Spacefox (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/40.24-Spacefox%20-%20Standard.zip)



A lot of ADD_COLOR graphics are grayscale rather than pure white.  I know that my Rally Ho tileset uses grayscale ADD_COLOR for the DEFAULT tile, but the rest of the the tiles are colored AS_IS

Horizontally mirroring tiles may have unintended effect for ASCII tilesets - turning b into d, p into q, etc.  This will probably be something that just needs those tiles changed by the artist, but it's something to be aware of during testing so you don't mistake it for the game grabbing the wrong tile.

Thank you and welcome Rydel!  I will then start with an AS_IS and cross check with each artist's txt files to adopt ADD_COLOR as needed.  For something like CLA's set, then it's no problem to do a mass-substitution as he/she suggests.   Regarding mirroring, I intended to verify with each artist (if active) and make a choice if not.   Certainly for ASCII we shouldn't do it.



Something else to note is that a lot of tile definitions have changed rather recently and some older tilesets will use old methods which, ideally, your script will catch and correct.
For example:
[DEFAULT:MyCreature:0:0:AS_IS:ANIMATED] should now be [ANIMATED:MyCreature:0:0:AS_IS:DEFAULT]
[CREATURE:TIGER_MAN] should be [CREATURE:TIGERMAN]
I'm pretty sure Phoebus still defines zombies the old way, and any tile set that hasn't updated in the last month will use the wrong name for tiger men.  For a more complete list for the latter, I'd need to diff the raws for each version

Thank you Rydel. I'm already using an exception file to identify discrepancies between the raws and the graphic set names.  I cross checked the raws with the  Graphics Wiki (http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#Creatures), and created an exception file that you can download here:

NAME_EXEPTIONS.txt (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/NAME_EXEPTIONS.txt)

@Rydel and @burned, you may be interested in knowing that you seem to be misspelling the giant cats (lion, tiger, etc).  At least my script seemed to isolate this.  Check the exeptions in my file and test them in the arena just in case.

Now onwards to the beastmen!  8)
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 01, 2016, 03:30:20 pm
STANDARD is used when a creature has no profession.  These are the peasants, haulers, etc.
DEFAULT is used when a creature's profession doesn't have a graphic defined.
[...]
For a more complete list for the latter, I'd need to diff the raws for each version
Oh FINALLY, I understand the difference between standard and default. Thank you so much. Added to the graphic set page.

As for changed creature names, I already diff'd the raws and all the changes are on the Graphic set talk page (http://dwarffortresswiki.org/index.php/DF2014_Talk:Graphic_set). I haven't added them to the page itself though.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 03:50:44 pm
STANDARD <<< similar to default. I forget which one takes precedence or why there are two defaults.
STANDARD is used when a creature has no profession.  These are the peasants, haulers, etc.
DEFAULT is used when a creature's profession doesn't have a graphic defined.
Thanks so much Rydel! I could never figure that out.

Something else to note is that a lot of tile definitions have changed rather recently and some older tilesets will use old methods which, ideally, your script will catch and correct.

Yes and I would argue that's what inspired this thread to exist. Heh.

Currently, I'm not sure how the script will catch a name change between versions since it would just think there is no tile created for it so to speak. I still see that aspect as a manual process. I'd love to be wrong.

[DEFAULT:MyCreature:0:0:AS_IS:ANIMATED] should now be [ANIMATED:MyCreature:0:0:AS_IS:DEFAULT]

My understanding is that it's always been [ANIMATED:MyCreature:0:0:AS_IS:DEFAULT]

Previously it was [SKELETON:MyCreature:0:0:AS_IS:DEFAULT] and [ZOMBIE:MyCreature:0:0:AS_IS:DEFAULT] and you could have all the weapon skills specified (e.g. SWORDSMAN:MyCreature:0:0:AS_IS:SKELETON, etc) When the switch was made, I too thought ANIMATED should go at the end - but are you saying it did at one point? What version does that actually work in and do you have any insight as to why it changed?

Since I have three artists reading this thread, I figured I would start with theirs (I used the 42.05 versions).
<snip>
-  Burned (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/42.05-Burnedfx%20-%20Standard.zip)
Could you remove this from dropbox please?

My set is not part of the LNP or the graphics repository. I have previously spoken to fricy about this (https://github.com/DFgraphics/DFgraphics/commit/2f5927fdee339f37a70939beebf951e60c323546). All the sets that are currently part of the repo are listed in the OP
. Thanks Quiet-Sun.


I'm already using an exception file to identify discrepancies between the raws and the graphic set names.  I cross checked the raws with the  Graphics Wiki (http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#Creatures), and created an exception file that you can download here:

NAME_EXEPTIONS.txt (https://dl.dropboxusercontent.com/u/408446/DF/Standard%202.0/NAME_EXEPTIONS.txt)

@Rydel and @burned, you may be interested in knowing that you seem to be misspelling the giant cats (lion, tiger, etc).  At least my script seemed to isolate this.  Check the exeptions in my file and test them in the arena just in case.


If you created the exception file based on the wiki list, the reverse is true. The discussion makes note of this (http://dwarffortresswiki.org/index.php/DF2014_Talk:Graphic_set), but these are the name changes Rydel mentioned.  You're script is not catching them in CLA or Spacefox, because they haven't been updated as of this post, whereas Rydel and myself have updated our definitions.




edit: Thanks again.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 03:53:15 pm
Ninja'd by CLA, regarding the name changes . . .

Oh FINALLY, I understand the difference between standard and default. Thank you so much. Added to the graphic set page.

I know, right!? Haha.

edit: Almost forgot . . .

(note that red tiles are supposed to be there to indicate missing tiles)

I know, but you're also defining the red tiles. That's not going to work.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 01, 2016, 04:08:18 pm
My understanding is that it's always been [ANIMATED:MyCreature:0:0:AS_IS:DEFAULT]

Previously it was [SKELETON:MyCreature:0:0:AS_IS:DEFAULT] and [ZOMBIE:MyCreature:0:0:AS_IS:DEFAULT] and you could have all the wepon skills specificed (e.g. SWORDSMAN:MyCreature:0:0:AS_IS:SKELETON, etc) When the switch was made, I too thought ANIMATED should go at the end - but are you saying it did at one point? What version does that actually work in and do you have any insight as to why it changed?

It is possible that I'm wrong on this point - I mostly gleaned this knowledge from seeing how there were written in some older tilesets, and I can't locate them, so I'm probably remembering it wrong, especially combined with the change from SKELETON/ZOMBIE to ANIMATED, and the weirdness of how the ANIMATED and GHOST texture tokens work as professions instead of textures could have caused the original files to do it wrong at some point, and I might have just checked before it was fixed.  Alternately, the change was very old and I can't find it from here.

I'll check old stuff when I get home.  If I don't edit this post, just assume that statement was me being wrong.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 04:26:04 pm
@Rydel No worries. I sincerely thought you had hidden insight, since you knew the difference between default and standard. Offhand,  do you see anything missing or incorrect on the wiki regarding graphic sets?

Regardless, thanks for your help.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 01, 2016, 04:30:23 pm
Everything looks good, though the Ghost and Animated tags might be less confusing if they were listed as professions instead of texture tokens, (or at least included in the profession list with a footnote) since that's where you put them.

Also, I noticed that there was a blank column in creature_amphibians in the graphics posted earlier.  Either the script is doing something odd, or a creature is missing.  I can check which in about an hour and post it.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 01, 2016, 05:24:56 pm

Something else to note is that a lot of tile definitions have changed rather recently and some older tilesets will use old methods which, ideally, your script will catch and correct.

Yes and I would argue that's what inspired this thread to exist. Heh.

Currently, I'm not sure how the script will catch a name change between versions since it would just think there is no tile created for it so to speak. I still see that aspect as a manual process. I'd love to be wrong.

If you created the exception file based on the wiki list, the reverse is true. The discussion makes note of this (http://dwarffortresswiki.org/index.php/DF2014_Talk:Graphic_set), but these are the name changes Rydel mentioned. You're script is not catching them in CLA or Spacefox, because they haven't been updated as of this post, whereas Rydel and myself have updated our definitions.

Ah! I see! I assumed the wiki was up to date.   So let me make sure I understand something.  The ultimate source of creature tags are the raws, every creature in the game can be found there, and their names are the correct ones.  Right?

Regarding creatures that change from one version to the next, for now let's do this manually.  When I'm done with the standardization, I can write a script that compares every creature in the raws of a given version with every creature in the raws another (ignoring creature names) and look for matches in their properties (i.e. DESCRIPTION, CREATURE_TILE, PETVALUE, etc.)  Then the script puts possible name changes in a conversion.txt file that a human can check and prune (but most of the work will be already done). 


Also, I noticed that there was a blank column in creature_amphibians in the graphics posted earlier.  Either the script is doing something odd, or a creature is missing.  I can check which in about an hour and post it.

Don't worry about it for the moment, it's caused by me assuming the wiki was up-todate and your files were not, while in reality it is the other way.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 01, 2016, 05:48:02 pm
Everything looks good, though the Ghost and Animated tags might be less confusing if they were listed as professions instead of texture tokens, (or at least included in the profession list with a footnote) since that's where you put them.
Yeah, I thought about doing that just now when I added the info about default and standard. I'll just go ahead and do it.

EDIT:

Don't worry about it for the moment, it's caused by me assuming the wiki was up-todate and your files were not, while in reality it is the other way.
On that note, the wiki wasn't updated yet because I haven't found time to fix my vim substitute command I used to generate the creature table in the first place. If you're willing and think that it requires only little more work than you're doing for the standardization, I would appreciate having a script that takes the raws and generates a table for the graphic set page. The syntax looks like this (variables in '<>', comments in '####'):
Spoiler (click to show/hide)
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 05:48:16 pm
So let me make sure I understand something.  The ultimate source of creature tags are the raws, every creature in the game can be found there, and their names are the correct ones.  Right?

Correct.

Regarding creatures that change from one version to the next, for now let's do this manually.

I agree with what CLA stated previously, standardization in the repo is a good step. Unfortunately, the reason I was even made aware of this was because of the trail of bread crumbs that pointed to the repo not being manually updated in the first place - they were just being "updated" without the definitions being touched for I don't know how many versions back.

I don't expect you to solve this problem (I definitely have hope and I am willing to help where I can!), but in order to address the current problem in the repo - assigining volunteers to specific sets to manually update them would be prudent until a better system exists. I am mainly speaking of the Frankenstein sets. Obviously the single author sets (like CLA) should be aware that they need to update.



Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 01, 2016, 05:57:17 pm
So let me make sure I understand something.  The ultimate source of creature tags are the raws, every creature in the game can be found there, and their names are the correct ones.  Right?

Correct.

Regarding creatures that change from one version to the next, for now let's do this manually.

I agree with what CLA stated previously, standardization in the repo is a good step. Unfortunately, the reason I was even made aware of this was because of the trail of bread crumbs that pointed to the repo not being manually updated in the first place - they were just being "updated" without the definitions being touched for I don't know how many versions back.

I don't expect you to solve this problem (I definitely have hope and I am willing to help where I can!), but in order to address the current problem in the repo - assigining volunteers to specific sets to manually update them would be prudent until a better system exists. I am mainly speaking of the Frankenstein sets. Obviously the single author sets (like CLA) should be aware that they need to update.

Thank you for your answers.  Then for the moment I will use CLA and Rally-Ho to define the standards since I know they are completely up to date.   Later, after I really understand how things work, I will see if there is a better way of dealing with frankensets.

One quick question:

- Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on February 01, 2016, 06:13:36 pm
Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?

Beastfolk and gorlaks can have any profession available to the civilization(s) they latch on to - which can be any civilization. Technically, so can elves, dwarves, humans, goblins, etc.

Plump Helmet Men can have any profession which doesn't require talking. (Not sure if worldgen respects this.)
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 06:14:14 pm
- Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?

This is the best answer I've read in regards to what you are asking . . .   Correction. See Button above.

. . . the subterranean animal people professions are restricted like the kobolds in the entity_default.txt
Does that change if they are citizens of your fort?

I can't say for sure, but I know that elven woodcutters are possible if they use your civ's ethics, so presumably all jobs available to the civ they're attached to are available.

Since you can't spawn professions in the arena at this time, you have to test it in game. This involves crossing your fingers and wishing on a star that you get the tiles you are testing. Heh.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 01, 2016, 06:16:35 pm
- Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?

This is the best answer I've read in regards to what you are asking . . .

. . . the subterranean animal people professions are restricted like the kobolds in the entity_default.txt
Does that change if they are citizens of your fort?

I can't say for sure, but I know that elven woodcutters are possible if they use your civ's ethics, so presumably all jobs available to the civ they're attached to are available.
[/quote]

Since you can't spawn professions in the arena at this time, you have to test it in game. This involves crossing your fingers and wishing on a star that you get . Heh.
[/quote]

:D Thank you! I think I'll start by using whatever is on entity_default.txt

since my script crosses my files against those of the original set, then I will be able to see what is left after the automatic process
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 01, 2016, 06:18:04 pm
Then for the moment I will use CLA and Rally-Ho to define the standards since I know they are completely up to date
To clarify:

CLA creature graphics, as they are in the CLA graphic set thread, on Mediafire/dffd and presumably in fricy's repo are NOT up to date. They neither have definitions for the new creatures nor the corrected creature names for some old ones and there are consequently several creature graphics missing (new giant and men variants, Tigermen, etc).

The whole reason I didn't update them yet and the whole reason I am enthusiastic about this script is because I don't have time to do it manually and I have no way to automate it.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 01, 2016, 06:29:46 pm
Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?

Beastfolk and gorlaks can have any profession available to the civilization(s) they latch on to - which can be any civilization. Technically, so can elves, dwarves, humans, goblins, etc.

Plump Helmet Men can have any profession which doesn't require talking. (Not sure if worldgen respects this.)

So in Summary, any Dwarf, Elf, Human, Goblin, Kobold, Animal Person, and Gorlak can theoretically have any profession, and consequently a graphic assigned to it correct?
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 01, 2016, 06:52:57 pm
Then for the moment I will use CLA and Rally-Ho to define the standards since I know they are completely up to date
To clarify:

CLA creature graphics, as they are in the CLA graphic set thread, on Mediafire/dffd and presumably in fricy's repo are NOT up to date. They neither have definitions for the new creatures nor the corrected creature names for some old ones and there are consequently several creature graphics missing (new giant and men variants, Tigermen, etc).

The whole reason I didn't update them yet and the whole reason I am enthusiastic about this script is because I don't have time to do it manually and I have no way to automate it.

Ooops! then make that Rally-Ho for 42.05 and CLA for 40.24 



Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?

Beastfolk and gorlaks can have any profession available to the civilization(s) they latch on to - which can be any civilization. Technically, so can elves, dwarves, humans, goblins, etc.

Plump Helmet Men can have any profession which doesn't require talking. (Not sure if worldgen respects this.)

So in Summary, any Dwarf, Elf, Human, Goblin, Kobold, Animal Person, and Gorlak can theoretically have any profession, and consequently a graphic assigned to it correct?

I'm going to use that as my working assumption.  We can trim things in a later pass.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 01, 2016, 07:04:05 pm
Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?

Beastfolk and gorlaks can have any profession available to the civilization(s) they latch on to - which can be any civilization. Technically, so can elves, dwarves, humans, goblins, etc.

Plump Helmet Men can have any profession which doesn't require talking. (Not sure if worldgen respects this.)

So in Summary, any Dwarf, Elf, Human, Goblin, Kobold, Animal Person, and Gorlak can theoretically have any profession, and consequently a graphic assigned to it correct?


That's how I understand it, but I haven't actually been able to test this in game.

In the spoiler are the animal people I use. The adventurer mode tiles are easily verified. It's the fortress mode ones that allude me, since my visitors are normally the standard races (elf, goblin, human).

Spoiler (click to show/hide)

If anyone has a save that has these animal people available in their fort for testing, please share! I'd prefer a vanilla save, but as I've said elsewhere, I'll take what I can get to verify this.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 01, 2016, 07:28:07 pm
In regards to testing, maybe a modded DF with elves, humans, goblins and kobolds removed would increase the chances of animal person visitors.
Similarly, wasn't there a way you could give a creature a natural talent/predisposition for a skill in the raws? Would that automatically make them display a profession graphic?
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 02, 2016, 01:19:49 am
Ok Guys.  Today I spent my time planning an not coding.  Animalmen and major races are significantly harder to conceptualize.   Here is what I'm going to do.  Based on Burned's list and the work that Telltale is doing in Mayday's thread, I made a list organizing all professions into groups with a default at the top.   Major races will have a tile for all professions in the list and beastmen will have tiles only for the defaults (the ones without tabs).  The remaining professions will pointing to the default tile in the text files.  If someone ever gets around to fleshing out a beastmen race we can turn it into a major race.  Let me know if you have comments or suggestions (for example if you think that a particular profession should belong in a different category).  I'll get back to coding :)   

Spoiler (click to show/hide)
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 02, 2016, 07:52:22 am
In regards to testing, maybe a modded DF with elves, humans, goblins and kobolds removed would increase the chances of animal person visitors.
Similarly, wasn't there a way you could give a creature a natural talent/predisposition for a skill in the raws? Would that automatically make them display a profession graphic?
That's a good idea and probably worth a shot.
I don't have the answers to your last two questions.

@Quiet-Sun

My list is combination of verified professions that have been present since 2D DF, additional ones that came in various updates, the old wiki discussion pages before CLA took on the momentous task of organizing it all into actual pages for the wiki, looking at other definition files, and guesswork. I have left professions that I have not been able to verify (and noted them as such), but that are referenced in various places. There are also professions that previously existed that I leave in because Toady has put back removed professions in the past.

Like a lot of things in DF, there are exceptions and odd inconsistency (e.g. ANIMATED and GHOST are "professions"). Tiles can be defined by not just creature or profession, but by skill or assigned job. For example, a unit's skill level means nothing if you assign them to be a TAVERN_KEEPER or the CAPTAIN_OF_THE_GUARD. On the other hand, someone in your military that is a MASTER_SWORDSMAN doesn't care that you assigned him to the crossbow squad and never allowed him to have a sword as part of his uniform. Until another weapon skill exceed his sword skill - he will still be a MASTER_SWORDSMAN with a crossbow in his hand. Also, a general set of skills will provide you the generic status, rather than a specific one.

From CLA, Rydel, to my own definitions - we all organized them in a way that made sense to us. They are the same definitions, so they all work (aside from the ones CLA mentioned so you have a test case for your script). Your organization is also based on a way that made sense to you. The problem is inexperience in knowing the exceptions and inconsistency. So based on what I've shared so far, here is what I see wrong with your list.

In several cases you listed a general skill as the "default" correctly, although not considered DEFAULT in game. In other cases you've listed the wrong profession as the default. For instance, while the average skill levels in a variety of related skills would produce a FARMER and ENGINEER, COOK and MECHANIC are specific skills levels respectively to those generics. Also, nobles are specific titles - there is no generic (unless they are not define per Rydel's info) and they have nothing to do with skill levels.

I don't follow why DUKE is the default in your list.  KING, KING_CONSORT, QUEEN_CONSORT don't exist as far as I know. They are covered by MONARCH and MONARCH_CONSORT. Again, I don't recall why I left QUEEN in there, because the elf QUEEN is the LEADER profession much like the PRINCESS position is actually defined by GENERAL for elves and I don't even see LEADER on your list. Nor do I see WARLORD or WARRIOR. And, PRIEST is not the default to DRUID - they are two different things.

PRISONER, COOK, MECHANIC, DOCTOR, DUKE, SHERIFF, MERCHANT and a few others are not default tiles. They are specific to those jobs/professions. MERCHANTS are not the default for DIPLOMAT or OUTPOST_LIAISON or SHOPKEEPER. PRISONER is not the default to SLAVE - again, these are different things in game.

Military: Aside from the assigned leadership positions - military tiles are based on skill level, much like ADVENTURER. And, ADVENTURER seems absent from your list entirely. DEFAULT, LAW_ENFORCE, TAX_ESCORT are assignments, SWORDSMAN and MASTER_SWORDSMAN are skill levels that exist together. Meaning you can have one tile for the DEFAULT WRESTLER that is completely different than the tile that represent the DEFAULT WRESTLER that is also LAW_ENFORCE.

I am sure there are more things I missed, but Rydel already pointed out that ALL profession defaults are DEFAULT.

Awhile back you said . . .

Since I'm so new to DF, I'm only familiarizing myself with the different sections so that's why I greatly appreciate your feedback.   

Look, I don't want you to think I'm being sarcastic again or condescending or whatever because I am clarifying what I see to be a lot of misunderstanding on your end. I don't have all the answers, but I do see problems with your list. Also, I am still unsure about a few things. Like, did you understand why the red tiles are a problem(?) or did you understand what CLA was asking in regards to writing a script for the graphic set page(?) - something I believe would have great value.

I hope this information is helpful.

As a random suggestion, treating the animal people like the standard races wouldn't hurt, since defining things that may or may not exist won't cause errors in DF.



Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 02, 2016, 09:12:41 am
As Burned mentioned, a lot of those "defaults" aren't actual defaults.  However, it would be possible to fake it when your script generates the text files for the graphics.

Take priests for instance:  If you wanted a race to use the same image for PRIEST, DRUID, HIGH_PRIEST, and ACOLYTE, you could put this:
[PRIEST:MyCreature:17:0:AS_IS:DEFAULT]
[DRUID:MyCreature:17:0:AS_IS:DEFAULT]
[HIGH_PRIEST:MyCreature:17:0:AS_IS:DEFAULT]
[ACOLYTE:MyCreature:17:0:AS_IS:DEFAULT]
Note how they all point to the same tile.

However, I'd recommend creating a full tile sheet for the race and just using this when a profession doesn't have an image.  That way, if a graphics set has images for a specific job, it won't remove those images.

A few other notes:
* As burned pointed out KING and QUEEN as well as KING_CONSORT and QUEEN_CONSORT have been merged to create MONARCH and MONARCH_CONSORT
* You are missing ADVENTURER graphics.  These will default (actual defaulting, not using tile pointing) to the :DEFAULT version if it isn't defined, but some graphic sets may create these manually.
* PRIEST is listed twice
* Noble and appointed positions can have different names for different entities.  I think Humans have CHIEF_MEDICAL_HUMAN instead of CHIEF_MEDICAL_DWARF.  Just something to be aware of.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 02, 2016, 09:40:37 am
I think at this point we do too much designing and too much optimizing too early. We should factor out problems and roadblocks for now and just get a working, simple script soon. So, I believe we would benefit from separating this whole task in piecemeals:


1st Milestone. Next:
2nd Milestone. Only then we
3rd Milestone. And finally.

Instead of grinding our teeth on making a wholesome script from the beginning, and burning out everyones energy and interest. I think this gives us more time (and maybe more people) to research positions and professions, and Quiet-Sun has a more reasonable sequence to get familiar with DF and the whole process without being weighted down by unsolved problems. Furthermore, this would open up the whole idea to more people earlier and give us a broader test-case environment.
Worst case, we have a script for creatures only that might or might not get expanded in the future. Best case, we have a solid base to slowly expand.
Right now, the worst case is endless development hell as we stumble upon problem after problem and never get anything finished.



Additionally, I suggest the graphic set talk page as place to collect all information, validated or speculative, about professions, positions, and priorities. It's all too easy to overlook good information buried in posts that discuss 3 different topics with 2 different people via 10 different quotes.
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on February 02, 2016, 12:00:04 pm
Can beastmen have all the professions available to dwarves and elves? or do they operate under a reduced set of professions?

Beastfolk and gorlaks can have any profession available to the civilization(s) they latch on to - which can be any civilization. Technically, so can elves, dwarves, humans, goblins, etc.

Plump Helmet Men can have any profession which doesn't require talking. (Not sure if worldgen respects this.)

So in Summary, any Dwarf, Elf, Human, Goblin, Kobold, Animal Person, and Gorlak can theoretically have any profession, and consequently a graphic assigned to it correct?

Not kobolds, not without modding. [UTTERANCES] prevents them from joining non-[UTTERANCES] civs.

The easiest way to test animal person profession graphics would be to change the species of the MOUNTAIN entity to the species you want to test, generate a test world, and embark with beastfolk of 7 professions at a time. (Though I believe with DFHack you can increase your embarkation party number, so if you use a version with a working DFHack you could get a full compliment of professions in a single embark.)
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 02, 2016, 01:43:11 pm
Thank you guys.  I'm not going to use quotes this time  but this is a response to @Burned, @Rydel and @CLA

I think that a lot of the issues arise from my use of the word DEFAULT because it is used extensively by DF.  When I said DEFAULT, I really meant a DEFAULT-VISUAL-QUEUE that has nothing to do with the inner workings of DF.  The goal of my grouping is to optimize the information that the player can grasp from the UI, but do so using only tokens that exist in DF.  When looking at these groups remember that if a tile exists for any of the professions in the list it will be used (if not, ONLY ITS APPEARANCE will be defaulted to the one at the top of the list). 

 My reasoning behind all this is:  Right now people only started to make art for beastmen.  By defining a prioritized set of tiles we provide a framework for artists to maximize the impact of their work (this is all inspired by the work of Telltale).   It is much better to have a basic set of 15 tiles for every beastrace (packing the greatest amount of visual information for the player) than 300 different tiles for a single race.  However, to reiterate my point, if a tile exists it will be used (no tile left behind!).

Let me give you an example.  Here is an excerpt from Burned's txt definitions:
Spoiler (click to show/hide)

And here one from Rally-Ho's
Spoiler (click to show/hide)

Pay close attention at the way Rydel and Burned are grouping their professions.   This grouping is completely irrelevant and transparent from DF's point of view, but highlights the way we humans like to streamline concepts so that we can manage things better.

Now look at mine:
Spoiler (click to show/hide)

My list contains the very same number of professions (grouped almost the same way).  The only difference is that I'm picking one profession from each group to work as DEFAULT-VISUAL-QUEUE  and title of the group (instead of a comment that DF doesn't read).  I'm simply formalizing what Rydel, Burned, and Telltale, (and all humans I would argue) naturally do.

Now, this is a very subjective choice and its highly constrained by the professions that exist in the game.  I would have preferred to use NOBLE or LEADER (as Burned, and Telltale do) instead of DUKE, but neither NOBLE nor LEADER are true professions so I picked DUKE because it seemed to me more universally applicable than MONARCH.



The other thing that I wanted to address is what defines a profession.  I agree with CLA that the best venue for such a discussion is the WIKI.  My working assumption from now on is that the wiki has up-to-date information.   If there is a problem with my profession list, then it means we need to fix it first in the wiki.  Now all this speculation would be over if Toady would simply include a complete list in the raws.  That would make our lives so much easier!  Is anyone here his friend? can we make a request for this information for future versions of DF?



Finally I want to agree with CLA that it's better to have a fully working published version, than strive for perfection on the first try.  However, I do think that so far we have been moving forward at a very good pace and all this discussion is very valuable for me.  It helps me understand how the game works better and polishes our ideas, so please keep your feedback going (including potential problems).  I'm good at making design decisions and turning up a working product so more feedback is better than less.



SOMETHING I COULD USE FEEDBACK WITH:

Working within my design concept, would you think than any profession would belong better in a different group?  Here is the latest version with adventurer tags (I seem to recall that adventurers can only have the military types):

Spoiler (click to show/hide)

 
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 02, 2016, 02:33:09 pm
Ah, I see what you were getting at now.  Yeah, the many uses of Default were causing confusion.

I think the reason that there isn't a list of professions included is that a lot of them are defined in the entity raws.  So, if someone mods in a new civilization, that can easily introduce a bunch of new professions.

Looking over your list, everything seems well placed.  My gut says Alchemist should probably go somewhere else, but I can't see any place it fits better, so I'd leave it as-is.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 03, 2016, 05:14:03 am
I don't care how you organize them. DF doesn't care how you organize them.

From CLA, Rydel, to my own definitions - we all organized them in a way that made sense to us. They are the same definitions, so they all work (aside from the ones CLA mentioned so you have a test case for your script). Your organization is also based on a way that made sense to you.

~~~~

Anyway . . .

When I said DEFAULT, I really meant a DEFAULT-VISUAL-QUEUE that has nothing to do with the inner workings of DF.  The goal of my grouping is to optimize the information that the player can grasp from the UI, but do so using only tokens that exist in DF. . . .
&
The only difference is that I'm picking one profession from each group to work as DEFAULT-VISUAL-QUEUE  and title of the group (instead of a comment that DF doesn't read).  I'm simply formalizing what Rydel, Burned, and Telltale, (and all humans I would argue) naturally do.

Your use of the word default was confusing, but if you're going to present a standard template that more than one person is going to use why not just use notes? I can easily look at Rydel's and understand his format as I'm sure he understood mine, because notes.


When looking at these groups remember that if a tile exists for any of the professions in the list it will be used (if not, ONLY ITS APPEARANCE will be defaulted to the one at the top of the list).

Do you mean that you're going to change vanilla behavior by having the tile "default" to something that is created to substitute something that is not? Meaning if you don't have a CHAMPION or an EXPEDITION_LEADER or whatever they will default to the SHERIFF? If so, why? One of the benefits of a graphic set is to parse the information without using "K" in fort mode or "L" in adventure mode. A visual glance of who is who and what is what. What's the point if your DIPLOMAT and OUTPOST_LIAISON look like MERCHANTS?

Leave it as is. This allows people to see the holes in sets and doesn't introduced, "Why does this DRUID look like a PRIEST?" or worse, "Why doesn't this graphic set work? The tiles are all wrong!"

I hope that I completely misunderstood what you meant.

~~~~

Regarding your list, these are still present regardless of what you meant by default.

. . . KING, KING_CONSORT, QUEEN_CONSORT don't exist as far as I know. They are covered by MONARCH and MONARCH_CONSORT. Again, I don't recall why I left QUEEN in there, because the elf QUEEN is the LEADER profession much like the PRINCESS position is actually defined by GENERAL for elves and I don't even see LEADER on your list. Nor do I see WARLORD or WARRIOR."

~~~~

"I would have preferred to use NOBLE or LEADER (as Burned, and Telltale do) instead of DUKE, but neither NOBLE nor LEADER are true professions"

While I oppose the idea of introducing a new type of "default" . . .

. . .the elf QUEEN is the LEADER profession . . .

~~~~

Now all this speculation would be over if Toady would simply include a complete list in the raws.  That would make our lives so much easier!  Is anyone here his friend? can we make a request for this information for future versions of DF?

I did send Toady an email hoping it was a quick addition to the arena like the north/south pole was to advanced world gen, but I don't expect him to drop everything to put it in there, especially if it's more complicated than just adding the poles.

~~~~

I think at this point we do too much designing and too much optimizing too early. We should factor out problems and roadblocks for now and just get a working, simple script soon.

<task list>

Instead of grinding our teeth on making a wholesome script from the beginning, and burning out everyones energy and interest. I think this gives us more time (and maybe more people) to research positions and professions, and Quiet-Sun has a more reasonable sequence to get familiar with DF and the whole process without being weighted down by unsolved problems. Furthermore, this would open up the whole idea to more people earlier and give us a broader test-case environment.
Worst case, we have a script for creatures only that might or might not get expanded in the future. Best case, we have a solid base to slowly expand.
Right now, the worst case is endless development hell as we stumble upon problem after problem and never get anything finished.

@CLA You're right and your task list is sound.

@Quiet-Sun I still have no idea what's going on with the red tiles or if a script for the graphic set page is possible or whether that new "default" even makes sense. However, I'm not going to comb through another list of yours to see if it's correct. The entire point, I thought, was to automate the process which in turn would solve the existing problems in the repo as it is right now instead of manually updating each set.

I sincerely appreciate the effort you are putting into this, but I feel it's a tangent overlooking the current problems of the repo and possibly just adding new ones. I would suggest using CLA's task list as a starting point and ignore everything else, because I'd love to see what you can come up with in the end.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 03, 2016, 11:42:48 am

@Quiet-Sun I still have no idea what's going on with the red tiles or if a script for the graphic set page is possible or whether that new "default" even makes sense. However, I'm not going to comb through another list of yours to see if it's correct. The entire point, I thought, was to automate the process which in turn would solve the existing problems in the repo as it is right now instead of manually updating each set.

I sincerely appreciate the effort you are putting into this, but I feel it's a tangent overlooking the current problems of the repo and possibly just adding new ones. I would suggest using CLA's task list as a starting point and ignore everything else, because I'd love to see what you can come up with in the end.

All tasks I'm doing are tied together.  You will see a completely automatic standardizing and updating script this week.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 03, 2016, 03:33:29 pm
Guys I wanted to let you know something I just discovered.  I think all valid professions and positions can already be found within the last DF builds!  If you check the graphics_example.txt you will see the following text:

40.24:
Spoiler (click to show/hide)

42.05:
Spoiler (click to show/hide)

Note how the new professions in 42.05 are appended at the bottom of the profession list below MASTER_LASHER.  Also the wording really makes it sound like an official list. What do you think?

EDIT:  What I don't understand is what is meant by "...but you should include all seven of these for a given creature..."  Which "these" can they be referring to?

Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 03, 2016, 04:41:57 pm
The graphics_example.txt is incomplete.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 03, 2016, 04:57:30 pm
The graphics_example.txt is incomplete.

It is :( sorry.   I was so happy for a second.  There are lots of professions in the entity_default.txt that are not in there.  Oh well
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 03, 2016, 07:39:35 pm
Quick Update.  I have encountered a limit in the amount of tiles you can load into DF.   I discovered it when I tried to load each creature with 5 tiles the majors with ~250 and each beastman race with 52.   I don't have exact numbers, but if I gradually remove professions from the beasmen the game finally works with about 35 tokens.  Has any of you heard of someone pushing the game to such a limit?
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 03, 2016, 08:02:48 pm
The limit was first noted in the 40d discussion (http://dwarffortresswiki.org/index.php/40d_Talk:Graphics_set_repository) (under limitations), whenever it comes up (http://www.bay12forums.com/smf/index.php?topic=154711.msg6717092#msg6717092), and it's been on the current notes of the wiki page (http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#Notes) ever since CLA started this thread (http://www.bay12forums.com/smf/index.php?topic=144246.0).
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 03, 2016, 08:12:14 pm
The limit was first noted in the 40d discussion (http://dwarffortresswiki.org/index.php/40d_Talk:Graphics_set_repository) (under limitations), whenever it comes up (http://www.bay12forums.com/smf/index.php?topic=154711.msg6717092#msg6717092), and it's been on the current notes of the wiki page (http://dwarffortresswiki.org/index.php/DF2014:Graphic_set#Notes) ever since CLA started this thread (http://www.bay12forums.com/smf/index.php?topic=144246.0).

Ah! that's exactly it!  thank you burned!!!
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 03, 2016, 08:42:18 pm
According the the wiki, this is a limit on the number of tiles per sheet.
Perhaps you could avoid this by separating the default and adventurer tiles into two separate sheets.
This would also allow you to drop the adventurer graphics entirely if a set doesn't define them.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 04, 2016, 02:18:00 pm
Guys, two quick questions: 

1. What defines a beastman (and by beastman I mean a species that can have professions)?    I'm assuming that everything that ends in "MAN]" or has the word "ELEMENTMAN", or is a "GORLAK" is a beastman.  Is this a good working assumption, will I miss something this way?

2. What is the difference between using "ADD_COLOR]" (like @Rydel in Rally-HO) and "ADD_COLOR:DEFAULT]" (like @CLA)?
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 04, 2016, 02:28:55 pm
:ADD_COLOR] is probably me making mistakes.

For having professions, I think it requires [CAN_LEARN] or [INTELLIGENT]


EDIT: Definitely a type - it looks like the error only exists for Kobolds, which aren't heavily tested - the full sheet's there since I'm working on porting the tileset for Masterwork.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 04, 2016, 03:01:04 pm
Blizzard man, Amethyst man, Blood man, Fire man, Gabbro man, Iron man, Magma man, Mud man end with "MAN," but are NOT considered animal people. Gorlak and Plump Helmet Man are exceptional animal people in that they are not based on animals.

There is a difference between tribal/legacy animal people and above-ground animal people (note: tribal/legacy animal people can be found aboveground). Currently, only a select few tribal/legacy animal people can join civs during w.g. and after. As for professions, see Button's previous response in this thread.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 04, 2016, 03:27:48 pm
Thank you so much @burned and @Rydel

So, if I base my script solely on the raws.  Would you agree with the following definition statement?

creatures that need profession tile art are those containing the text: ":ANIMAL_PERSON]",  " [INTELLIGENT]" , &/or "[CAN_LEARN]"


By the way, I think I found a way of updating a tileset automatically.  I will upload the application today for you to test with your sets.  This application will not include the standardization.  I don't want to try to coerce anyone into using it (I hope to convince you!  :-*).    That one is also almost ready and working really, really great.  I will upload an application pack with the updater, the standardizing application, and a tile resizer to a separate thread today or tomorrow.



Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 04, 2016, 04:46:14 pm
Maybe there is an exception I missed, but I don't think any animal person has the [INTELLIGENT] tag. Offhand, pixies and fairies are [INTELLIGENT], but do not have professions.

As mentioned earlier, Gorlaks and Plump Helmet Men are an odd exception; they don't have ANIMAL_PERSON, but they do have [CAN_LEARN] (and theoretically can have professions), but blind cave ogres, maneras, dark gnomes, mountain gnomes and many other things have [CAN_LEARN] and they definitely do NOT have professions.


Title: Re: Dwarf Fortress graphics repositories
Post by: Button on February 04, 2016, 05:06:02 pm
Maybe there is an exception I missed, but I don't think any animal person has the [INTELLIGENT] tag. Offhand, pixies and fairies are [INTELLIGENT], but do not have professions.

As mentioned earlier, Gorlaks and Plump Helmet Men are an odd exception; they don't have ANIMAL_PERSON, but they do have [CAN_LEARN] (and theoretically can have professions), but blind cave ogres, maneras, dark gnomes, mountain gnomes and many other things have [CAN_LEARN] and they definitely do NOT have professions.

All animal people are [INTELLIGENT], or at least [CAN_LEARN][CAN_SPEAK]. It's in their creature variation.

Thank you so much @burned and @Rydel

So, if I base my script solely on the raws.  Would you agree with the following definition statement?

creatures that need profession tile art are those containing the text: ":ANIMAL_PERSON]",  " [INTELLIGENT]" , &/or "[CAN_LEARN]"

The ones that need profession tiles are the ones with:

Though I'm not sure if there are any CONTROLLABLE that don't meet one of the previous two.

The animal person templates contain both HEROES and CONTROLLABLE, so any critter with either of those templates meets the requirements. So if you want to just search the creature raws without applying the creature variation templates, searching just for :ANIMAL_PERSON or [LOCAL_POPS_ should be fine. Not sure if dwarves/humans/elves/goblins/kobolds will be picked up by that algorithm, but those pretty much go without saying by now.

There is an additional wrinkle regarding UTTERANCES. If an entity's core species has UTTERANCES, only creatures with UTTERANCES can join it; and if an entity's core species doesn't have UTTERANCES, only creatures without UTTERANCES can join it. Happily in vanilla this tag only exists in kobolds.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 04, 2016, 05:28:36 pm
Maybe there is an exception I missed, but I don't think any animal person has the [INTELLIGENT] tag. Offhand, pixies and fairies are [INTELLIGENT], but do not have professions.

As mentioned earlier, Gorlaks and Plump Helmet Men are an odd exception; they don't have ANIMAL_PERSON, but they do have [CAN_LEARN] (and theoretically can have professions), but blind cave ogres, maneras, dark gnomes, mountain gnomes and many other things have [CAN_LEARN] and they definitely do NOT have professions.

All animal people are [INTELLIGENT], or at least [CAN_LEARN][CAN_SPEAK]. It's in their creature variation.

The ones that need profession tiles are the ones with:
  • Their own entity;
  • LOCAL_POPS_PRODUCE_HEROES; or
  • LOCAL_POPS_CONTROLLABLE.

Though I'm not sure if there are any CONTROLLABLE that don't PRODUCE_HEROES or have their own entity.

The animal person templates contain both HEROES and CONTROLLABLE, so any critter with either of those templates meets the requirements.


Ah Button, thank you! this is very helpful!  What do you mean by "having their own entity"?


-----------------------------------------------------------------------------------------------------------


Also, guys! here is the updater for your enjoyment:

QS DF Graphics set updater (https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater.zip)

I believe that it only works for Windows 64-bit and you need the Windows 64-bit version of the MATLAB Runtime for R2015b, which you can find here: http://www.mathworks.com/products/compiler/mcr/index.html (http://www.mathworks.com/products/compiler/mcr/index.html)

Place it in its own folder and be sure to run it from the command promt or it will run in the background and do its thing without giving you feedback.   

1. It will ask you for a folder with the DF creature object files; use one of the two that I provided
2. It will ask you for the folder of the graphic set you want to update.

then it will create an updated version of this set inside the folder with the EXE.  It will have the same name with the text "-Updated" appended.   Inside it you will find the same files as in the originals, but will have substituted names that it thinks have changed from one version to the other.   Also, inside the "-Updated" folder you will find three text files (see the spoliers for an example ran with CLA):

Name_Changes.txt:  Will tell you the names that it thinks changed from the previous version, but point to the same creature
Spoiler (click to show/hide)

Names_in_Set_not_in_Raws.txt:  Will tell you which creatures no longer exist in the raws
Spoiler (click to show/hide)

Names_in_Raws_not_in_Set.txt:   Will tell you which creatures are in the raws for which there is no entry in the set.  IMPORTANT this assumes the name substitutions are correct so check that first.
Spoiler (click to show/hide)

I think it works flawlessly between 40.24 and 42.05.  We will have to wait and see what else Toady throws at us.

Please keep in mind I have never released an app before so kindly assume is bugged, broken, and sucks.  Let me know if you were able to make it work and if there is a way I can make it better.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 04, 2016, 05:36:57 pm
All animal people are [INTELLIGENT], or at least [CAN_LEARN][CAN_SPEAK]. It's in their creature variation.

I definitely see [CAN_LEARN] and [CAN_SPEAK] in the creature variation, yes. But, I only see [INTELLIGENT] in the standard races, pixies and fairies. I suppose it doesn't matter, since it doesn't narrow down the scan for animal people.

Also, isn't scanning for ANIMAL_PERSON and/or ANIMAL_PERSON_LEGLESS tags better than scanning for the LOCAL_POPS tags?

Maybe I'm wrong, but my understanding of Quiet-Sun's script is that he'll get zero CREATURES tagged with either LOCAL_POPS tags, because he's scanning the raws as is not how they are defined in game. Maybe that's why I don't see where the [INTELLIGENT] tag for animal people is located? Do you mean that tag is applied in game?


edit: Quiet-Sun, I'll check out your updater asap. =]
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on February 04, 2016, 05:40:33 pm
Ah Button, thank you! this is very helpful!  What do you mean by "having their own entity"?

I also edited the post a little more after you had already replied, FYI.

A creature has an entity if there exists within an entity raw file the tag [CREATURE:<CREATURENAME>]. For example, the dwarven civilization is defined thus:

Code: [Select]
[ENTITY:MOUNTAIN]
[CREATURE:DWARF]
...
...
...
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on February 04, 2016, 06:00:38 pm
All animal people are [INTELLIGENT], or at least [CAN_LEARN][CAN_SPEAK]. It's in their creature variation.

I definitely see [CAN_LEARN] and [CAN_SPEAK] in the creature variation, yes. But, I only see [INTELLIGENT] in the standard races, pixies and fairies. I suppose it doesn't matter, since it doesn't narrow down the scan for animal people.

[INTELLIGENT] is shorthand for [CAN_LEARN] + [CAN_SPEAK]. They're exactly equivalent.

Quote
Also, isn't scanning for ANIMAL_PERSON and/or ANIMAL_PERSON_LEGLESS tags better than scanning for the LOCAL_POPS tags?

Maybe I'm wrong, but my understanding of Quiet-Sun's script is that he'll get zero CREATURES tagged with either LOCAL_POPS tags, because he's scanning the raws as is not how they are defined in game.

You'd need to scan for all creatures that have ANIMAL_PERSON or LOCAL_POPS, because ANIMAL_PERSON will pick up the creatures that get LOCAL_POPS through their templates, but won't pick up gorlaks or plump helmet men; while LOCAL_POPS will pick up gorlaks & plump helmet men, but won't pick up animal people (because animal people's LOCAL_POPS are wrapped up in their templates).

This is of course assuming that you only want to scan vanilla raws. If you want to make a tool that will find the creatures that need tiles for any given modification, you'll need to delve into the template files.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 04, 2016, 06:02:14 pm
Thank you very much @Button!  This is exactly the information I was looking for!  For now only vanilla, one baby step at a time!
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 04, 2016, 06:25:57 pm
Oh guys sorry! I discovered a bug and the updater was not updating  ::)     Now it's doing the name substitutions:

https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater.zip (https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater.zip)
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 04, 2016, 07:15:53 pm
@Button Thanks

@Quiet-Sun I hope to have more time to check it out later, but I did run it once using my set (since I am familiar with what is and isn't in there). The spoiler contains the results I got for "Names_in_Raws_not_in_Set.txt," . . . *ahem* Apparently I exceeded the maximum allowed length for a post with that spoiler.

Anyways, at a quick glance, the Names_in_Raws_not_in_Set.txt seem to list everything I DO have in the set contrary to its name. My "Names_in_Set_not_in_Raws.txt" was blank, so it looks like the text file names are just reversed, but I thought it was going to call out some of the names I have in there that I know are outdated (which it didn't).

My "Name_Changes.txt" was also blank, but I didn't expect there to be any outdated names.

Aside from what I got above - my image files and definitions were untouched.

Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 04, 2016, 10:16:09 pm
Hi Burned!  Thank you for trying.  You have been very helpful all throughout this process and I appreciate it very much!

I suspect there is a problem with the paths in the application.  I will upload a new version later that writes more stuff in the files to see if we can isolate the problem.  In the meantime here is the output in my computer:

Name_Changes.txt
Spoiler (click to show/hide)

Names_in_Raws_not_in_Set.txt
Spoiler (click to show/hide)

Names_in_Set_not_in_Raws.txt
Spoiler (click to show/hide)

Does it make sense?
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 04, 2016, 10:41:21 pm
Ok now it should print the full paths.  Please try and see if it works and the paths make sense:

https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater.zip (https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater.zip)
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 05, 2016, 03:26:39 am
The results for the three files you posted are correct; those are the results I expected to see sans the vermin. Ha!

Name_Changes.txt lists the TOAD_GIANT I left in, the Names_in_Raws_not_in_Set.txt lists all the animal people and giant animals I removed (plus vermin that have no tiles), and Names_in_Set_not_in_Raws.txt list one out of date creature (werewolf) and the temporarily removed scorpion.  While I think listing vermin is overkill, the script is correct as far as I can tell.

Unfortunately, I must be doing something wrong because I am not getting the results you are listing.

Name_Changes.txt is blank.
Names_in_Raws_not_in_Set.txt lists everything tile or no tile.
And, Names_in_Set_not_in_Raws.txt is blank.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 05, 2016, 01:16:23 pm
Ok, I tried to fix the problem.  Can you please download it again and try.  This version should open a console window if you double click it and it should say that it's version 1.01.  Can you please give it a try and let me know?

https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater.zip (https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater.zip)
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 05, 2016, 02:05:38 pm
I tried version 1.01 with the same results.

We can work this out via email to avoid turning this thread into a pattern of not working, try now replies.



Update: *a few emails later* The first step in the form of 1.02 works perfectly. Step two is in progress.
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 05, 2016, 06:32:18 pm
Hi All!

I just wanted to update the upload after cleaning the output files a little.  Burned was of great help and it seems to be working as inteded now.

This is the updater.  It's a tool for finding and correcting simple but easy-to-miss name changes (a space with an underscore, changing the order of words).   It will also give you a comprehensive list of all the differences between your set and the raws.  This way you can fix manually whatever is wrong and add whatever is needed.   A copy of spacefox is included for testing.

https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater_1.03.zip

Now to finish up the standardizer!
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on February 05, 2016, 07:53:00 pm
Hi All!

I just wanted to update the upload after cleaning the output files a little.  Burned was of great help and it seems to be working as inteded now.

This is the updater.  It's a tool for finding and correcting simple but easy-to-miss name changes (a space with an underscore, changing the order of words).   It will also give you a comprehensive list of all the differences between your set and the raws.  This way you can fix manually whatever is wrong and add whatever is needed.   A copy of spacefox is included for testing.

https://dl.dropboxusercontent.com/u/408446/DF/Applications/QS_GraphicsSet_Updater_1.03.zip

Now to finish up the standardizer!

Can I suggest starting a new thread for this, since it's not only relevant to the df-graphics organisation?

(It would also be good to host code on Github or similar, or at least binaries on DFFD)
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 05, 2016, 07:59:08 pm

Can I suggest starting a new thread for this, since it's not only relevant to the df-graphics organisation?

(It would also be good to host code on Github or similar, or at least binaries on DFFD)

Yes! I'm almost done with the standardizer.  Things are already on github, but I was waiting to be done with the standardizer to open a new thread with the updater, the standardizer, and the tile resizer.   It will be an awesome mega-combo of applications that is going to make our lives much easier from now on  8)  You can expect to see it over the weekend.

Once this thread is up I will post here my suggestions regarding the repository.  Sorry of kind of hijacking this thread temporarily, but I was not completely sure I could make it work, but I did and it's great! more soon!
Title: Re: Dwarf Fortress graphics repositories
Post by: Quiet-Sun on February 09, 2016, 02:59:50 am
Ok Guys! the thread is up!  I added one more tool to merge different sets into one!  Now to sleep! more tomorrow.
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on February 11, 2016, 01:26:34 pm
FYI fricy I've made a couple pull requests.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on February 15, 2016, 09:40:40 pm
Excerpted from a recent PM I received - an additional 12x16 Spacefox tileset.

I'm Das24680 from YouTube and you had kindly linked my tutorial series into your Starter Pack.

I'm in the middle of quite an epic series at the moment on YouTube and have tweaked a few elements within DF. I post the links online to my personalised files but I still get questions from people on why my version looks different to theirs. Recently I've revised the Spacefox text to make it 12x16 which makes portions of the game much more legible (trading etc) so was wondering if you wanted to include it in your pack along with some of the other files I use? No problem if you prefer to not include the files. :)

Link to the Spacefox 12x16 Text is http://www.mediafire.com/view/ym7n9bn6x6z160n/Das_Spacefox_Text.png
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on February 15, 2016, 10:49:41 pm
By the way, did fricy mention to anyone that he was going to be away? He hasn't logged onto the forum or github in 2 weeks. I'm getting a little worried.
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on February 16, 2016, 12:36:37 am
By the way, did fricy mention to anyone that he was going to be away? He hasn't logged onto the forum or github in 2 weeks. I'm getting a little worried.

Ok, ok, I'll get on it. It will be needed anyways, since I won't have much time on my hand between April-August. I'll make a thread for the repo, and create an organization on Github with all the individual packs, so others can keep the flame going when I'm not around.

But, I'm not sure if his recent absence is related.

Title: Re: Dwarf Fortress graphics repositories
Post by: BlackSmokeDMax on February 16, 2016, 08:40:12 am
Excerpted from a recent PM I received - an additional 12x16 Spacefox tileset.

I'm Das24680 from YouTube and you had kindly linked my tutorial series into your Starter Pack.

I'm in the middle of quite an epic series at the moment on YouTube and have tweaked a few elements within DF. I post the links online to my personalised files but I still get questions from people on why my version looks different to theirs. Recently I've revised the Spacefox text to make it 12x16 which makes portions of the game much more legible (trading etc) so was wondering if you wanted to include it in your pack along with some of the other files I use? No problem if you prefer to not include the files. :)

Link to the Spacefox 12x16 Text is http://www.mediafire.com/view/ym7n9bn6x6z160n/Das_Spacefox_Text.png (http://www.mediafire.com/view/ym7n9bn6x6z160n/Das_Spacefox_Text.png)

That is a great text set. Been using it since he first mentioned it in one of his "Let's Plays"

Think you'll include it in some future version of the pack?
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on February 18, 2016, 07:44:17 pm
I've created a GitHub repo of my graphics set - how would I request inclusion if it's a separate repo rather than a fork?
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on April 07, 2016, 08:10:28 am
Unfortunately Fricy vanished *slightly* too early, having added me as an admin but not owner.  Since nobody else can add members or repos to the organisation, we'll need to start over with a new one.  (I tried, but GitHub is understandably reluctant to transfer ownership of an organisation without permission)

I'll get that set up in the next few days, make a new thread, and send out a bunch of invites - suggestions for a name most welcome!
Title: Re: Dwarf Fortress graphics repositories
Post by: Max™ on April 07, 2016, 08:42:32 am
FricaseedGraphics
DeeEffGraphics+1
Repositoranumanomanonemonom
FricyFricyHeIsOurManIfHeCantOhWaitCrap
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on April 07, 2016, 11:25:16 am
DFGraphics2 would be my suggestion. Something obvious.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on April 07, 2016, 11:39:16 am
Agreed on DFGraphics2. Ideally Fricy would be back at some point and we could merge/link/whatever the two as DFGraphics again.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on April 07, 2016, 01:34:15 pm
Wait, Fricy vanished?

Damn...
Title: Re: Dwarf Fortress graphics repositories
Post by: Button on April 07, 2016, 01:54:12 pm
Wait, Fricy vanished?

Damn...

He hasn't even answered my "Hey, are you OK?" PM/presumably email.  :-\
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on April 07, 2016, 10:53:44 pm
FricyFricyHeIsOurManIfHeCantOhWaitCrap
Nearly spat out my drink.

DFGraphics2 would be my suggestion. Something obvious.
Agreed on DFGraphics2. Ideally Fricy would be back at some point and we could merge/link/whatever the two as DFGraphics again.
I considered this, but I think it's more likely to cause confusion - people will leave off the "2", and then links are wrong, and so on.  A distinct name would be better, and no impediment to merging if that ever happens.


Wait, Fricy vanished?  Damn...
He hasn't even answered my "Hey, are you OK?" PM/presumably email.  :-\
I've been sending semiregular emails to gmail, and PMs here and on Reddit for about a month with no response.
Hence, time to work out alternatives  :'(
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on April 09, 2016, 08:25:52 pm
Woohoo, good news everyone!  By the grace of Fricy or GitHub staff, I'm now an owner of the organisation and that means I can add people!
We'll use this thread to get set up, so please reply with or PM me your GitHub username, and I'll add you to the team!

Here's my TODO list, which I'm hoping people can help me with:

 - Add people to the "maintainers" group (read/write on all packs)
 - merge Afro and Spacefox back to a single pack
 - Assign more granular teams (admins for particular packs, etc)
 - DOCUMENTATION of format, how to use, authors, etc.  Using Sphinx, like DFHack docs.
 - Set up some tests
 - Use standard layout per up-thread, check all raws, etc.

There's some far-future stuff too, but let's get the basics working first :)
Title: Re: Dwarf Fortress graphics repositories
Post by: Thundercraft on May 22, 2016, 04:48:59 am
PTW.
Title: What's the scope of DFGraphics?
Post by: jecowa on May 23, 2016, 03:45:29 am
I heard that this project was made to add TWBT support to everything. Is this still a goal of this project?

Concerning the purpose of this group, Peridexis told me, "The aim is to make maintaining a pack as easy as possible." Does everyone agree with this?

What determines which packs are included in this repository? It looks like maybe it's any pack whose author choses to host it here, in addition to all the packs that are no longer being maintained by their original authors?

What determines which version of Dwarf Fortress we make these packs compatible with? Do we just always update them for compatibility with the latest version of Dwarf Fortress? To clarify this question a bit, Dwarf Fortress v0.42.06 is the latest version supported by DFHack; Dwarf Fortress v0.43.02 is the latest version supported by Dwarf Therapist; and Dwarf Fortress v0.43.03 is the latest version.
Title: Re: What's the scope of DFGraphics?
Post by: PeridexisErrant on May 23, 2016, 05:54:41 am
My two cents, which will eventually make it into actual project documentation... after thesis.  (I've been saying that a lot lately.  Not long to go.)

I heard that this project was made to add TWBT support to everything. Is this still a goal of this project?
Yes, but not super-high-priority unless an artist volunteers.  We've added generic items to everything thanks to Meph, and some packs (eg Doren's Phoebus) have more overrides.  Reducing support for non-TwbT installs is unlikely - while I'd like to undo the raws mess that is "tile magic", too many sets rely on it and the tradeoffs aren't worth it.

Concerning the purpose of this group, Peridexis told me, "The aim is to make maintaining a pack as easy as possible." Does everyone agree with this?
If there's a volunteer to make it harder, they can have my job  :P
More seriously, I'm planning some tools for semi-automatic updating DF versions, validating raws, etc. to write over coming months.

What determines which packs are included in this repository? It looks like maybe it's any pack whose author choses to host it here, in addition to all the packs that are no longer being maintained by their original authors?
Exactly.  Some, which have been picked up by maintainers who don't want to use GitHub (eg Ironhand by GoldenShadow), should probably be suspended or something but I haven't looked at this lately.

What determines which version of Dwarf Fortress we make these packs compatible with? Do we just always update them for compatibility with the latest version of Dwarf Fortress? To clarify this question a bit, Dwarf Fortress v0.42.06 is the latest version supported by DFHack; Dwarf Fortress v0.43.02 is the latest version supported by Dwarf Therapist; and Dwarf Fortress v0.43.03 is the latest version.
The latest graphics pack should match the latest version of DF.  Older versions are/should be supported via releases -- on Github you can 'tag' a given state for anyone to download later.
Title: Re: What's the scope of DFGraphics?
Post by: Thundercraft on May 23, 2016, 06:15:04 am
...Dwarf Fortress v0.42.06 is the latest version supported by DFHack; Dwarf Fortress v0.43.02 is the latest version supported by Dwarf Therapist; and Dwarf Fortress v0.43.03 is the latest version.

That's a good point. Several of the packs in this GitHub collection are lagging behind. Several of these like Ironhand, Mayday, Taffer, and Tergel have only been updated to 0.42.05 and are months old. This, despite the fact that some have used and uploaded (DFFD) packages for later DF versions (42.06, 43.02, 43.03...), without updating the GitHub repositories.

What this means is that, while DFHack and certain utilities that many players swear are essential for them to play are (currently) limited to 42.06, while finding graphics packs for that version is becoming difficult since GitHub is still stuck at .05 and a lot of people seem to be jumping on the 0.43 bandwagon.

Imagine, say, months from now where someone regales an amazing story about an amazing fort or adventure they had in a world generated with a mod based on 42.06. And he shares his save or worldgen settings, with others wanting to look at it or play the same world/site. Except, DFFD doesn't have their favorite graphics pack for 42.06 any more because it got updated-over or deleted and the GitHub for their favorite pack either stopped dead at 42.05 or skipped over 42.06 from .05 to some later version.

Edit:
...Exactly.  Some, which have been picked up by maintainers who don't want to use GitHub (eg Ironhand by GoldenShadow), should probably be suspended or something but I haven't looked at this lately.
Maybe send a friendly reminder about their GitHub repository?
Wait... They said that they don't want to use GitHub? Or does it just seem that way?
Title: Re: What's the scope of DFGraphics?
Post by: CLA on May 23, 2016, 06:22:51 am
Concerning the purpose of this group, Peridexis told me, "The aim is to make maintaining a pack as easy as possible." Does everyone agree with this?
I agree, moving to github makes it easier to maintain. Having a standard distribution format helps too. Though I would argue that a big part is "to make getting an up to date pack as easy and fast as possible for the end user". It does make it easier for maintainers too, but not more simple. It makes it easier by adding another layer of complexity (standard distribution format, github organization).
Title: Re: What's the scope of DFGraphics?
Post by: PeridexisErrant on May 23, 2016, 06:39:41 am
Several of the packs in this GitHub collection are lagging behind. Several of these like Ironhand, Mayday, Taffer, and Tergel have only been updated to 0.42.05 and are months old. This, despite the fact that some have used and uploaded (DFFD) packages for later DF versions (42.06, 43.02, 43.03...), without updating the GitHub repositories.

What this means is that, while DFHack and certain utilities that many players swear are essential for them to play are (currently) limited to 42.06, while finding graphics packs for that version is becoming difficult since GitHub is still stuck at .05 and a lot of people seem to be jumping on the 0.43 bandwagon.

Imagine, say, months from now where someone regales an amazing story about an amazing fort or adventure they had in a world generated with a mod based on 42.06. And he shares his save or worldgen settings, with others wanting to look at it or play the same world/site. Except, DFFD doesn't have their favorite graphics pack for 42.06 any more because it got updated-over or deleted and the GitHub for their favorite pack either stopped dead at 42.05 or skipped over 42.06 from .05 to some later version.

Edit:
...Exactly.  Some, which have been picked up by maintainers who don't want to use GitHub (eg Ironhand by GoldenShadow), should probably be suspended or something but I haven't looked at this lately.
Maybe send a friendly reminder about their GitHub repository?
Wait... They said that they don't want to use GitHub? Or does it just seem that way?

Yeah, it's not a great situation.  Things are improving though, in that we've got more people signed up who can update anything on Github - better that nobody *has* fixed it yet than nobody *can* fix it, IMO.  And ideally when you're updating a pack you do it in stages if there's something not compatible with either end, and add the tag.  (eg .05 is compatible with .06, and 43.x is all compatible too so far).  So fewer actual compatibility breaks than you might think!

GoldenShadow and Burned politely declined to join, and Burned asked that his graphics not be added.
Others have joined or have pending invites, but haven't pushed updates to github.  I think it's pretty rude to bug people about it though, since I'm not helping either due to lack of time.


Concerning the purpose of this group, Peridexis told me, "The aim is to make maintaining a pack as easy as possible." Does everyone agree with this?
I agree, moving to github makes it easier to maintain. Having a standard distribution format helps too. Though I would argue that a big part is "to make getting an up to date pack as easy and fast as possible for the end user". It does make it easier for maintainers too, but not more simple. It makes it easier by adding another layer of complexity (standard distribution format, github organization).

Fair.  I do think the simple/easy tradeoff is worth making in this case though, since it could allow eg. automatic DF-updates and computer checking of the format.
Title: Re: What's the scope of DFGraphics?
Post by: jecowa on May 23, 2016, 10:20:18 am
What determines which packs are included in this repository? It looks like maybe it's any pack whose author choses to host it here, in addition to all the packs that are no longer being maintained by their original authors?
Exactly.  Some, which have been picked up by maintainers who don't want to use GitHub (eg Ironhand by GoldenShadow), should probably be suspended or something but I haven't looked at this lately.

Here's some info that might help determine which repos should be considered active and which ones should be suspended.

The GitHub versions of these packs look to be the most up-to-date versions, and these packs have no other active maintainers:

These packs are being actively maintained by their original authors here on GitHub:

This pack's author is inactive, but it only edits a single objects file:

These packs are being maintained by people who don't/no longer use GitHub:

These packs' authors don't don't use GitHub. And they also don't edit any objects files:
Title: Re: What's the scope of DFGraphics?
Post by: Taffer on May 23, 2016, 10:50:39 am
These packs' authors don't don't use GitHub. And they also don't edit any objects files:
  • Taffer (http://www.bay12forums.com/smf/index.php?topic=107924.0)

Github (https://github.com/nc-z/taffer). It's been the primary download for my pack ever since DFFD switched hosts. I've also updated to new releases on Github pretty consistently, often the same day the release hits.

Minimal update to 0.43.01, including the keybindings. Life is happening. All current projects put on hold--not abandoned--for the next while (ie, rewritten creature descriptions (https://github.com/nc-z/df-raws), updates to my tilesets, updating my DFgraphics (https://github.com/DFgraphics/Taffer) repository).

I'll still keep things updated for new releases, but I don't even have DF installed at the moment. I still welcome contributions to the rewritten creature descriptions.

I'll see what I can do about updating the DFGraphics repository for my pack. I wasn't aware it needed any updating apart from the actual tilesets: it uses older versions of my work, but I don't edit any raw files and no new jobs were added for the creature graphics recently so there shouldn't be anything to update at present. I've been putting off updating the tilesets because I knew I'd be redrawing them in a few weeks anyways.

Kindly consider my DFgraphics repository (https://github.com/DFGraphics/taffer) maintained, and poke me in my tileset thread (or via PM or over Github) if it needs touching. I'll use that as the "Starter/Newbie Pack" resource for my set, for maintainers or users. I'm not sure what access I have to it at at present.

While I'm here, what subset of my tilesets should I include? I have almost no usage data whatsoever on what parts of my pack get used, and no interest in cluttering up the Newbie Pack with all 32 current tilesets. I've been maintaining a sans-serif font for years now at the request of one user (http://www.bay12forums.com/smf/index.php?topic=107924.msg4194036#msg4194036) (who didn't post after), but I haven't ever seen it used to my recollection, nor have I had any comments on it. By what little usage metrics I have, I'd be better off including my "dwarves are letters" version, because I have at least 2 users of those sets. I've seen screenshots of all four wall variants, so I'll keep those in.
Title: Re: What's the scope of DFGraphics?
Post by: jecowa on May 23, 2016, 11:57:03 am
These packs' authors don't don't use GitHub. And they also don't edit any objects files:
  • Taffer (http://www.bay12forums.com/smf/index.php?topic=107924.0)

Github (https://github.com/nc-z/taffer). It's been the primary download for my pack ever since DFFD switched hosts. I've also updated to new releases on Github pretty consistently, often the same day the release hits. Evidently I need to start doing so in the DFGraphics repo as well.

Sorry, I was looking at the commits to each pack on the DFGraphics repo to try to determine who used GitHub. Maybe we can make the one on the DFGraphics automatically sync itself with your repo. Or maybe we can replace the copy on the DFGraphics group with a link to your repo. I'm not really sure what the options all are. It seems silly for you to have to update two, though.

While I'm here, what subset of my tilesets should I include? I have almost no usage data whatsoever on what parts of my pack get used, and no interest in cluttering up the Newbie Pack with all 32 current tilesets. I've been maintaining a sans-serif font for years now at the request of one user (who didn't post after), but I haven't ever seen it used to my recollection, nor have I had any comments on it. By what little usage metrics I have, I'd be better off including my "dwarves are letters" version, because I have at least 2 users of those sets. I've seen screenshots of all four wall variants, so I'll keep those in.

I don't have usage data for your graphics pack, but from the poll from about a week ago, most users use the default settings. I think your "serif, straight, hollow" has been the default for a while, but it looks like your "serif, straight, solid" was the original version. So those might be good ones to include.

I don't think it's necessary to include all 32. I think you only need to include the 16 "dwarf letters" versions. With graphics turned on, a user will get the smiley faces; with graphics turned off, a user will get the dwarf letters. I don't have a problem with having more tilesets as options, but that seems like extra work for you to maintain.
Title: Re: What's the scope of DFGraphics?
Post by: Taffer on May 23, 2016, 12:35:34 pm
Sorry, I was looking at the commits to each pack on the DFGraphics repo to try to determine who used GitHub. Maybe we can make the one on the DFGraphics automatically sync itself with your repo. Or maybe we can replace the copy on the DFGraphics group with a link to your repo. I'm not really sure what the options all are. It seems silly for you to have to update two, though.

I don't think it's necessary to include all 32. I think you only need to include the 16 "dwarf letters" versions. With graphics turned on, a user will get the smiley faces; with graphics turned off, a user will get the dwarf letters. I don't have a problem with having more tilesets as options, but that seems like extra work for you to maintain.

Good ideas there. It would be a mistake to link the whole repositories: the folder structures are too different. My folder structure works well for people installing manually, but isn't a good fit for DFGraphics. (Contrariwise, the DFGraphics format isn't great for manual installations without the aid of a program). The tilesets would need to be renamed links directly to my own tilesets. I'll find time this week to get it all sorted out. Thanks. Perhaps we could do the same for other repositories (eg, CLA).
Title: Re: What's the scope of DFGraphics?
Post by: CLA on May 23, 2016, 03:02:41 pm
the DFGraphics format isn't great for manual installations without the aid of a program
How so?
Title: Re: What's the scope of DFGraphics
Post by: Taffer on May 23, 2016, 03:09:26 pm
the DFGraphics format isn't great for manual installations without the aid of a program
How so?

Editing ini files is, in my limited experience, far beyond the comfort level of most users for some reason, even if the change is simple. My understanding it that the Starter Pack/Lazy Newb Pack themselves originated to help users avoid this, not just as "DFHack and related utilities conveniently pre-packaged". Even Masterwork includes an elaborate GUI at this point. I have no direct evidence to back this, but I expect that if my download was restructured to require users to change the tileset in init.txt to use the alternates, hardly anybody would do so. As it is, they just have to copy and paste.

For example, I put some alternate dwarf graphics in my pack after I read some discontent that I removed beards and include helpful instructions on how to enable them in the graphics files, but I'm reasonably sure that hardly anybody that downloads my pack has even noticed them.
Title: Re: What's the scope of DFGraphics
Post by: CLA on May 23, 2016, 03:58:08 pm
[...]

I agree, but that's not the case with the dfgraphics repo. At least not with mine, maybe I'm doing something wrong. The init file that comes with the download already contains the references to the CLA tileset and has GRAPHICS set to YES. You only need to download the "source code" and drop it into a fresh DF install.
Title: Re: What's the scope of DFGraphics?
Post by: Rydel on May 23, 2016, 04:59:57 pm
These packs are being maintained by people who don't/no longer use GitHub:
...
  • Rally-Ho (http://www.bay12forums.com/smf/index.php?topic=149172.0)

I do use GitHub.  I just apparently misunderstood how this was linked the the original repo - I thought it was a clone that would update automatically.  I've pushed the missing changes to this version and shouldn't have issues in the future.
Title: Re: What's the scope of DFGraphics
Post by: Taffer on May 23, 2016, 05:53:15 pm
I agree, but that's not the case with the dfgraphics repo. At least not with mine, maybe I'm doing something wrong. The init file that comes with the download already contains the references to the CLA tileset and has GRAPHICS set to YES. You only need to download the "source code" and drop it into a fresh DF install.

You're not doing anything wrong: I just have a lot of tileset options, which need to be set in init.txt (if you use the DFGraphics repository)
Title: Re: What's the scope of DFGraphics
Post by: PeridexisErrant on May 23, 2016, 06:52:42 pm
I agree, but that's not the case with the dfgraphics repo. At least not with mine, maybe I'm doing something wrong. The init file that comes with the download already contains the references to the CLA tileset and has GRAPHICS set to YES. You only need to download the "source code" and drop it into a fresh DF install.

You're not doing anything wrong: I just have a lot of tileset options, which need to be set in init.txt (if you use the DFGraphics repository)

Check out how the Phoebus repo does this - a default tileset (etc) is set in init.txt, and all the others are in data/art.  You can install just by drag-and-drop, and then change the init if you don't like the default tiles.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on May 24, 2016, 01:09:57 am
You can install just by drag-and-drop, and then change the init if you don't like the default tiles.

Taffer doesn't think most players would be comfortable with editing the init files, so he prepares several copies of his tileset each with its own init files. It looks like a third of Taffer users use the LNP. The rest of Taffer users prefer to play on the latest version of Dwarf Fortress.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on May 25, 2016, 10:07:48 pm
     Here's an updated breakdown of packs in the DFGraphics repo:

These packs are "orphans" and are only being maintained in the DFGraphics repo:
       
Pack Name
Last Updated
Afro (https://www.reddit.com/r/dwarffortress/comments/2dy7lo/afros_graphics_pack_13pc_dual_release_edition/)  5 days ago (Jecowa)
Obsidian (http://www.bay12forums.com/smf/index.php?topic=126934.0)  5 days ago (Jecowa)
Phoebus (http://www.bay12forums.com/smf/index.php?topic=137096.0)  5 days ago (Jecowa)

These packs are being maintained on the DFGraphics repo by their original or current authors:
       
Pack Name
Last Updated
Rally-Ho (http://www.bay12forums.com/smf/index.php?topic=149172.0)  2 days ago (Rydelfox)
Spacefox (http://www.bay12forums.com/smf/index.php?topic=129219.999999)  4 days ago (Nopal-Dwarf)
CLA (http://www.bay12forums.com/smf/?topic=105376.0)  13 days ago (claireanlage)
AutoReiv (http://www.bay12forums.com/smf/index.php?topic=152432.0)  15 days ago (AutoReiv)
Taffer (http://www.bay12forums.com/smf/index.php?topic=107924.0)  soon (Taffer)

These packs are being maintained by people who don't/no longer use GitHub (sorted by activity level):
       
Pack Name
Last Update
for Version          Notes
Ironhand (http://www.bay12forums.com/smf/index.php?topic=53180.999999)2016-05-22 (GoldenShadow) v0.43.03
Mayday (http://www.bay12forums.com/smf/index.php?topic=137370.999999)2016-04-07 (Tallcastle) & 2016-05-13 (Habbadax) v0.43.03
GemSet (http://www.bay12forums.com/smf/index.php?topic=150753.0)2016-01-15 (DragonDePlatino) v0.42.04Modifies the most objects files.
Wanderlust (http://www.bay12forums.com/smf/index.php?topic=145362.0)2015-12-10 (Kynsmer) v0.43.03Only copy "inorganic_stone_soil" and not any objects or init files. (It only need to modify that one objects file.)
Tergel (http://www.bay12forums.com/smf/index.php?topic=145802.0)2015-12-06 (CowThing) v0.43.03Does not modify any objects files.

This pack's author is very inactive, but it only edits a single objects file:
       
Pack Name
Last Updated
for Version          Notes
Jolly Bastion (http://www.bay12forums.com/smf/index.php?topic=104261.0)2012-05-21 (AlexanderOcias) v0.34.10Only modifies "plant_standard" and "d_init".
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on May 25, 2016, 10:36:12 pm
Habbadax (Mayday maintainer) expressed interest and has a pending invite, but is not yet on Github.  I'll invite Tallcastle too.  Will also invite Kynsmer (re: Wanderlust). 

Jolly Bastion doesn't need a dedicated maintainer -- luckily!

And @Jecowa, thanks heaps for stepping up and helping out with this :D

Title: Re: Dwarf Fortress graphics repositories
Post by: Taffer on May 27, 2016, 01:08:43 pm
Don't multi-racial forts imply that all races need all jobs in their graphics definitions, or am I missing something? This is how I've been distributing my own sets for some time. I'm re-basing my graphics text files on Rally-Ho's (https://github.com/DFgraphics/Rally-Ho) graphics text files because they're clean and nicely formatted, but the goblins, for example, are missing almost every non-military job. Many graphics sets have this same problem from what I've seen, although I haven't looked at all of them. I have nice goblin and kobold monarch graphics, for example, but they'd never show according to these RAW files.

Information on these issues would please be greatly appreciated. In the past I just copied and pasted all of the dwarf definitions to the elves, humans, goblins, and kobold files. Ideally, I'd just like to sync my own graphics files (apart from filenames and TILE_PAGE names, of course) with Rally-Ho or something, but these files don't appear complete to me. (Or I have graphics that can't ever be used by the game, like my goblin/kobold monarch tiles).

When goblins end up ruling a dwarven civilization in the current version, do they just revert to a "g" with most graphics packs? Do they revert to using the default definition ([DEFAULT:RALLYHO_GOBLIN:0:0:AS_IS:DEFAULT])? These files have been a sore point for me for a while, for that matter: while documentation exists, when I last looked at these files it was scattered and most graphics files were/are half-filled with commented out graphics definitions and odd formatting/spacing issues, making it difficult to decipher them. It doesn't help that I haven't played Dwarf Fortress with graphics enabled--apart from brief testing--in a long time.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on May 27, 2016, 01:53:04 pm
Don't multi-racial forts imply that all races need all jobs in their graphics definitions, or am I missing something?

Yes, you can define profession tiles for all the intelligent races now. I think most of them should be able to hold most positions. (Kobolds are too dumb to have professions though, except perhaps in mods. Meph might know more about that.)

 
This is how I've been distributing my own sets for some time. I'm re-basing my graphics text files on Rally-Ho's (https://github.com/DFgraphics/Rally-Ho) graphics text files because they're clean and nicely formatted, but the goblins, for example, are missing almost every non-military job. Many graphics sets have this same problem from what I've seen, although I haven't looked at all of them. I have nice goblin and kobold monarch graphics, for example, but they'd never show according to these RAW files.

It's a lot of work to draw tiles for everything, so drawing professions for all the Goblins might have been a lower priority. CLA released a graphics pack template earlier that might have some info for goblin monarchs. http://dffd.bay12games.com/file.php?id=12090


 
Help would please be greatly appreciated cleaning my files up, if anybody's willing. In the past I just copied and pasted all of the dwarf definitions to the elves, humans, goblins, and kobold files. Ideally, I'd just like to sync my own graphics files (apart from filenames and TILE_PAGE names, of course) with Rally-Ho or something, but these files don't appear complete to me. (Or I have graphics that can't ever be used by the game, like my goblin/kobold monarch tiles).

I'm willing to help with stuff. Are you making a bunch of creature graphics for your tileset?

When goblins end up ruling a dwarven civilization in the current version, do they just revert to a "g" with most graphics packs? Do they revert to using the default definition ([DEFAULT:RALLYHO_GOBLIN:0:0:AS_IS:DEFAULT])?

From what I've read (http://www.bay12forums.com/smf/index.php?topic=155882.msg6781874#msg6781874), yes, that's what happens.
Title: Re: Dwarf Fortress graphics repositories
Post by: Taffer on May 27, 2016, 02:13:42 pm
I'm willing to help with stuff. Are you making a bunch of creature graphics for your tileset?

Thank you kindly for the answers and for the offer. I'm not making any new creature graphics: all of my graphics consist of:

 • A standard/default creature tile
 • A military tile
 • A monarch/ruler tile
 • A skull/dead body tile (which only shows up on zombies of course, as I don't rely on TWBT)

These are just for dwarves, humans, elves, goblins, and kobolds, which I perceive--perhaps wrongly-- to be the set of "intelligent races".

EDIT: I'm an idiot.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on May 27, 2016, 03:09:55 pm
I'm pretty sure that if there's not a match, it falls back to the default graphic.

Races like Goblins don't have a full sheet yet since I'm working on another, unrelated project at the moment.  Once that's finished, I plan to expand them to a full sheet.
Title: Re: Dwarf Fortress graphics repositories
Post by: Taffer on May 27, 2016, 03:34:41 pm
I'm pretty sure that if there's not a match, it falls back to the default graphic.

Races like Goblins don't have a full sheet yet since I'm working on another, unrelated project at the moment.  Once that's finished, I plan to expand them to a full sheet.

Apologies: I wasn't asking why art wasn't available, I was operating under a faulty premise that undefined creatures don't get graphics. Really, I shouldn't have posted at all.

I'm almost done finishing my graphics files at present, as soon as I fix some issues. A few things I've noticed while syncing my files to (portions) of Rally-Ho's files:

The first isn't in the game anymore, and the rest are either typos or don't appear in wiki.

   [DUNGEONMASTER:TAF_D:1:0:ADD_COLOR:DEFAULT]
   [DUNGEON_MASTER:TAF_D:1:0:ADD_COLOR:DEFAULT]
   [DUNGEON_KEEPER:TAF_D:1:0:ADD_COLOR:DEFAULT]
   [DUNGEON_LORD:TAF_D:1:0:ADD_COLOR:DEFAULT]
   [DUNGEON_NASTER:TAF_D:1:0:ADD_COLOR:DEFAULT]

EDIT: I'm an idiot.
Title: Re: Dwarf Fortress graphics repositories
Post by: Taffer on May 27, 2016, 04:05:52 pm
Sorry for the noise and confusion. I blame stress and lack of coffee. I just committed an update, bringing it in-line with Taffer 31.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on May 27, 2016, 04:50:36 pm
Don't worry, I too am suffering from stress and lack of coffee.

Since I used Phoebus when designing the sprite sheets, there were some defunct jobs on there.  I put them in since they didn't take much time, and it will be useful if those jobs are brought back later or used in mods.  I'm not sure why the different naming schemes are there for them.

TLDR: Those are there because Phoebus had them.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on May 28, 2016, 12:29:24 am
I'm not sure why your Goblin tile kind isn't working, Taffer. I was comparing your Goblin definitions file with the one CLA posted. Here's somethings from Quiet-Sun's graphics standard that you may want to add:

Monarchs & High Priests:
  MONARCH_CONSORT   (You have King_Consort define, so you might want this defined too just in case.)

Officers:
  HAMMERER   (You have executioner defined, so this one might be good to have too.)
  CAPTAIN_OF_THE_LAW_ENFORCE
  RANGER_CAPTAIN

I'm not sure if you need to declare law_enforcement and tax_collector variants of the military, but I guess it doesn't hurt just to be sure.
Title: non-TWBT GemSet
Post by: jecowa on June 02, 2016, 01:07:31 pm
I was going to make GemSet usable without TWBT. The current issue with it is that most of the standard alphabetical characters are being used for something other than letters (corpses, I believe). My plan is to reassign anything assigned to those characters to a single pile of bones tile using tile 255 (as recommended here (http://www.bay12games.com/dwarves/mantisbt/print_bug_page.php?bug_id=3197)).

I think it might be best to add a new tilesheet to the art folder and make an "objects (non-TWBT)" folder that can be swapped out for anyone wanting to use GemSet without TWBT. I'll put comments next to any lines changed for non-TWBT compatibility to make it easier to keep the two objects folders in-sync with each other.

Or would it be better to have the non-TWBT version of GemSet in its own branch or fork or something?

Edit: Is there any reason that all the GemSet corpses aren't in their own overrides tilesheet? If it's possible to put corpses in an overrides tilesheet, that seems like it would be the ideal way to make GemSet work both with and without TWBT.
Title: Re: non-TWBT GemSet
Post by: jecowa on June 02, 2016, 08:08:16 pm
Maybe use 4 corpse tiles for a non-TWBT version. There's a few unused tiles in GemSet.

From left-to-right: dead dwarf, dead other humanoid, dead other creature, dead titan/forgotten beast.
Spoiler: corpses (click to show/hide)

Maybe that's too many tiles to spend on corpses, though.
Title: Re: non-TWBT GemSet
Post by: Dirst on June 02, 2016, 08:14:04 pm
Maybe use 4 corpse tiles for a non-TWBT version. There's a few unused tiles in GemSet.

From left-to-right: dead dwarf, dead other humanoid, dead other creature, dead titan/forgotten beast.
Spoiler: corpses (click to show/hide)

Maybe that's too many tiles to spend on corpses, though.
I think just a single Jolly Roger would suffice, though a special one for dwarves with a beard might be nice.  This probably isn't the thread to discuss it, though.
Title: Re: Dwarf Fortress graphics repositories
Post by: Taffer on June 03, 2016, 10:12:25 am
Please continue using TWBT as much as possible, even if things lag behind. I'm happy to keep my DFGraphics repository in line with the current version that DFHack supports, rather than the current version. I know that increased community reliance on DFHack and TWBT is an odd request coming from somebody who never uses them, but there's just far too many problems with the default graphics engine and by his own admittance Toady has no interest in fixing them, nor does he know how. (Also, almost every graphics bug that I've seen in the bug tracker is flagged "Minor", and barely considered worth fixing).

This approach will turn every creature into a corpse whenever they're out of sight (yet still onscreen) in Adventure mode and also whenever they're engraved anywhere. Probably other places that I'm forgetting. It's not going to be fun playing adventure mode and never being certain what anything is, because everything looks like a corpse until you get close to it. This is besides all of the art DragonDePlatino added that requires TWBT.

TWBT is the best thing that's happened to the graphics packs in a long time, and I would truly hate to see things regress to not using it. Any feature or bug fix added by a version newer than DFHack supports isn't worth the ugliness and problems introduced by removing TWBT (for the sets that rely on it). I know you're not planning on removing the TWBT version, only adding a non-TWBT variant, but in my opinion you're taking away too much to make the branch worth it.

In my personal opinion, the only graphics sets that don't need to rely on TWBT are the almost-ASCII variants like mine, where turning off the graphics doesn't affect the aesthetic and creature graphics aren't used much. (And perhaps CLA, who specifically drew creature graphics knowing these limitations, but that's up to CLA).
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 06, 2016, 12:44:51 am
Please continue using TWBT as much as possible, even if things lag behind. I'm happy to keep my DFGraphics repository in line with the current version that DFHack supports, rather than the current version.
PeridexisErrant said it would be better to update for the current version of Dwarf Fortress and use releases to support older versions. By keeping the graphics packs up with the latest version, it should getting a version that supports TWBT faster once it gets updated for the latest Dwarf Fortress. Do you remove old profession tokens from your pack? I don't think there's a problem with leaving them all in if you want your pack to work with every version of Dwarf Fortress.

I know that increased community reliance on DFHack and TWBT is an odd request coming from somebody who never uses them, but there's just far too many problems with the default graphics engine and by his own admittance Toady has no interest in fixing them, nor does he know how. (Also, almost every graphics bug that I've seen in the bug tracker is flagged "Minor", and barely considered worth fixing).
Yeah, I get the feeling that he's not too fond of graphics packs.

This approach will turn every creature into a corpse whenever they're out of sight (yet still onscreen) in Adventure mode and also whenever they're engraved anywhere. Probably other places that I'm forgetting. It's not going to be fun playing adventure mode and never being certain what anything is, because everything looks like a corpse until you get close to it. This is besides all of the art DragonDePlatino added that requires TWBT.
Thanks for the explanation. I had heard about this bug before. Someone said he heard that the bug with creatures turning into their letters when out-of-sight might have been the inspiration for the design of CLA's graphics pack – by drawing the creatures as letters, they still look about the same when they transform into letters. (I was googling trying to find that thread, but couldn't come up with anything.) That's clever how DragonDePlatino dealt with the issue of corpses turning into letter. It seems like CLA's solution was designed to work without TWBT, though, making it great for adventurers who wish to run the latest version.

I didn't realize that corpse tiles were used in unobscured engravings; that explains why GemSet's corpse tiles look like they do.

TWBT is the best thing that's happened to the graphics packs in a long time, and I would truly hate to see things regress to not using it. Any feature or bug fix added by a version newer than DFHack supports isn't worth the ugliness and problems introduced by removing TWBT (for the sets that rely on it). I know you're not planning on removing the TWBT version, only adding a non-TWBT variant, but in my opinion you're taking away too much to make the branch worth it.
I don't think a non-TWBT GemSet would have to look ugly. I think it'd be best to start with gemset_text.png for the main tilesheet and use CLA, Grim Fortress, Tergel, and the tileset wiki page (http://dwarffortresswiki.org/Tilesets) as guides on where graphics should be placed. I think Tergel does a good job of drawing things that look like they could represent more than one thing. CLA and Grim Fortress both place graphics in the main tilesheet conservatively, and they all make few to no edits to the objects files.

GemSet is the second highest-resolution graphical tileset after DawnFortress 32x32. I think it'd be nice to have GemSet as an option for users who want a graphics pack but who don't want to use TWBT / DFHack.

In my personal opinion, the only graphics sets that don't need to rely on TWBT are the almost-ASCII variants like mine, where turning off the graphics doesn't affect the aesthetic and creature graphics aren't used much. (And perhaps CLA, who specifically drew creature graphics knowing these limitations, but that's up to CLA).
I think it's just fine to have lots of creature graphics without using TWBT. Most players (maybe about 61%) don't use TWBT. Of those players, about 45% use a graphics pack that contains lots of creature graphics. Most graphics packs work well enough without TWBT; it looks like DungeonSet and GemSet are the only ones that really require it at the moment.

I think it's best if new players start playing without DFHack, but ASCII can be intimidating for many users. Even for someone coming from Dungeon Crawl Stone Soup, the large variety of ASCII in Dwarf Fortress is kind of overwhelming at first. Graphics packs make getting into Dwarf Fortress easier for new players. And by not starting out new players out on DFHack and TWBT, they won't become reliant on them from the start and will make a future transition into vanilla easier for them, if they ever decide they'd like to try it.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 12, 2016, 04:07:43 am
The trick GemSet is using for corpses looks pretty good. I imagine it would be good to use it in the other graphical packs too. Until TWBT offers a way to include both TWBT and vanilla versions of object files and possibly init files in the same pack, what would be the best way to maintain TWBT and non-TWBT versions of a graphics pack? I'm guessing using branches would be the appropriate option for this.
Title: Re: Dwarf Fortress graphics repositories
Post by: Thundercraft on June 12, 2016, 04:26:23 pm
I'm happy to keep my DFGraphics repository in line with the current version that DFHack supports, rather than the current version.
I think it's best if new players start playing without DFHack...

Why suggest that newbies try to get into DF without DFHack? As long as the installation is hassle-free, it can just sit in the background, right? It comes with Stonesense and I think that, alone, can be a big help getting into DF.

I think some players do not appreciate how much certain Utilities and Mods rely on either DFHack or the memory offsets that team finds, or at least underestimate how many players use DFHack in some way. For instance, some players can't imagine playing DF without Dwarf Therapist.

Anyway, all the Starter Packs still come with DFHack. Those are still recommended to newbies, aren't they?

...there's just far too many problems with the default graphics engine and by his own admittance Toady has no interest in fixing them, nor does he know how. (Also, almost every graphics bug that I've seen in the bug tracker is flagged "Minor", and barely considered worth fixing).

Good points.

PeridexisErrant said it would be better to update for the current version of Dwarf Fortress and use releases to support older versions. By keeping the graphics packs up with the latest version, it should getting a version that supports TWBT faster once it gets updated for the latest Dwarf Fortress.

That makes sense, as long as maintainers try not to skip over an older version that DFHack is currently still limited to (or erase/overwrite their older versions).

Please continue using TWBT as much as possible, even if things lag behind.
I think it's just fine to have lots of creature graphics without using TWBT. Most players (maybe about 61%) don't use TWBT. Of those players, about 45% use a graphics pack that contains lots of creature graphics.

I've been away from DF for a while and I only recently heard about TWBT. So I haven't tried it, yet. And, actually, while it looks and sounds great, I'm a bit hesitant...

...And by not starting out new players out on DFHack and TWBT, they won't become reliant on them from the start and will make a future transition into vanilla easier for them, if they ever decide they'd like to try it.

Never! ASCII is not for me. Graphics packs and sprites all the way! :D

That said, I can see how TWBT might not be well-suited for newbies. At least, I would think it would make things a bit more confusing, esp. when trying to follow the wiki or follow along with a video tutorial. (Multi-tile sprites come to mind...)

...but ASCII can be intimidating for many users. Even for someone coming from Dungeon Crawl Stone Soup, the large variety of ASCII in Dwarf Fortress is kind of overwhelming at first.

ASCII is a huge turnoff for many. This was true years ago when DF was really starting to catch on. But this is even truer today, since gamers more-or-less expect flashy graphics, if not 3D. Gamers who have some experience with ASCII roguelikes are much more likely to give DF a try. But I suspect many of that crowd have already tried DF.

ASCII roguelikes don't seem as popular as they used to be. Some of them have been abandoned. And some of the most popular roguelikes are often played with graphical or isometric front-ends these days. I've read comments on Dwarf Fortress reviews and threads on reddit where a lot of gamers are saying the same things: The graphics is too primitive and its too indimidating to get into.

In contrast, I think a lot of the same gamers are willing to give Gnomoria and similar clones a try because they look a lot less intimidating and a lot more modern.
 
Indeed, I've played ASCII roguelikes before and I thoroughly enjoyed them - even without graphic front-ends. Even so, I never would have given DF a try had I not seen videos with Stonesense and graphics packs.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 13, 2016, 02:14:53 am
Why suggest that newbies try to get into DF without DFHack? As long as the installation is hassle-free, it can just sit in the background, right? It comes with Stonesense and I think that, alone, can be a big help getting into DF.

I think some players do not appreciate how much certain Utilities and Mods rely on either DFHack or the memory offsets that team finds, or at least underestimate how many players use DFHack in some way. For instance, some players can't imagine playing DF without Dwarf Therapist.

Anyway, all the Starter Packs still come with DFHack. Those are still recommended to newbies, aren't they?

I would recommend that new players try out vanilla a bit before turning on DFHack. It's nice to know what features are vanilla and what features are added by mods. This will make it easier for them to know where to look up information when they have trouble with something.

Most starter packs come with DFHack, but I think it should be turned off by default. Some players may want to be able to play the latest version after they've played with a Lazy Newb Pack for a bit, but it will probably be harder for them to adjust if they have become accustomed to all the features and graphics of modded Dwarf Fortress.

Stonesense seems like a more advanced feature. It looks like you have to pull up a terminal window and type in some commands to enable it. That doesn't seem like something intended for beginners. I've never used it before, though.

That makes sense, as long as maintainers try not to skip over an older version that DFHack is currently still limited to (or erase/overwrite their older versions).

My favorite thing about GitHub is how well it organizes older versions. Any older version of a project on GitHub can be downloaded. PeridexiErrant mentioned that GitHub even allows for particular versions of a project to be flagged as a release, which will allow us to mark an old version as being the recommended version to use for a particular version of Dwarf Fortress.

I wish more projects were on GitHub.

I've been away from DF for a while and I only recently heard about TWBT. So I haven't tried it, yet. And, actually, while it looks and sounds great, I'm a bit hesitant...

If you decide to use TWBT, you might consider turning off multilevel view. It allows you to see more Z-levels at once, but some users have reported that Dwarf Fortress seems to crash more often with it turned on. Note that most packs have it enabled by default against the recommendations of its author. It seems to be tricky to disable. There's going to be one to three init files that have to be edited to disable it; you have to delete the line that says something like "multilevel 5" or "multilevel 6". The names of the files are called something like "onLoad.init", "DFHack.init", "onWorldLoad.init". They can be found in the Dwarf Fortress folder, the "raw" and "objects" folders, and even in graphics packs (The GemSet graphics pack, for example, includes one).

It's nice to be able to see more than one Z-level at once, but it can make it kind of confusing to determine which level the cursor is on.
Title: Re: Dwarf Fortress graphics repositories
Post by: mifki on June 15, 2016, 06:53:57 am
It's nice to be able to see more than one Z-level at once, but it can make it kind of confusing to determine which level the cursor is on.

I remind that for some tilesets, fog colour/density and shadow colours may need to be adjusted. I only use Spacefox tileset and the default parameters are tweaked for it. Other tilesets are usually darker and indeed with the default parameters it's hard to distinguish between z-levels.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 15, 2016, 09:25:54 am
I remind that for some tilesets, fog colour/density and shadow colours may need to be adjusted. I only use Spacefox tileset and the default parameters are tweaked for it. Other tilesets are usually darker and indeed with the default parameters it's hard to distinguish between z-levels.

Thanks for the help. More fog sounds like what I need until I get more used to it. I remember seeing an old screenshot and wishing it could still look like that.
Spoiler: old TwbT example (click to show/hide)

But maybe a change to the fog color would also make it much more obvious to me.
Title: Re: Dwarf Fortress graphics repositories
Post by: mifki on June 15, 2016, 04:17:32 pm
I remind that for some tilesets, fog colour/density and shadow colours may need to be adjusted. I only use Spacefox tileset and the default parameters are tweaked for it. Other tilesets are usually darker and indeed with the default parameters it's hard to distinguish between z-levels.

Thanks for the help. More fog sounds like what I need until I get more used to it. I remember seeing an old screenshot and wishing it could still look like that.
Spoiler: old TwbT example (click to show/hide)

But maybe a change to the fog color would also make it much more obvious to me.

There are also fog start and step parameters (https://github.com/mifki/df-twbt#multi-level-rendering), so you can configure bigger difference between z-levels (with step), or bigger difference between the current z-level and the first z-level below (with start).
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 24, 2016, 11:56:37 am
Is it the eventual plan for this repo to have PyLNP use it to auto update its tilesets? If so, is there any special format for commit messages or something we should use to make it readable by PyLNP?
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on June 25, 2016, 10:36:46 am
No, there are no plans for auto-updates.

I've written a tool that downloads the latest version, but that simply operates on the latest tag/release (so be sure to hit the 'release' button on GitHub when it's ready).

Best practice for git commit messages is pretty easy to google up; basically a short one-line summary, then a paragraph explaining in more detail if required (usually not, for this project).
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 25, 2016, 11:47:18 am
I've been watching starter-pack.log (http://peridexiserrant.neocities.org/starter-pack.log.txt) trying to figure it out. I was starting to think that it didn't update things automatically. I flagged a commit on Phoebus for DF v0.43.03, but nothing seemed to happen.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 27, 2016, 10:03:23 pm
Is it okay to update Rally-Ho on the repo for DF v0.43.03, or should I send a pull request?

Just line 62 of inorganic stone layer needs updating (to remove the [REACTION_CLASS:FLUX] bit).
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on June 27, 2016, 10:55:28 pm
I can update it tomorrow directly.  I've just been a bit busy on another project.

Edit: Updated.

Also: Rally Ho! uses separate plant raws for Text Will Be Text.  This can cause some unwanted behavior if the user isn't using TWBT.  Currently, I have a separate download on DFFD without the plant raws and with a different init.ini file, but I'm not sure of the best way to handle it for the GitHub repo.  I put in instructions for non-TWBT users, but if someone's using it through LNP, they likely won't see that file.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on June 28, 2016, 05:39:47 pm
Do you think it's okay to update AutoReiv's pack on the repo? He just needs a few config settings added to his d_init.txt for compatibility with Dwarf Fortress v0.42.01 through v0.43.04.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on June 28, 2016, 07:36:58 pm
Of course it's fine, the fact that anyone who has time can do all the basic updates is practically the whole point of the DFgraphics project and group!

Other goals were added later, but even they boil down to 'make that as easy as possible'.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 28, 2016, 09:37:35 pm
Unless HaterSkater is against it, is it okay to add Duerer to the repo? It doesn't look like he had a problem with fricy having his own distribution of Duerer, since he's still linking to it in his OP.

While the non-TWBT version still works fine, the TWBT version of Duerer could use a little work. I kind of thought this pack would already be on the repo, since I think the DFGraphics repo originated as fricy's graphics repo. Maybe Duerer was removed at some point? It's listed in the OP as being one of the packs maintained here. Maybe I'm just overlooking it?
Title: Re: Dwarf Fortress graphics repositories
Post by: burned on July 29, 2016, 12:58:19 am
. . . I kind of thought [Duerer] would already be on the repo . . .
. . . Maybe Duerer was removed at some point?
. . . Maybe I'm just overlooking it?

Based on what I understand about the LNP (and this post (https://github.com/DFgraphics/DFgraphics/pull/14)), it wasn't so much the character tileset versus creature tileset distinction that concerned him, so much as it came down to "Does this set need modified graphic raws?" If the answer was no, he removed the set from the graphics repo.

In other words, I don't believe PE removed Duerer from his LNP, so much as "a graphics pack must contain graphics raws, or be one of the two ASCII special cases" when it comes to the repo.

Just a guess, since Duerer is a character tileset and not a creature tileset.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 29, 2016, 01:41:37 am
That's interesting. Maybe this explains why I can never remember which graphics packs are in the repo. I keep thinking Grim Fortress is in there. It seems that the default Curses graphics were in the repo at one point too. It looks like these are the ones that were removed:


It looks like Jolly Bastion was one of the two special cases. I guess it got to stay because it modifies an objects file. That why I was wanting Duerer on here, because it modifies 3 objects files to assign custom tiles.

I wonder what the other special case was. Maybe AutoReiv didn't have racial graphics at the time? I guess Tergel didn't have racial graphics at the time either. Maybe this is why he added them.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on July 29, 2016, 02:22:23 am
As I recall it was just the issue of graphics raws - but I have no objection to packs with object raws if someone is maintaining them.

Really its just that a little extra has to be drawn between tiles eland graphics pack *somewhere* (too many of the former and they need no maintenance), and this seemed like a good idea at the time. Now that there are multiple members, I'd be happy for someone else to decide :)
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 29, 2016, 06:21:26 am
I was wanting to provide updates for Duerer 0.6.A (http://dffd.bay12games.com/file.php?id=10007) on the repo. You would have to initialize it, though, since you're owner. It shouldn't be to hard to keep updated with just three objects files.
Title: d_init.txt and init.txt settings
Post by: jecowa on July 29, 2016, 12:59:23 pm
I've noticed that some packs in the repo change some settings from their defaults.


I'm guessing these are being changed to the settings that fricy liked to play with. I think that these all should be set back to default, though.

1. It's not really the business of a graphics pack to change settings completely unrelated to graphics.

2. The Lazy Newb Pack doesn't allow graphics packs to modify these settings anyway.

3. For users not using a Lazy Newb Pack who are using custom settings in their d_init.txt or init.txt, they will probably expect that those will get reverted back to default settings when they install a graphics pack. But users using vanilla settings in those files will probably be confused when they stop getting migrants or when the sound turns off.

I'm not seeing an advantage to these settings being set to something other than default. Even if the custom settings are better, I don't think this is the way to go about getting people to change their settings. I've been switching the above listed settings back to their defaults when updating packs in the repo.



There's a few settings, though, that perhaps graphics packs should be able to modify, but I'm not completely sure about:
Do you think graphics packs should be allowed to modify any of those? Currently the Lazy Newb Pack doesn't allow graphics packs to modify any of those three. Should a graphics pack artist have any control over any of those?
Title: Re: d_init.txt and init.txt settings
Post by: Taffer on July 29, 2016, 01:56:25 pm
  • engravings start obscured
  • varied ground tiles
  • show flow amounts
Do you think graphics packs should be allowed to modify any of those? Currently the Lazy Newb Pack doesn't allow graphics packs to modify any of those three. Should a graphics pack artist have any control over any of those?

I don't believe I modify any of that (let me know if I do) and I agree that graphics packs shouldn't modify the settings you specified. Almost all of them are clearly personal preferences and have nothing at all to do with the graphics: with a few exceptions, which I shall detail below.

I think it's important to exempt the following:

 • intro,
 • engravings start obscured, and
 • varied ground tiles.

In my opinion, all of the above look pretty lousy with the default settings for any graphics pack or tileset that isn't an ASCII lookalike (most of them). In addition, several sets are drawn with the assumption that varied ground tiles are turned off, so reverting that breaks visual consistency. It would be a mistake in my opinion to revert any of these settings for graphics sets that weren't meant for them.

You didn't mention it and it's not obvious, but I find that zooming looks much better for my set with texture_param set to nearest, so I do think that setting is also acceptable to modify.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 29, 2016, 03:33:09 pm
I don't believe I modify any of that (let me know if I do) and I agree that graphics packs shouldn't modify the settings you specified. Almost all of them are clearly personal preferences and have nothing at all to do with the graphics: with a few exceptions, which I shall detail below.

Yours is good. I noticed on the DFGraphics repo that your first edit was setting all those back to default.

I think it's important to exempt the following:

 • intro,
 • engravings start obscured, and
 • varied ground tiles.

In my opinion, all of the above look pretty lousy with the default settings for any graphics pack or tileset that isn't an ASCII lookalike (most of them). In addition, several sets are drawn with the assumption that varied ground tiles are turned off, so reverting that breaks visual consistency. It would be a mistake in my opinion to revert any of these settings for graphics sets that weren't meant for them.

Maybe we can get "engravings start obscured" and "varied ground tiles" whitelisted in the Lazy Newb Pack. I don't think odd-looking intros are a very big deal.

You didn't mention it and it's not obvious, but I find that zooming looks much better for my set with texture_param set to nearest, so I do think that setting is also acceptable to modify.

I think I like "Nearest" scaling best too. It doesn't work with "2D" rendering, though. It will work with "Standard" and maybe "TWBT", though. But the TrueType font feature only works with "2D".

Currently the Lazy Newb Pack already allows graphics packs to modify TrueType Fonts, Print Mode, and Texture_Param (Scaling Method). You can see the full white list by going to the graphics.py (https://bitbucket.org/Pidgeot/python-lnp/src/6dd574e25f773638a3a086fc4b6d480323e4f238/core/graphics.py?at=default&fileviewer=file-view-default) page and searching for "_fields = [" without the quotes.

Edit: Oh, it looks like graphics packs actually aren't allowed to change the texture_param.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on July 29, 2016, 07:29:52 pm
Is suspect that most packs that change these (not sure if mine does) come from the author basing it off their own init file, forgetting that they'd made modifications to it.

Does Rally Ho! change any of those?
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 29, 2016, 07:36:45 pm
Is suspect that most packs that change these (not sure if mine does) come from the author basing it off their own init file, forgetting that they'd made modifications to it.

Does Rally Ho! change any of those?

No, it doesn't change any of those. It makes engravings start obscured, but I think that's an okay thing to do.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on July 30, 2016, 03:46:55 am
In response to the init field question:  on the PyLNP end, I've opened this issue (https://bitbucket.org/Pidgeot/python-lnp/issues/119/) to have the launcher handle more - I think this is now the full list of graphics-related init fields.

On the DFgraphics side of things, once I get to writing some automated tests it should be easy to ensure that we only change whitelisted fields.  Tests and better documentation have been planned for a long time... but they're moving up my todo list, now that the pace of updates has slowed and I've written PyLNP a new-pack-from-old importer.
Title: Re: Dwarf Fortress graphics repositories
Post by: Taffer on July 30, 2016, 09:41:03 am
In response to the init field question:  on the PyLNP end, I've opened this issue (https://bitbucket.org/Pidgeot/python-lnp/issues/119/) to have the launcher handle more - I think this is now the full list of graphics-related init fields.

I don't have a bitbucket account, but I'm not sure it's necessary to modify SHOW_FLOW_AMOUNTS. Some tilesets draw 1-7 with water graphics with that setting in mind, but in every case disabling the setting doesn't disrupt the aesthetic as the artist also needs to draw the water tile. The majority of graphics packs that I've seen modifying that setting don't do anything special with 1-7 and the set actually looks better with it off. It's simply the artist's preference.

Still, I'm content to be overruled on this. I clearly have bias: I prefer the setting as default. If it does stay, perhaps it can be disabled for sets that weren't drawn with it in mind (1-7 appear as normal numbers).
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on July 30, 2016, 11:34:46 am
I'm in the same boat as Taffer on this, SHOW_FLOW_AMOUNT is definitely a personal preference for me, but contrary to other personal preferences (varied ground tiles for example), that doesn't have any impact on the tileset art or color scheme, and consequently, it's not necessary to modify it for my tileset.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 30, 2016, 02:26:07 pm
Taking off "show_flow_amounts" sounds good to me. I don't know of any tileset that tries to change that.

For the "varied_ground_tiles" setting, Grim Fortress and Jolly Bastion are the only tilesets I know of that try to disable that. I'm not very familiar with this stuff, but from looking at Jolly Bastion's tilesheet, it looks like it's designed to only use the period for the ground and not the comma.

I'm kind of curious about the "mouse_picture" setting. I don't think that feature currently does anything. There aren't any tilesets that try to change it, but GemSet includes a custom "mouse.png". But I'm pretty sure it only includes that to fulfill PyLNP's requirement that all graphics packs include both a "mouse.png" and "font.ttf".
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on July 30, 2016, 04:35:18 pm
Quote
PyLNP's requirement that all graphics packs include both a "mouse.png" and "font.ttf".

There was an issue with a library on OSX/Linux if I understood that correctly:
http://www.bay12games.com/dwarves/mantisbt/view.php?id=2688
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 30, 2016, 07:27:47 pm
There was an issue with a library on OSX/Linux if I understood that correctly:
http://www.bay12games.com/dwarves/mantisbt/view.php?id=2688

I guess this bug report is saying that it's a bug that Dwarf Fortress requires both a "mouse.png" and a "font.ttf" to run?

I was thinking PyLNP could leave behind the default "mouse.png" and "font.ttf" files in the art folder so that every graphics pack doesn't have to include them both. If a graphics pack did include a custom font or mouse, then they would overwrite the defaults when that graphics pack was copied over.

Or if there's a problem with that method, maybe PyLNP could check if there was a "mouse.png" and "font.ttf" in the art folder after installing a graphics pack and copy over those files from the baselines folder if needed.

Or how does it do it with the reduced raws format? It might be easiest to do it however it does it with that. That is already being used for the raw/objects/ and data/init/ folder, it probably wouldn't be too hard to expand it to work on those two files in the data/art/ folder.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on July 30, 2016, 11:19:59 pm
If you put those files in "LNP/Tilesets/", they can be omitted from graphics packs.

It should be pretty easy to pull them from the vanilla copy if missing, too - I'll make a pull request to add that to PyLNP.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on July 30, 2016, 11:34:16 pm
Thanks for the tip!
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on August 02, 2016, 12:43:13 pm
That would break the desirable property that copying graphics over a DF install correctly installs the graphics - we want them to be very simple to do manually or automatically.

An alternative approach (which I prefer) would be to add "twbt_data" and "raw/twbt_objects" folders, and copy them over the non-twbt pack if TwbT is enabled.  Regardless, any further discussion should happen in the DFgraphics thread, as it's not really about the Starter Pack.

That sounds good. I was thinking that they could share art folders, but it would be kind of nice to have separate art folders.

I did some testing to see how many extra MB all these duplicate folders would take in the DFHack Lazy Mac Pack.


At first glance it looks like duplicating the data folders of all graphics packs with TWBT overrides would take an extra 14.9 MB, but after removing all the TWBT overrides stuff it goes down to 9.9 MB, and then removing DejaVu Sans brings it down to 6.2 MB. Then compressed in a zip, it's only an extra 4 MB.

Duplicates of the raw/objects folder of TWBT graphics packs are an extra 13.1 MB. I'm really surprised that text files take up so much space. But they compress very well. They only take up 958 KB compressed in a zip file.

So in total, it would be like an extra 5 MB for the user to download.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on August 02, 2016, 06:42:17 pm
The trick would be to only retain twbt_X files that are different to the standard ones - I think that would cut the file size down some more.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on August 02, 2016, 06:53:31 pm
The trick would be to only retain twbt_X files that are different to the standard ones - I think that would cut the file size down some more.

That sounds good. So first it would install everything from the "data" and "raw/objects" folders, and then it would install everything from the "twbt_data" and "raw/twbt_objects" folders?
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on August 02, 2016, 08:42:53 pm
Yep, but the latter only when TwbT is enabled.

Supporting this isn't going to be a priority for me though - there's a lot to do still in documenting the current setup, writing tests, and maybe some tools for (semi-) automatic updates.
Title: Rectangular TWBT text fonts
Post by: jecowa on August 13, 2016, 07:32:30 am
TWBT allows for non-square fonts to be used in conjunction with square tilesets. The biggest advantage of non-square fonts is that they are easier to read. GemSet and DungeonSet both take advantage of this if you want to see an example. I'm working on one for Phoebus, and was planning on making one for Spacefox and Afro next, unless maybe Nopal would rather do it. Just something to consider.

If you want to make one, its tile height needs to be the same as the tile height of your main tilesheet, but the tile width should be more narrow. GemSet, for example, is a 24x24 tileset with a 16x24 font. It would probably look best to draw the text a little bit shorter on the TWBT text tilesheet so that there's a little more vertical space between the lines of text. I didn't do that with first draft for Phoebus, and wasn't happy with the way it looked. (seen in first spoiler below)

Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on August 26, 2016, 06:50:00 am
I'd tend to ship all the tilesets, and stick with the original sizes by default (for better small-screen and creature tile compatibility).  However we should probably take any references to tilesize out of the pack names, and maybe think about improving PyLNP support for variable-tilesize packs and a target tilesize.

Supporting multiple tile sizes sounds awesome! I guess it would need to swap out the graphics folder, the art folder, and the init.txt file?

There's several tilesets that can currently take advantage of this.(e.g. AutoReiv, Bisasam, Curses, DawnFortress, GeoDuck, Jolly Bastion, Taffer, Tergel, Wanderlust, and probably others). I'm also considering adding more TWBT text sizes to some of the more popular old ones (Phoebus, Spacefox, Ironhand, Mayday, Obsidian, and maybe others).
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 06, 2016, 10:25:05 am
PyLNP is adding a new line to the manifest.json file called "folder_prefix". I'm guessing we want to use it, but I'm fine with whatever.

Just put the name of your pack there. It supports spaces and punctuation (just tested it with "Rally Ho!").

If Nopal or AutoReiv are here, I've already added it to your packs. (I thought you two would probably be okay with that.)
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on September 06, 2016, 01:56:58 pm
What's it for?
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 06, 2016, 05:47:08 pm
The folder_prefix tells PyLNP which folder(s) to look in to find a graphics pack. I think it's supposed to allow a user to have two versions of a graphics pack installed at the same time and allow the folders to be renamed from, for example, "Phoebus" to something like "Phoebus 0.42.06" and "Phoebus 0.42.03". (That feature doesn't work on Mac, though. On Mac the folder_prefix has to be exactly the same as the folder name.)

In PyLNP v0.10f and v0.11, it was the job of the manifest title to determine what the folder name should start with. The "folder_prefix" change moves that functionality to its own field so that the manifest title and folder name don't have to have anything to do with each other. This allows users to have two different packs with similar titles, like "Geoduck 12x12" and "Geoduck 16x16", for example.

The "folder_prefix" field will probably make it easier to transfer a save file from one LNP to another while still being able to switch the graphics pack (if both LNPs are using the same folder_prefixes, that is.) This could be convenient for players of succession games.

If you don't put a folder_prefix in the manifest, then the LNP will identify the pack by it's folder name. This is how graphics pack were identified back in PyLNP v0.10e and earlier.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 27, 2016, 02:15:16 pm
I noticed Rally Ho getting some support for Fortress Defense (one of the most popular mods). Is it okay to start adding mod support to some of other graphics packs (i.e. Afro, GemSet, Ironhand, Mayday, Obsidian, Phoebus, and Spacefox)?
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on September 28, 2016, 02:49:43 am
I noticed Rally Ho getting some support for Fortress Defense (one of the most popular mods). Is it okay to start adding mod support to some of other graphics packs (i.e. Afro, GemSet, Ironhand, Mayday, Obsidian, Phoebus, and Spacefox)?

I've got mixed feelings about this actually - I like the idea of mod support, but here are some of the downsides:
My general impression comes out somewhere around "not worth it, given that mods can include the extras themselves".  The effort upstream would be better placed improving our formats or PyLNP's merge capability, to minimise the difficulty of managing "graphics extension packs" (a thing I just made up) or similar.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on September 28, 2016, 05:14:04 am
I'm against that and agree with PE's points.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 28, 2016, 07:38:33 am
Eventually it would be nice for the mods to be able to include their own graphics. Currently there are some downsides to doing this, though:
For now, the simplest way to add graphics support for mods is to include them in the graphics packs themselves.

Initially, I was wanting to add graphics for mods that include their own graphics. (Only ones I know of are Fortress Defense, The Earth Strikes Back, and Dwarven Haberdasher aka cloth armor mod.) This would add about 0.75 MB to each graphics pack, which would be an extra 5MB total for a LNP that includes 7 graphics packs.

Graphics support for mods would mostly be chosen like graphics support for vanilla stuff; it's decided by whoever is willing to draw stuff for it. Doren, for example, decided that Phoebus (and then many other packs) should have support for most vanilla creatures. Instead of deciding which mods should be supported, we should be keeping track of which mods are supported. This info would be helpful to users picking out which graphics pack and mod(s) they want to use.

There probably isn't any need to worry about conflicts. If two mods are both adding an alien_lizard_man or dark_elf, the same sprite will probably work just fine for both mods.

Mod support shouldn't make graphics packs take any longer to update for new versions of Dwarf Fortress. Mods aren't necessarily updating at the same time that Dwarf Fortress updates, and the graphics and graphics definition files don't need to be updated often anyway. Unless Bay12 renames creatures or whatever, updating a graphics pack is mostly just updating the raw/objects files.

In a survey from last month, only a quarter of respondents reported that they use mods, but making mods more accessible to users may help to raise this. Most players who prefer playing with graphics packs are going to be using a Lazy Newb Pack, so it'd be helpful to some users to have a bit of graphics support for mods. Putting a mod's graphics in the graphics packs may not be the best way to add graphics support for a mod, but it's the best, most-user-friendly option at the moment.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on September 28, 2016, 09:43:36 pm
Eventually it would be nice for the mods to be able to include their own graphics. Currently there are some downsides to doing this, though:
<snip>
For now, the simplest way to add graphics support for mods is to include them in the graphics packs themselves.

Initially, I was wanting to add graphics for [three] mods that include their own graphics.  This would add about 0.75 MB to each graphics pack, which would be an extra 5MB total for a LNP that includes 7 graphics packs.

I am sympathetic, but this would be too large already and likely to grow.

My ruling:  no mod support in vanilla graphics packs.

To balance this: once someone designs a format for graphics to be included in mods, I'll give feedback and once happy implement it in PyLNP.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on September 28, 2016, 10:20:16 pm
If my graphics pack already includes support for some mods, would I need to remove them?  My graphics pack currently includes sprites for all the Fortress Defense races and Succubus sprites for Succubus Fortress.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on September 28, 2016, 11:44:51 pm
No need to remove support that's already in, though once a better solution exists it would be good to convert over.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 29, 2016, 01:37:29 am
I am sympathetic, but this would be too large already and likely to grow.

My ruling:  no mod support in vanilla graphics packs.

To balance this: once someone designs a format for graphics to be included in mods, I'll give feedback and once happy implement it in PyLNP.

== Requested Features ===

Mod graphics addons should probably be able to do these three things at minimum:

Bonus Points: Stonesense support
Normal graphics packs don't really need to add graphics to the Stonesense, but if a mod adds a new workshop or type of furniture, it would be nice if graphics for this could be added to Stonesense.

Stonesense support would need to at least be able to do these two things:


== Graphics Addon format suggestions ==

In the root directory of a mod, have some kind of "Graphics Addons" folder. This folder could contain a bunch of different "addons". These "addons" would be like mini graphics packs. A mod could have a different addon for each graphics pack, or it could have several graphics packs share a single addon.

So inside your /LNP/mods/Fortress Defense Mod II/ folder you would have a /Graphics Addons/ folder (or whatever name works best). And inside that folder you might have several more folders. For example:

/Graphics Addons/GemSet/
/Graphics Addons/Rally Ho/
/Graphics Addons/Spacefox/
/Graphics Addons/Sphr16/
/Graphics Addons/Sphr18/
/Graphics Addons/Sphr32/

Since a graphics pack won't always have the same name in every Lazy Newb Pack, maybe it would be possible to search the value of "get_folder_prefix(pack)" to make a guess at what is currently installed. Maybe it could be made to look for either an exact match or a partial match. First it would check for exact matches, and if doesn't find any, then it would check for any partial matches. At first I was thinking it would be good to specify an addon to be the "Else" addon in case there are no matches, but you probably wouldn't want graphics addons to be installed into more ASCII-ish graphics packs like Taffer and AutoReiv.

The mod's manifest.json might be a good place to specify which addon should be used with particular graphics packs. I'm not sure if that would work very well for that, though.

Spoiler: Manifest.json mockup (click to show/hide)
(Sorry, I'm not very familiar with how JSON works, but it gives you an idea.)

Sorry for any parts that don't make sense.


If my graphics pack already includes support for some mods, would I need to remove them?  My graphics pack currently includes sprites for all the Fortress Defense races and Succubus sprites for Succubus Fortress.

I didn't realize Rally Ho had so much Fortress Defense support. I thought it only had graphics for its dark elf race. If you're going to continue adding support for more mod content, it would be convenient for me if there was a list somewhere (if there isn't already) that indicates what mods are supported.

Maybe not every graphics pack needs to use the same method of supporting mods. There's advantages to bundling mod-graphics with the mods, and there's advantages to bundling mod-graphics with the graphics packs.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on September 29, 2016, 03:41:36 am
Mod graphics addons should probably be able to do these three things at minimum:
  • Add PNGs to the <DF>/data/art/ folder
  • Concatenate text to the bottom of the <DF>/data/init/overrides.txt file (if one exists)
  • Add PNGs, TXT files, and folders to the <DF>/raw/graphics/ folder
Bonus Points: Stonesense support
  • Add folders to the <DF>/Stonesense directory (if it exists). These folders would be able to contain XML files, PNGs, TXTs, and more folders.
  • Concatenate text to the bottom of the <DF>/index.txt file
Quote
== Graphics Addon format suggestions ==

Hmm, I'm not a huge fan of any of these.  My instinct is to start by supporting one graphics style per mod, and if people want more they can publish variants of the mod.  This is not ideal, but much easier to get right.  Basically we need better raw merging in general before we get more complicated...


Summary:
- graphics install pulls files from /data/art/ in installed mods
- mods can add but not change files in /raw/graphics/

- other changes only possible if PyLNP magically gets much better at merging things
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on September 29, 2016, 07:45:40 am
It seems like it would be easier overall to store the mod-specific graphics with the graphics pack instead of the mod.  That way, anyone working on the graphics pack only has to deal with one location, instead of working with files spread across multiple folders.

So, instead of each mod having a Graphics Addons folder with a folder per graphics pack in it, each graphics pack would have a Mods folder with a list of mods it has graphics for.

For overrides, the best I could think of would be to allow each Graphics Mods folder to have its own overrides file, which would overwrite the one added by the graphics pack itself if present.  The problem with using multiple overrides.txt files for this (if it gets implemented) is that a lot of mods (i.e. every single one I've looked at) that add items end up changing the internal ID of existing items, which it what TWBT uses.  This also means that combinations of mods need their own overrides.txt file.

Ideally, mifki would add support for using the object's name instead (e.g. ITEM_TOOL_CAULDRON instead of "0"), since that won't change, but until then, override files will be specific to the exact combination of mods being used.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on September 29, 2016, 08:48:41 am
Remember that changing the art style of a mod should only affect the .png file(s).  The text-based stuff is shared across all of them, so that should only exist once.

It's conceivable that an author might want to use a hunting-trained sprite it one style and color-adding in another, but frankly that's pretty weird and I have no problem skipping support for that case.

So the JSON file would only need to specify a suffix to use for each graphic pack (probably many-to-one, for example several packs assigned to each suffix _bright, _dark and _cartoon).  I would request the ability to include a null suffix, so that the old-fashion drag-and-drop install method would still work... and I would not need to make an LNP-specific build.

Stonesense support is made tricky by the absolute need to override at least one default file.  Adding a comment at the end of the default content might be enough to restore the original.  My mod is pretty rare in having Stonesense content, but it could turn out to be good practice for Armok Vision support.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 29, 2016, 10:52:34 am
That gives me an idea. Mods add the PNG files, but the graphics packs add the text files. When a mod is installed, all the graphics for every graphic pack get installed, but the graphics pack can select the version of the graphics that fit best with it, since it has the text files. This way if mod graphics get added to graphics packs where you probably wouldn't want them (like into ASCII-like ones such as Taffer or AutoReiv), they don't do anything because those graphics packs probably wouldn't be including the text files for mod graphics.

Maybe the Mod addon graphics could be applied to raws before the graphics pack. That way the graphics pack can provide updated graphics for things that haven't been bundled into mods yet.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on September 29, 2016, 04:52:06 pm
Remember that changing the art style of a mod should only affect the .png file(s).  The text-based stuff is shared across all of them, so that should only exist once.

If a graphics pack doesn't have a one-to-one relationship on creatures (i.e. some very similar creatures sharing graphics), this wouldn't work.  Also, I think it would cause blank sprites if one graphics set has hunting/war sprites for an animal but another does not.

Having additional copies of the Graphics text files isn't going to use a lot of space - Fortress Defence's Graphics Text files are a combined 50 KB in Rally Ho!.

That gives me an idea. Mods add the PNG files, but the graphics packs add the text files.
This still has the issue of spreading the art assets over multiple unrelated folders, making maintenance more difficult that it needs to be.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on September 29, 2016, 04:55:52 pm
Quote
Also, I think it would cause blank sprites if one graphics set has hunting/war sprites for an animal but another does not.
It would crash at worldgen. If a graphics file directs to an image that does not exist, the game crashes to desktop.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on September 29, 2016, 04:59:07 pm
I think it crashes if it points to an out of bounds area.  In some cases, like if the standardizer is used, which be be pretty much required if everyone's going to use the same text file, there's be a spot there, it would just be blank.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on September 29, 2016, 05:10:03 pm
Remember that changing the art style of a mod should only affect the .png file(s).  The text-based stuff is shared across all of them, so that should only exist once.

If a graphics pack doesn't have a one-to-one relationship on creatures (i.e. some very similar creatures sharing graphics), this wouldn't work.  Also, I think it would cause blank sprites if one graphics set has hunting/war sprites for an animal but another does not.

Having additional copies of the Graphics text files isn't going to use a lot of space - Fortress Defence's Graphics Text files are a combined 50 KB in Rally Ho!.

That gives me an idea. Mods add the PNG files, but the graphics packs add the text files.
This still has the issue of spreading the art assets over multiple unrelated folders, making maintenance more difficult that it needs to be.
Sorry I wasn't clear... I meant the sprite sheet that deals with a particular mod.  The sprite sheet that comes packaged with The Earth Strikes Back! looks like this
(http://i8.photobucket.com/albums/a28/8gon/TESB_graphics.png)
and if it's redrawn in a cartoonish style the same critters will still be in the same places, meaning the textfiles do not need to change.  Just rename the selected .png to be the target of that graphics file and you're done.

Edit: Now that missing pixel on the Awakened Storm is going to haunt me until I push out a new version of the mod...
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 29, 2016, 06:18:56 pm
Having PyLNP rename PNGs sounds good to me, if PE's okay with doing it like that.

1. PyLNP merges the mod(s) into the temp folder.
2. PyLNP checks the graphics pack's manifest.json to get its "Graphics_Suffix" (or whatever that should be called).
3. PyLNP renames every PNG in the graphics folder to add "disabled_" before the file name.
4. PyLNP renames any PNG in the graphics folder whose name ends with the value of the "Graphics_Suffix" to both remove that suffix and remove the "disabled_" prefix.
5. PyLNP merges the normal graphics pack into the temp folder.

This disabled flag is so it renames the default graphics that don't have a suffix. (Which are there so the format installs easily without the PyLNP installer.)

Also, maybe we can make an exception for Rydel to add mod support directly to his graphics packs.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on September 29, 2016, 07:06:08 pm
If there's a standard for it, I can adapt my mod to fit the standard.  I'm just worried that splitting the files into a bunch of different places will make maintainance harder.

Currently, when I make a new version, I have to make a zip with and without the TWBT specific files.  With this, I'd also have to move out the mod specific files and upload them into a different location.  Same would be true for any other graphics pack with files specific to a mod.

Most of this would be fixed just by keeping the mod specific graphics in a subfolder of the graphics pack.
In that case, the steps would go more like this
1. PyLNP meges the mod into the temp folder.
2. PyLNP check's the graphic's pack's mod folder for a subfolder that matches the mod (or some identifier in the mod's manifest if they are getting those.)
3. If a subfolder was found, PyLNP renames any PNG in the graphics folder to add disabled_ in front of it
4. PyLNP merges the normal graphic pack into the temp folder.
5. PyLNP merges the graphics pack's subfolder for that mod into the temp folder's graphics folder.

4 and 5 could be switched, but this order allows mod specific stuff to override generic graphics if someone feels the need to do that.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 29, 2016, 07:27:23 pm
Putting the mod support in the graphics pack would be easier, but PE doesn't want mod graphics included in his pack (to keep his pack's download size small). So mods need to be able to add their own graphics to make mods more user-friendly for Lazy Newb Pack users to install.

But since it will make it harder for artists to maintain their packs, I think artists should be allowed to add mod support to their packs directly. I think the only graphics pack artists who might be interested in making mod art would be you, utkonos, and Tallcastle. For packs whose artists are inactive (such as Afro, GemSet, Ironhand, Mayday, Obsidian, Phoebus, and Spacefox), it's probably okay to add their mod graphics to the mods.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on September 29, 2016, 08:39:00 pm
Will the LNP be pulling the graphics from the repo directly, or will they be bundled before it's uploaded?
If it's the latter, it should be easy to programatically remove the mods folder from all the graphics set.  This way, they won't increase the size for the end user.

Wouldn't moving the extra graphics from the graphics pack to the mod not change the size at all? Or is PE not going to include mods in his?

Come to think of it, it may be possible to have the best of both worlds if the LNPs will move folders before release.
Each Graphics set would have a Mods folder, with subfolders for each mod they have unique graphics for.
A script could then iterate over each graphics pack and move the folders for each mod to the folder where the mod itself it stored.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on September 29, 2016, 09:11:00 pm
Or is PE not going to include mods in his?

I get the impression that he doesn't want mods included in his pack. Otherwise, he probably wouldn't mind graphics packs having built-in mod support.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on September 30, 2016, 11:16:43 am
I think that mod-specific graphics belong in the mods... just have the LNP fish out the appropriate graphics from the mods/raw/graphics folder during the merge process.

Now, that opinion is based on a couple assumptions that I want to verify with the experts in the room:

1. There are more LNP players who use multiple graphics packs and zero/one mods than there are LNP players who use multiple mods and zero/one graphics packs.

2. Mods will be a separate download from the LNP.

Though not essential, the case is stronger if the major graphics packs are going to remain bundled with the LNP.

While it's better to have a simple solution than a complex one, the complexity largely falls on the mod authors, graphic artists and pack maintainers (rather than players), so as far as I'm concerned technical complexity is not a major concern.

So, the reasoning for keeping graphics-pack-specific files in a mod rather than mod-specific files in a graphics-pack is to minimize the amount of content that gets downloaded and never used.  The Earth Strikes Back! contains about 7 KB of images for creature graphics (out of 312KB).  Even if I went overboard and made twenty different versions and they were different enough to get no gains from ZIP compression, that's still adds less than 50% to the size of the pack.  This is about the same amount of space as if each graphics pack included its own graphics for The Earth Strikes Back!, but with the crucial difference that the content is only downloaded by people who choose to sample/play the mod.

For reasons that elude me, somewhat less than 100% of LNP players use my mod :)  Therefore, it makes sense to package extended graphics content with the mod than with the graphics pack.  As the amount of graphic content goes up in a mod, the potential impact on non-users increases.

Even if the LNP were split up so that graphics packs are downloaded separately (perhaps upon first selection in the GUI), the same reasoning holds.  In fact I would suggest the LNP go to a fetch-on-first-use model for both graphics packs and mods, with some option to pre-fetch for those who don't play connected to the Internet (How do you survive without the wiki?!).
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on September 30, 2016, 02:59:46 pm
I've been working with the assumption that both 1 and 2 are false.
For one, I mostly am thinking off personal experience - I try a lot of mods, but stuck almost entirely to Phoebus until I created my own graphic set.
With two, since the graphics packs are already in the download, and I'd figured mods would be the same.

If mods aren't a separate download, then the amount of data that's downloaded but not used is the same in either method.

As for complexity, both put the complexity on the mod authors, graphic set artists, and pack maintainers.
Additionally, I work in IT, and one of the security principles is "Principle of least privilege" - you give someone the lowest level of access necessary to do what they need.
If all graphics pack specific files are with the graphic pack, each artist needs access only to graphics packs they work on.
If graphics pack specific files are stored with the mods, then each artist also needs the ability to commit changes to every mod they plan to include support for.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on September 30, 2016, 06:43:31 pm
PE is worried about the bandwidth required to include mods, so I figured we could go the other way if there was a painless-to-the-player method of getting graphics packs out of the base download.

The key word being "painless."
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 01, 2016, 05:36:51 pm
I think PE might want to keep it really simple for now. I'm wondering if filename switching is more complicated than what he wants to implement at this time. If PyLNP would install the graphics from the mods, then install the graphics pack, then install the mods' raws and stuff, I think think that would work just fine for initial support.

This would allow mods to have their own default graphics. It would allow graphics packs to override a mod's graphics with their own (or with nothing for ASCII-ish packs). And it would allow the mods' raw objects files to override the graphics pack's raw objects files.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on October 01, 2016, 07:06:06 pm
I think PE might want to keep it really simple for now. I'm wondering if filename switching is more complicated than what he wants to implement at this time. If PyLNP would install the graphics from the mods, then install the graphics pack, then install the mods' raws and stuff, I think think that would work just fine for initial support.

This would allow mods to have their own default graphics. It would allow graphics packs to override a mod's graphics with their own (or with nothing for ASCII-ish packs). And it would allow the mods' raw objects files to override the graphics pack's raw objects files.
Pro: I can support this standard with my current distro, meaning zero marginal effort on my part.  Artists can support modded stuff at their own pace with no compatibility-breaking jumps in the graphics pack format.  And, did I mention zero marginal effort for me? :)
Con: Seems likely to cause filesize creep in the LNP.  This was precisely what PE was trying to avoid, though this plan inflates the size a lot less than the original idea of bundling a couple dozen mods.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 01, 2016, 08:31:13 pm
It wouldn't save as much space, but it would be easier to implement. And it would allow mod packs to install their own graphics while still allowing Rally Ho to have its own graphics for mods too.

Being able to switch file names depending on the pack installed would be great, though, if PE's okay with that.
Title: Re: Dwarf Fortress graphics repositories
Post by: Tallcastle on October 01, 2016, 08:36:30 pm
i confess i haven't gone back to read this entire conversation so i'm kind of jumping in the middle here.  If "Filesize Creep" is an issue, why not include the graphics packs as a separate download?  Like an Expansion pack.  This also arranges it so that the "Graphics" component of the LNP can be updated regularly without needing to repackage the entire LNP.

Again, i haven't taken the time to see what came before.  so if this has been suggested earlier than sorry for the necromancy.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 01, 2016, 09:25:50 pm
Making users download and install graphics packs seperately seems like it might be confusing for new players, and there's probably several users annoyed at the more complicated installation process.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on October 02, 2016, 08:14:53 am
If it could be dead simple, such as fetch-on-first-use, that should work just fine.  The hard part is the player with intermittent/slow Internet who wants to prefetch some packs (at the extreme, think of someone downloading to a thumb drive on a library computer that is too locked down to actually run the LNP).

It shouldn't be too easy to prefetch everything (for example, a "fully loaded" pack) since a large number of players would simply pick that pack.  The ability to "fetch" from a local file should suffice for the constrained users without enticing general players to download everything at once.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on October 02, 2016, 10:10:45 am
Alright, I'm caught up and finally have time for a longish response. 

I started by replying to individual posts, but that got too long and hard to follow; so instead it's organised by topic.
Quotes omitted due to length and duplication, so if I've missed or misrepresented anything let me know please!

And now, the essay:



I think we're actually discussing a couple of separate things together, and it would be useful to distinguish them.

Types of content (for things added by mods):
 - Native DF creature graphics
 - TwbT item and building overrides
 - Stonesense content and other visualizers

Issues:
 - Should graphics support for mods be part of the mod, or the graphics?
 - What file structure or conventions should we use?



Where should graphics extensions be stored?

Ruling:  A graphics pack should be for vanilla DF only.  A mod in PyLNP format may include a "graphics extension" for one [or more] graphics packs.

Rationale summary:


TwbT extensions
These are actually pretty tricky right now, given that they should be flexible in PyLNP but also work when drag-and-dropped.

Support will be very easy if/when Mifki supports multiple overrides files, or changes to the alternative sprite-per-file filename-based scheme he mentioned a while ago.

@Rydel:  items can in fact be referenced by name (https://github.com/mifki/df-twbt#overrides), so we can avoid the problem of numeric IDs changing.  Perhaps we (or Mifki) should detect use of numbers and issue a deprecation warning?



Stonesense content is out of scope for this discussion.  We can - and I intend to - consider supporting utilities after native creature graphics and TwbT support works.  Armok Vision plans to use Stonesense format directly, when customization creatures are added, so no extra work will be required there.  Other utilities are further away still - let's get the basics nailed down first!



Format Ideas and Constraints

I'll list some contraints that our format must satisfy (non-negotiable); then features that would be nice if someone can work out how, and last describe a starting point for more discussion.

Constraints:
Nice (but optional) features:
Suggested starting point:


Other Comments

A few other things came up in the discussion; here are some replies.


So, uh, I hope that makes sense!  I know I'm coming across as the no-fun-guy in a few places, and maybe acting on authority I don't really have - but I hope you can see there's some reasons behind this, and I'm still happy to hear other opinions.  And frankly if anyone else wants to code up a better system they're entirely welcome to; and it's not like the format is enforcable in any way other than compatibility with the PyLNP code...

Regardless, keep up the discussion - I've already learned a lot!  :D
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on October 02, 2016, 11:01:10 am
TWBT overrides accept item subtypes now?
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on October 02, 2016, 12:49:31 pm
@Rydel:  items can in fact be referenced by name (https://github.com/mifki/df-twbt#overrides), so we can avoid the problem of numeric IDs changing.  Perhaps we (or Mifki) should detect use of numbers and issue a deprecation warning?

That link shows names for Ids and Types, but last I heard, subtypes are still numerical.  This would be a problem for mod support since subtypes for existing entries can change when you install a mod.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 02, 2016, 03:19:04 pm
That was a well-organized post.

  • It must be possible to make a Graphics extension compatible with an arbitrary graphics pack, without the cooperation of the graphics maintainer.  I understand that most of the people in this thread fill both roles, but the format we choose should work for more casual creators too - or we'll never see much takeup! (ie: mods mst be entirely self-contained)
Are you talking about making this a new folder in the /LNP/ folder? Like a /LNP/GfxExt/ folder or something? (There's probably a better name for it than that, though.)

  • Support multiple graphics extensions in a single mod.  Easy to store in a subfolder; but how would you flexibly decide which to install?  Possible:  categories like MIME types, regular expressions, something else.  (Must support a default for drag-and-drop install)
Instead of trying to use existing lines in the manifest, it might be better to have new lines just for this instead of trying to use regular expressions on existing lines. Make special manifest lines just for mod graphics so that there's no need to change them except to adjust what graphics extensions get used with a graphics pack or mod. Then the manifests of the graphics extension packs could have two lists, one to list the graphics packs it should be used with and another to list the mods it should be used with. Whenever a user installs a graphics pack or a mod, PyLNP would also install every graphics extension pack that had at least one hit in both lists.

The manifest of the graphics extension would also have a line that indicates if the extension is intended to replace the mod's built-in graphics or supplement them. Graphics extensions would be installed after mods, so that the extension in "supplement" mode can still comment out any of the default graphics that it wants to replace.

  • To support graphics, mods just include graphics raws and images.  As long as they don't have the same name as a file in the graphics pack, this already works!  (I just checked the code - apparently I thought of this two years ago :) )
I thought I tested to see if this would work like a month ago and PyLNP v0.11 wouldn't install any PNG files from mods.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on October 02, 2016, 07:43:42 pm
I think "in the mod" meant under mods/___/raw/graphics/.  The tricky part is having multiple versions for different graphics packs, which could be done with a file suffix (before the extension) or subfolders.  The mod's manifest calls out a specific set of graphics to use when specific graphics packs are active, for example the _cartoon version for GemSet and Spacefox.  Unnamed graphics packs get the mod's default.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 02, 2016, 09:12:29 pm
He talked about people being able to make graphics extensions without needing to collaborate with mod authors or graphics packs authors, so I thought graphics pack extensions might be getting their own spot.

Would I still be able to specify graphics packs by name instead of by category? I'd like to have the ability to specify different sets of graphics for Spacefox and GemSet.

Would each graphics pack need to tag it's manifest with categories to describe itself? Maybe use tags like "cartoon" "ASCII-shaped" "monochrome".

Maybe there could be a way to categorize by size too, so you could provide different-sized versions of graphics in the same category.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on October 02, 2016, 10:41:18 pm
Once you guys figure out a standard, PLEASE, let me know. I hate doing all kinds of work, then re-do it because I have to sort it or name it differently. ;)
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on October 02, 2016, 11:13:10 pm
He talked about people being able to make graphics extensions without needing to collaborate with mod authors or graphics packs authors, so I thought graphics pack extensions might be getting their own spot.

Would I still be able to specify graphics packs by name instead of by category? I'd like to have the ability to specify different sets of graphics for Spacefox and GemSet.

Would each graphics pack need to tag it's manifest with categories to describe itself? Maybe use tags like "cartoon" "ASCII-shaped" "monochrome".

Maybe there could be a way to categorize by size too, so you could provide different-sized versions of graphics in the same category.
I was envisioning an arbitrary list in the mod's JSON file, not tagging the graphics packs themselves.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 03, 2016, 03:08:15 am
Graphics extension module organization

Subfolders might be better than putting suffixes on everything. It sounds more simple to copy everything from a certain folder than to copy only the files with certain suffixes. And subfolders would keep the modules better organized too. Even if we use suffixes, I'd like to have the ability to put each module in its own folder if possible.


Graphics extensions modifying objects files

While thinking about how to add graphics support for Arctic Additions, I realized that graphics modules will need to be able to edit the /raw/objects/ of the mod in order to change which tiles get used and the colors. Most if not all of the more graphical tilesets move a bunch of stuff around in the main tilesheet. Examples:
A plant in a mod like Arctic Additions would need to be able to specify different tiles depending one what graphics pack is being used, since that tile could very likely be in a different location in every tileset. So it would be helpful for graphics extensions to edit the objects files.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on October 03, 2016, 03:26:53 am
You can circumvent that by using TWBT overrides for the plant. Regardless of how the tilesets look you use, the mod graphics for the plant would always look the same.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 03, 2016, 03:40:46 am
Graphics modules won't support overrides until multiple overrides.txt sheets can be used alongside each other. And it would be nice for graphics to look good for players not using TwbT.

Also, Rydel was saying that overrides might have some problems when used with mods, but I'm not really sure how much of a problem it is.

@Rydel:  items can in fact be referenced by name (https://github.com/mifki/df-twbt#overrides), so we can avoid the problem of numeric IDs changing.  Perhaps we (or Mifki) should detect use of numbers and issue a deprecation warning?

That link shows names for Ids and Types, but last I heard, subtypes are still numerical.  This would be a problem for mod support since subtypes for existing entries can change when you install a mod.

I don't know what a subtype is. Can I make a TwbT override without using a subtype? (Do "main types" or "super types" exist or something?) If the subtypes can never be known, are overrides completely useless?
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on October 03, 2016, 06:42:52 am
Quote
Graphics modules won't support overrides until multiple overrides.txt sheets can be used alongside each other.
Why? Can't you merge the override file just like you merge the raws?

Since you can call override tilesets like this:
Code: [Select]
[TILESET:tileset:tileset.png:_any_name_you_want], modders could just use unique names for each set, like "_MEPHS_STUFF_VERSION_1_2".
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 03, 2016, 10:34:26 am
If multiple mods try to add overrides, all but one overrides file would get overwritten.

A work-around would be to combine and store the overrides files for every mod for a graphics pack in every graphics extension module for that graphics pack. I'd be fine with it working like that until it's possible to use multiple overrides.txt files.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on October 03, 2016, 12:21:07 pm
Suggested starting point:
  • (recap:) A mod in PyLNP format is simply a DF install with the vanilla files removed.  We then merge the raw/ and data/speech/ files.
  • To support graphics, mods just include graphics raws and images.  As long as they don't have the same name as a file in the graphics pack, this already works!  (I just checked the code - apparently I thought of this two years ago :) )
  • Support for data/art and data/init/<overrides files> will be added once Mifki supports multiple overrides files.
  • (for discussion) How should non-default graphics extensions be stored?  How should PyLNP choose which to install?  Does this work with multiple mods?

FYI, I've never been able to get the mod merging tool to import graphics under Mods/___/raw/graphics/
Code: [Select]
Running PyLNP 0.12 (OS: win, Compiled: True)
WARNING: JSONConfiguration: File PyLNP.user does not exist
ERROR: In "TESB": : file "graphics\graphics_TESB_creatures.txt": : Writing to .\LNP\Baselines\temp\raw\graphics\graphics_TESB_creatures.txt failed
Adding that capability would simplify installation.

As for TWBT overrides, the easiest solution (for everyone except mifki) would be to use named subtypes.  With numeric subtypes, it's possible that more than one mod could add subtypes to the same type.  One solution would be to keep a running map of each mod's subtypes to a global list of subtypes (they should each be written as if they were modifying vanilla entries or adding directly to the end of vanilla).  Though conceptually simple, it just feels error-prone and thus would need the option for verbose logging.

Vanilla ARMOR:1 -> ARMOR:1
Vanilla ARMOR:2 -> ARMOR:2
...
Vanilla ARMOR:11 -> ARMOR:11
SuperMod ARMOR:12 -> ARMOR:12
SuperMod ARMOR:13 -> ARMOR:13
UltraMod ARMOR:12 -> ARMOR:14
UltraMod ARMOR:13 -> ARMOR:15
UltraMod ARMOR:14 -> ARMOR:16
XenoMod ARMOR:7 -> ARMOR:7 // Redefining a vanilla object.

Slip the mapped values into the tempfiles, then concatenate them into a single overrides.txt file.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on October 03, 2016, 12:23:15 pm
If multiple mods try to add overrides, all but one overrides file would get overwritten.
How do you want to install mods in that case? I was under the impression that the PyLNP merges the raw files. If it would just replace files, you can barely install any two mods at the same time. Anything that changes vanilla DF files would be impossible.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 03, 2016, 01:17:44 pm
PyLNP merges things by copying both of their files into the same folder. If two mods try to add a /data/init/overrides.txt, only of those files will survive the merger.

As for TWBT overrides, the easiest solution (for everyone except mifki) would be to use named subtypes.  With numeric subtypes, it's possible that more than one mod could add subtypes to the same type.  One solution would be to keep a running map of each mod's subtypes to a global list of subtypes (they should each be written as if they were modifying vanilla entries or adding directly to the end of vanilla).  Though conceptually simple, it just feels error-prone and thus would need the option for verbose logging.

Vanilla ARMOR:1 -> ARMOR:1
Vanilla ARMOR:2 -> ARMOR:2
...
Vanilla ARMOR:11 -> ARMOR:11
SuperMod ARMOR:12 -> ARMOR:12
SuperMod ARMOR:13 -> ARMOR:13
UltraMod ARMOR:12 -> ARMOR:14
UltraMod ARMOR:13 -> ARMOR:15
UltraMod ARMOR:14 -> ARMOR:16
XenoMod ARMOR:7 -> ARMOR:7 // Redefining a vanilla object.

Slip the mapped values into the tempfiles, then concatenate them into a single overrides.txt file.

It reminds me of the strife specibus from Home Struck (browser game that teaches data structures). It seems like the map would need to know the sub types for every combination of mods. (I think mods can't pick subtypes for their items, but the game assigns them dynamically based on the alphabetical order of something or other.) And that subtype map would probably need to be updated every time any mod adds a new item or maybe renames one or something. I think this task sounds impossible to solve from PyLNP's end. TwbT might be the more logical place to take care of figuring out subtypes, since TwbT has access to DFHack and can probably get the subtype numbers for items from it.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on October 03, 2016, 01:36:08 pm
Correct, mods cannot pick the subtype numbers, but they are assigned in a deterministic order based on the known order in which raws are parsed.  The thing to prevent is someone jumping in front of the vanilla stuff.
Title: Re: Dwarf Fortress graphics repositories
Post by: Rydel on October 03, 2016, 02:13:56 pm
You can circumvent that by using TWBT overrides for the plant. Regardless of how the tilesets look you use, the mod graphics for the plant would always look the same.
Plant overriding in TWBT requires a lot of raw edits.  TWBT on its own can't tell the difference between shrubs, fruit, leaves, dead plants, etc, so all of those need to be overrided in the raws.

I don't know what a subtype is. Can I make a TwbT override without using a subtype? (Do "main types" or "super types" exist or something?) If the subtypes can never be known, are overrides completely useless?
Subtypes are how the game identified items in the same category.
Say we were looking at tools.  All of them have the ID TOOL and the Type TOOL, an the subtype tells which tool it is.
These are automatically numbered in the order listed in the raws, so 0 is Cauldron, 1 is Ladle, 2 is Bowl, etc.

The subtypes are known, but the person writing the override file has to know what order items are in the raws in advance.  Stuff getting added to the end won't mess stuff up, but files getting added at the beginning or in the middle will cause problems.

Let's say a mod added a file called item_mod_tools.txt with a new tool, called NEWTOOL.
Now 0 is NEWTOOL, 1 is Cauldron, 2 is Ladle, 3 is Bowl...
Since your overrides were written without the mod, all the NEWTOOLs will look like cauldrons, the cauldrons will look like ladles, etc.

You can't work without subtypes because (aside from TWBT just throwing an error and probably crash) it would make every look look identical.


As for TWBT overrides, the easiest solution (for everyone except mifki) would be to use named subtypes.
I agree, but I think I've asked about that feature before and it wasn't possible.  I think DFHack doesn't know the names the raws used, it just has the number pulled from the program itself.  For things like Id and Type, those won't change, so they are known at compile time and can be given specific names.

Correct, mods cannot pick the subtype numbers, but they are assigned in a deterministic order based on the known order in which raws are parsed.  The thing to prevent is someone jumping in front of the vanilla stuff.
For preventing files from jumping in front of vanilla stuff, the easiest way I can think of it to prepend all the mod files with "zz_", forcing them to be last alphabetically.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on October 04, 2016, 12:06:11 am
Alright, time for another (shorter...) essay.

TwbT Overrides
@Meph @Rydel:  OK, apparently I'll also be asking Mifki to support subtypes by name.  (it would also be good to log a deprecation warning for numerical indicies, both to encourage change and explain to users why things are broken...).  DFHack does have access to the names - look for example at the view-item-info and item-descriptions scripts: the former looks up text in the latter by item type or subtype name.

We can't reliably merge overrides files, as they would not have the level of context that raws generally provide (ie unchanged vanilla lines).  TBH I'm surprised sometimes that the merge works for raws at all...  improving this has been a PyLNP goal for years, but it's a really hard problem.  Unique names so we can just keep adding files is therefore my favorite solution!

Preprocessing files to keep track of item subtype ID numbers... just no.  It's may be technically possible, but it's not happening.  I'd improve raws merging instead if I could do this.


Where do Graphics extensions live
As @Dirst says, it's just be whatever goes in "LNP/Mods/<some_mod>/raw/graphics" - not a new kind of thing (which would be a combinatorial explosion at best).  This is obviously a default-only thing; extensions for other graphics packs will live in subdirectories ("LNP/Mods/<some_mod>/graphics-extensions/<name-of-gfx>/", for example).

@Jecowa - creating a graphics extension for a mod makes you a mod author (or contributor, in which case you do the usual sned-changes-to-author dance); but you don't need the graphics pack to help you or even know graphics extensions exist - so they'll be backwards-compatible :)

No suffixes or renaming.  It's just to fragile, and prone to breaking in hard-to-diagnose ways!


How does PyLNP decide which graphics extension to install?
The manifest for a mod will have a new attribute "graphics-extensions", which is a dictionary mapping graphics packs to subdirectory names (see above).  Technically, the extension is just another mod to merge (so object files can be changed); by policy it should just override the relevant graphics bits.

PyLNP will install the default mod (which includes graphics), then if the installed graphics pack matches a subdirectory (per the mapping) will install the relevant extension.  Mod authors may therefore wish to provide nothing as the default extension (ie leave /raw/graphics and twbt files nonexistent), to avoid installation clashes.


Other comments
@Meph - we feel exactly the same way about standards; I'll be sure to let you know.  Honestly I'd love to bring PyLNP and the masterwork launcher closer together or even merge them, but that's a conversation for another time and place (and after some upgrades...).

Maybe mods writing to raw/graphics is banned by code elsewhere?  Anyway, the core code handles this OK and I should be able to make it work without too much trouble.

PyLNP copies image files and DFHack scripts, but merges .txt (eg raws) and .init files.  Otherwise we would indeed have (even more) trouble installing more than one mod at a time.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 05, 2016, 08:10:16 am
but you don't need the graphics pack to help you or even know graphics extensions exist - so they'll be backwards-compatible :)
[...]
How does PyLNP decide which graphics extension to install?
The manifest for a mod will have a new attribute "graphics-extensions", which is a dictionary mapping graphics packs to subdirectory names (see above).
The dictionary map would be a sort of spreadsheet in the mod's manifest that tells PyLNP which graphics extension goes with which graphics packs, right? Can it do a partial match on the graphics pack, in case a user has renamed two copies of Phoebus to things like "Phoebus v0.43.03" and "Phoebus v0.43.05", for example?

Technically, the extension is just another mod to merge (so object files can be changed)
That's good to hear. That's something that's currently not possible with mod support added to graphics packs.

PyLNP will install the default mod (which includes graphics), then if the installed graphics pack matches a subdirectory (per the mapping) will install the relevant extension.  Mod authors may therefore wish to provide nothing as the default extension (ie leave /raw/graphics and twbt files nonexistent), to avoid installation clashes.
That sounds good.

Maybe mods writing to raw/graphics is banned by code elsewhere?
Sorry, I just tried it again (in PyLNP v0.11) and it worked. I was probably looking in the wrong Dwarf Fortress folder when checking to see if it installed. I think I did that at one point and couldn't figure out why nothing would install.

PyLNP [...] merges .txt (eg raws)
It's not working like that for me. From what I can tell, if a user tries to install more than one mod (like Spellcrafts, Haberdasher, and The Earth Strikes Back, for example) that all want to edit to the same file (such as entity_default.txt), only highest-listed of those mods in the "Merged" list of the "Mods" tab will have a green background color (and will installed normally). The lower-listed mods trying to edit that file will have white backgrounds (and will silently not be installed).

I kind of thought the files it really could mess with (other than its own) were init.txt and d_init.txt.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on October 05, 2016, 05:58:01 pm
Can it do a partial match on the graphics pack, in case a user has renamed two copies of Phoebus to things like "Phoebus v0.43.03" and "Phoebus v0.43.05", for example?

PyLNP [...] merges .txt (eg raws)
It's not working like that for me. From what I can tell, if a user tries to install more than one mod (like Spellcrafts, Haberdasher, and The Earth Strikes Back, for example) that all want to edit to the same file (such as entity_default.txt), only highest-listed of those mods in the "Merged" list of the "Mods" tab will have a green background color (and will installed normally). The lower-listed mods trying to edit that file will have white backgrounds (and will silently not be installed).

I kind of thought the files it really could mess with (other than its own) were init.txt and d_init.txt.

I certainly plan to support a degree of fuzzy matching.  My current favourite test is to lower-case everything (so it's not sensitive to capitalisation), then for each graphics pack check if the ID-to-load is part of the pack name.  This is easy to write and understand, unlike (eg) regular expressions.

If you're seeing unmerged mods *without* an incompatible (red) line above them, please post your logs in the PyLNP thread and we'll investigate there.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on October 05, 2016, 06:45:41 pm
In my experience, a white line usually means there was some kind of error (e.g., unable to copy a graphics txt file).  The log ought to have something about it.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 11, 2016, 12:48:43 am
Quote
Also, I think it would cause blank sprites if one graphics set has hunting/war sprites for an animal but another does not.
It would crash at worldgen. If a graphics file directs to an image that does not exist, the game crashes to desktop.

I think it might not crash if the graphics are for creatures that do not exist in the game.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on October 11, 2016, 12:34:47 pm
Quote
Also, I think it would cause blank sprites if one graphics set has hunting/war sprites for an animal but another does not.
It would crash at worldgen. If a graphics file directs to an image that does not exist, the game crashes to desktop.

I think it might not crash if the graphics are for creatures that do not exist in the game.
This is fine. Graphics yes, creature no = no problem.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 11, 2016, 05:59:27 pm
Sorry, I didn't say it very well. Missing PNG files don't make the game freeze or crash if those PNGs were only needed for creatures that don't exist in the game.

I think it doesn't cause a crash to have "graphics_*.txt" files with missing PNG files, if they are for a creatures that aren't in the /raw/objects/.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 12, 2016, 09:25:51 pm
I'm having a problem with Mayday. I need to balance having editable versions for graphics authors with having low-disk space versions for inclusion in distributed packs.

I want to move all of Mayday's uncompressed editable graphics to something like a "dev" folder in the root directory and replace them with compressed flattened versions. Eventually I'd probably want to do something like this with other packs and would like to have a consistently-named folder in every pack to make it easy for script users to remove this folder automatically.

Is this okay to do? If so, any opinions on what the folder should be called?
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on October 12, 2016, 10:12:04 pm
Sounds good.  "source" is a traditional name, and add a source/README.md file that explains how to convert the source to output version.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on October 12, 2016, 11:25:06 pm
It's not saving as much space as I thought it would. Every bit helps, though, I guess.

New flat compressed graphics: 2.3 MB (1.3 MB zipped)
Old uncompressed graphics:5.9 MB (4.6 MB zipped)

Note: 1 MB = 1000000 bytes

add a source/README.md file that explains how to convert the source to output version.
I don't know if that's necessary. There are flat versions ready for distribution in the normal locations. To turn the layered versions into flat versions, you just have a graphics program flatten the layers. That would probably happen anyone to someone editing those files who wasn't aware that they had layers. Edit: Just thought of something to write there that should be helpful for graphics artists.

To make this ready for a pack, just delete the "source" folder and it should be ready-to-go.
Title: Re: Dwarf Fortress graphics repositories
Post by: Dirst on November 18, 2016, 05:21:16 pm
Is there any working standard for including graphics extensions?  I have done a spectacularly bad job of imitating the art styles of three graphics packs, and would like to slip these into the next release of my mod (in addition to any other unwise extensions between now and when PyLNP officially supports the standard).

Default (32x32)
(http://i8.photobucket.com/albums/a28/8gon/TESB_graphics_1.png)

DungeonSet (24x24)
(http://i8.photobucket.com/albums/a28/8gon/TESB_graphics_DungeonSet.png)

GemSet (24x24)
(http://i8.photobucket.com/albums/a28/8gon/TESB_graphics_GemSet_1.png)

CLA (18x18)
(http://i8.photobucket.com/albums/a28/8gon/TESB_graphics_CLA.png)

GemSet and CLA may be nigh-invisible in the forum's light color scheme.

Edit: Corrected one image.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on November 18, 2016, 05:56:45 pm
Those look good, Dirst.

Mods don't have graphics support yet in PyLNP. The best method at this point is to install the mod graphics into the graphics packs. Note that installing mods removes the graphics folders, so you will have to go back to the Graphics tab and double-click on the graphics pack you want to use after installing a mod.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on November 30, 2017, 06:00:16 am
The default tile for Display Cases seems good. (It uses the Bookcase tile.) But for the more graphical packs, it might be a good idea to switch the tile for Pedestal. I'm thinking about changing the Pedestal on the graphical packs to lowercase "i" for now, then if we ever get custom graphics for them, we can set TwbT Overrides for them in the future.

These are the one's that could benefit from a different tile for the Pedestal:Or maybe it doesn't matter so much what the Pedestal looks like since you only really see it when it's empty?
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on November 30, 2017, 07:26:55 am
Are there no free tiles left in the tilesets? Or is the issue just that there is no graphical tile for them yet?
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on November 30, 2017, 02:18:28 pm
Both. There's was never any extra space on the main tilesheet. It's always been a balancing act there. I haven't drawn anything for any of these yet. The chair being shaped like a little stool might make it a good fallback tile for the pedestal for people playing without TwbT, but then that would make it easy to get confused with real chairs. Maybe the pedestal is important enough as a decorative item to get its own tile on the main tilesheet? I wouldn't really know what to move around, though, and I'm not much of an artist.

The flask (n6) might make an okay pedestal in Phoebus, Mayday, Ironhand, and Afro.
Spoiler: Phoebus (click to show/hide)
Spoiler: Mayday (click to show/hide)
Spoiler: Ironhand (click to show/hide)
Spoiler: Afro (click to show/hide)

Maybe tile e16 in Rally Ho! and Obsidian
Spoiler: Rally Ho! (click to show/hide)
Spoiler: Obsidian (click to show/hide)

And maybe tile m6 in Spacefox
Spoiler: Spacefox (click to show/hide)
Title: Re: Dwarf Fortress graphics repositories
Post by: Ultimuh on December 02, 2017, 02:48:20 pm
Just waiting for one of these to update.
I prefer playing with a tile/graphics set rather than plain ASCII.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on December 09, 2017, 12:45:14 pm
Spacefox has a Pedestal graphic and Dwarf sprites for the new professions thanks to Nopal and CLA.

These ones all have tiles set for the new Pedestal but don't have graphics for the v0.44 professions:

 * Mayday
 * Phoebus
 * Ironhand
Title: TWBT and non-TWBT versions
Post by: jecowa on January 01, 2018, 08:52:18 pm
Having separate versions of files for TWBT and non-TWBT installs is becoming more important what with TWBT multi-layer rendering on the main tilesheet and TWBT unit transparency on the creature graphics.

Spoiler: quote jecows (click to show/hide)

That would break the desirable property that copying graphics over a DF install correctly installs the graphics - we want them to be very simple to do manually or automatically.

An alternative approach (which I prefer) would be to add "twbt_data" and "raw/twbt_objects" folders, and copy them over the non-twbt pack if TwbT is enabled.  Regardless, any further discussion should happen in the DFgraphics thread, as it's not really about the Starter Pack.

This layout sounds good. Maybe a "twbt_data" folder and a "twbt_raw" folder so that it can have separate creature graphics too. I'm working on TWBT creature graphics right now.

Some data in case you are worried about the additional creature graphics taking up too much space. (tl;dr - I would guess they would make a LNP with all the graphics packs take up less than 5 more MB in total.)
Spoiler (click to show/hide)

Edit: finished Ironhand creatures. The Ironhand TWBT creature graphics are less than 0.3 MB. Old ones were around 0.7 MB.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on January 02, 2018, 09:46:03 am
Curious thing I noticed about the creature file space.

The new -bg file is only black and white, mostly transparent. The original file suddenly has a lot more transparency instead of solid colors. This lead to a file-size reduction in the latter, that makes up for the file-size of the new file.

Example: My original creature_domestic was 100kb... now, with transparency it's only 31kb, plus 7kb for the -bg file.

In addition to that, most tileset authors can save about 50% size on all creature graphics by removing the war/hunting versions. Most sets I've seen just brute-force that, make a war/hunting version for every creature... but only 22 creatures out of 767 are actually trainable.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on January 06, 2018, 02:07:32 am
In addition to that, most tileset authors can save about 50% size on all creature graphics by removing the war/hunting versions. Most sets I've seen just brute-force that, make a war/hunting version for every creature... but only 22 creatures out of 767 are actually trainable.

Thanks for the info! I was trying to figure out how you determined which animals were trainable and found 3 tags: [TRAINABLE], [TRAINABLE_WAR], and [TRAINABLE_HUNTING]. These are the 22 creatures with those tags:

[TRAINABLE] (trainable for war or hunting):
1.DOG  creature_domestic.txt
2.BEAR_GRIZZLY  creature_large_temperate.txt
3.PANDA, GIGANTIC  creature_large_temperate.txt
4.ELEPHANT  creature_large_tropical.txt
5.LION  creature_large_tropical.txt
6.LEOPARD  creature_large_tropical.txt
7.JAGUAR  creature_large_tropical.txt
8.TIGER  creature_large_tropical.txt
9.CHEETAH  creature_large_tropical.txt
10.MANDRILL  creature_large_tropical.txt
11.GORILLA  creature_large_tropical.txt
12.BEAR_POLAR  creature_large_tundra.txt
13.JABBERER  creature_next_underground.txt
14.CAVE_DRAGON  creature_next_underground.txt
15.DRAGON  creature_standard.txt
16.BIRD_ROC  creature_standard.txt
17.BOBCAT  creature_temperate_new.txt
18.OCELOT  creature_tropical_new.txt
19.LYNX  creature_tundra_taiga_new.txt

[TRAINABLE_HUNTING]:
20.BAT_GIANT  creature_subterranean.txt
21.BIRD_SWALLOW_CAVE_GIANT  creature_subterranean.txt

[TRAINABLE_WAR]:
22.RHINOCEROS  creature_large_tropical.txt

With only 20 creatures trainable for war and 21 trainable for hunting, it's quite a bit more be feasible to give them more than an icon. Maybe a piece of armor for war animals and a scarf for hunting animals?

Edit: With the tag inheriting thing, maybe the _giant and _man versions could also be trainable?
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on January 06, 2018, 10:39:37 am
Since you collect and update sets, you might want to know that traps can use overrides for each trap type. I haven't seen anyone using it, so here you go, in case you want to add that feature to any set.

Code: [Select]
[OVERRIDE:old-tile:B:TRAP:Trap:1:new-tileset:new-tile] pressure plate
[OVERRIDE:old-tile:B:TRAP:Trap:2:new-tileset:new-tile] cage
[OVERRIDE:old-tile:B:TRAP:Trap:3:new-tileset:new-tile] stonefall
[OVERRIDE:old-tile:B:TRAP:Trap:4:new-tileset:new-tile] weapon

Title: Re: Dwarf Fortress graphics repositories
Post by: mifki on January 06, 2018, 10:59:52 am
Levers are traps too with subtype 0 and track stops with subtype 5.
Title: Re: Dwarf Fortress graphics repositories
Post by: Meph on January 06, 2018, 11:34:56 am
Oh... didn't knew about the levers. That gives 2 more free tiles in the main tileset. :)
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on January 07, 2018, 04:25:09 am
Instructions for switching graphics pack to use TWBT:

This is so the TWBT versions can have alpha transparency on the main tilesheet and creatures, and non-TWBT versions can have simplified language files and true black color palettes (for TTF fonts). And also there's some difference in other objects files that might be needed for keeping tiles in different spots for overrides or using TWBT's large color palette feature. And also different print modes.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on February 23, 2020, 03:06:54 am
I'm thinking about moving the onLoad files back into the /raw/ folder. I moved them out as part of separating TWBT stuff from non-TWBT stuff, but now that PyLNP can take the GitHub repo graphics packs and move all the files around to their proper places to enable the TWBT features (except for the onLoad file), and since the onLoad file shouldn't affect anything if it's installed without TWBT (except maybe some errors in the DFHack Terminal), it would probably be nice to have it back in the /raw/ folder so it's in the place it needs to be.

Just curious if anyone has any opinions for or against moving the onLoad_gfx.init file back to the /raw/ folder.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 23, 2020, 05:53:30 am
I don't know anything about TWBT and PyLNP. What's the difference? As far as I understand, the file needs to be in the /raw/ folder to work correctly, and if it's not there, users have to move it manually. Is that correct?
What about other LNP-like utilities? Do they all work the same?
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on February 23, 2020, 06:03:52 am
Yeah, it doesn't do anything unless it's in the /raw/ folder. A lot of the graphics packs include both a TWBT version and a non TWBT version. Those packs are in non-TWBT form until the TWBT parts of those packs are moved into the main parts. (e.g. moving /data/twbt_init/ into /data/init/).

I don't understand the question about LNP-like Utilities? Are there alternatives to PyLNP?
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 23, 2020, 03:49:34 pm
I don't understand the question about LNP-like Utilities? Are there alternatives to PyLNP?
I thought so. Isn't there one for Mac, one for Linux, the original LNP, etc? I'm probably wrong though, I never used these packs.

Those packs are in non-TWBT form until the TWBT parts of those packs are moved into the main parts.
So users will still have to move the other 4 folder contents, even if onLoad.init is placed by the maintainer?
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on February 24, 2020, 12:27:22 am
All three packs today use PyLNP, but I just checked out the old MacNewbie launcher made by iXen, and it still works on Mojave and probably Catalina too. It wouldn't be compatible with the separated TWBT components, but it's already not compatible with the reduced raw format used by PyLNP graphics packs.

If the onLoad.init is in a graphics pack's /raw/ folder, PyLNP and MacNewbie will both install it into the proper location. Before PyLNP was able to install the TWBT components itself, LNP maintainers would need to move the TWBT files into the proper folders if they wanted the TWBT version of the graphics pack in their LNP. Anyone wanting to make a pack with one of the predecessors to PyLNP should probably install those themselves if they want the TWBT or maybe include two copies of the same graphics pack if they want both versions.
Title: Re: Dwarf Fortress graphics repositories
Post by: CLA on February 24, 2020, 04:46:02 am
I see, thanks. So, back to the original question then: the proposed change looks good.
Title: Re: Dwarf Fortress graphics repositories
Post by: PeridexisErrant on February 29, 2020, 12:25:19 am
Sounds good to me too.  Thanks for all your work on this Jecowa <3
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on April 05, 2020, 02:59:45 am
I just figured out how to add graphics packs to the repo.
Title: Re: Dwarf Fortress graphics repositories
Post by: rmblr on July 30, 2020, 04:14:57 am
@fricy @PeridexisErrant Could I propose adding a "Colors" repo to the DFGraphics github org?

We could definitely use a single repo containing community colors.

Edit: scratch that, apparently the lnp shared core project has it https://github.com/Lazy-Newb-Pack/LNP-shared-core/tree/master/colors
Title: Re: Dwarf Fortress graphics repositories
Post by: FantasticDorf on July 30, 2020, 05:40:00 am
I thought i already P2W on this, but grim fortress isn't compatible with the new sets and needs some maintenance, mainly in the areas of updating the init from its download links with contempoary tree tiles.

It is very very pretty when it works however, some of my best forts.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on December 27, 2022, 01:52:56 pm
Is there any point in keeping the “ASCII” versions of these graphics packs updated ice they are converted to the new graphics format? I can go back and make a branch or something from an old commit at a later point if it turns out they’re needed, right?
Title: Re: Dwarf Fortress graphics repositories
Post by: FantasticDorf on December 28, 2022, 03:38:53 pm
Is there any point in keeping the “ASCII” versions of these graphics packs updated ice they are converted to the new graphics format? I can go back and make a branch or something from an old commit at a later point if it turns out they’re needed, right?

well right now, everything on the non-steam/itchio versions is stuck at a fixed 10x10 dimension or the menu's break the distortion, but otherwise it works just fine.
Title: Re: Dwarf Fortress graphics repositories
Post by: jecowa on December 30, 2022, 11:47:14 am
From what I can tell, the three fonts need to all be 8x12 and the game tiles need to be 32x32. I'm guessing these will both get fixed later, and the graphical version will have all the features of the "ASCII" version.