Bay 12 Games Forum

Dwarf Fortress => DF Modding => Utilities and 3rd Party Applications => Topic started by: Mason11987 on July 22, 2013, 04:52:23 pm

Title: World Viewer for DF 44! [Updated 12-16-17]
Post by: Mason11987 on July 22, 2013, 04:52:23 pm
World Viewer has been updated for DF 44!  Version 2.5.0

World Viewer provides an alternative way of viewing Dwarf Fortress world information (Similar to Legends Viewer by Parker147 (http://www.bay12forums.com/smf/index.php?topic=72702.0)), created by parsing information supplied by the export option in legends mode.  Here are some screenshots of World Viewer in action (http://imgur.com/a/ISGwl#0).

Download (Windows): on DFFD (http://dffd.bay12games.com/file.php?id=7851) - 64 bit version here (http://dffd.bay12games.com/file.php?id=11454) (use the first link if you aren't sure which you need)  Source is on BitBucket (https://bitbucket.org/Mason11987/world-viewer).

Spoiler: Select screenshots (click to show/hide)


Please let me know what you think and if you have any suggestions!
Title: Re: World Viewer Updated! (for 34.11)
Post by: PeridexisErrant on July 22, 2013, 07:48:13 pm
PTW
Title: Re: World Viewer Updated! (for 34.11)
Post by: laularukyrumo on July 23, 2013, 12:27:14 am
Figures this would come out right as 34.11 is on the chopping block. :P Hoping you update it for the new version when it comes around though.
Title: Re: World Viewer Updated! (for 34.11)
Post by: Matoro on July 23, 2013, 02:31:22 am
So it's like a Legends Viewer but better?
Title: Re: World Viewer Updated! (for 34.11)
Post by: Mason11987 on July 23, 2013, 05:46:18 am
Figures this would come out right as 34.11 is on the chopping block. :P Hoping you update it for the new version when it comes around though.

Yeah.... I just kept adding features trying to wait Toady out but I didn't have anything else I could think of :).

Edit -

I completely forgot to mention I included complex (and simple) filtering, using the textbox and the "filter" button at the bottom of every list of things.  This is a screen shot of the filter screen accessed from that button:

Spoiler: Filtering (click to show/hide)

The textbox filters like legends mode does, or at least as close as I could get.
Title: Re: World Viewer Updated! (for 34.11)
Post by: ORCACommander on June 19, 2014, 08:24:40 pm
i am gona bump this since my 10k year world gen forced him to do some overhauls :P
Title: Re: World Viewer Updated! (for 34.11)
Post by: BigD145 on June 19, 2014, 10:06:02 pm
Figures this would come out right as 34.11 is on the chopping block. :P Hoping you update it for the new version when it comes around though.

It shouldn't be that hard to update since it's just taking legends exports.
Title: Re: World Viewer Updated! (for 34.11)
Post by: Mason11987 on June 20, 2014, 02:16:16 pm
It depends on how much has changed. The last update gave a ton of new data but little of the old stuff was changed and that will probably be the case with this one too (hopefully).  Like orca said though his 4.7 GB XML file made me improve some things and optimize the whole process. It also fails a lot more gracefully when you try to kill it with a file that big on a 32 bit os.  Finally that file showed me a few rare world gen situations I hadn't seen before. So all that and a few more details will be incorporated into the new version as well as any new world details.

I'm most excited to be able to retire multiple forts and have things happen in the world around them.
Title: Re: World Viewer Updated! (for 34.11)
Post by: Timeless Bob on June 21, 2014, 03:26:28 am
I'd suggest you run this bad boy on the latest save file for "The Museum".  I think there's 80+ years of Player-made history in there, not to mention some pretty interesting embarks.
Title: Re: World Viewer Updated! (for 34.11)
Post by: Mason11987 on June 23, 2014, 12:55:48 pm
I'd suggest you run this bad boy on the latest save file for "The Museum".  I think there's 80+ years of Player-made history in there, not to mention some pretty interesting embarks.

Good call, was able to run this and it actually identified a bug (something called "autochop" stores stuff in a historical figure ID -100?) which I addressed, and three historical events I had not ever actually seen, and only knew existed because today had posted the full list of events, two of which provide a lot of good data "site died" and "hf reached summit".  Thanks for the suggestion!  I'm going to address these and maybe use this as a test bed for some story generation ideas I've had.
Title: Re: World Viewer Updated! (for 34.11)
Post by: PeridexisErrant on June 23, 2014, 05:13:53 pm
This is a know bug - dfhack stores lot of data in fake historical figures in binary format, which then corrupts xml exports.  I think it's meant to be fixed in r5, but I haven't tested that yet and it's good to hear it can be addressed on your end too (since exporting from a game without dfhack would keep the corruption). 
Title: Re: World Viewer Updated! (for 34.11)
Post by: ORCACommander on June 23, 2014, 11:31:16 pm
want to run some 10k's on smaller worlds as well?
Title: Re: World Viewer Updated! (for 34.11)
Post by: Mason11987 on June 25, 2014, 07:22:54 am
This is a know bug - dfhack stores lot of data in fake historical figures in binary format, which then corrupts xml exports.  I think it's meant to be fixed in r5, but I haven't tested that yet and it's good to hear it can be addressed on your end too (since exporting from a game without dfhack would keep the corruption).

Yeah, I just assume that all negative IDs in the XML are ignored now (you get a note about it when it happens).  I don't see any reason for Toady to start using them.  If something changes (like a dfhack add-on creates data in those that might be useful to me) I can check it out then.

As far as I know it doesn't really corrupt the export outside of adding a weird additional item with a negative id.  Unless there are some examples of dfhack that does break it I've never seen.  Feel free to forward me any good examples you have of that.

want to run some 10k's on smaller worlds as well?

Yeah, I did this a lot while testing, since they were great ways of testing huge family trees.
Title: Re: World Viewer Updated! (for 34.11)
Post by: Mason11987 on July 08, 2014, 07:37:42 am



It looks like there are two sets of events that occur right at the beginning of each world, which are related to the new Created Site, and Created Structure events.

Here are some details (for me) and for anyone who cares:

All together

All together

It looks like if the first set exists, the other does as well for the same Demon, but there may be multiple ones of these, and all of the multiples of the first set come before the second set.



Now just need to get my hands on some saves which have a decent amount of play time on a fortress to verify nothings changed there, and I should be able to update this today.
Title: Re: World Viewer Updated! (for DF 40.01)
Post by: Mason11987 on July 09, 2014, 08:23:54 am
World Viewer has been updated!

There is likely still some work to do, my testing for player-ran forts has been limited.  So if you run into any issues with mature forts please let me know and upload your save and I'll take a look.

There are a significant amount of changes since the last release, including of course support for the changes in world generation in the new version.  I'll post a summary of the other changes soon.


So let me know what you think!

DFFD Link (http://dffd.wimbli.com/file.php?id=7851)

Edit - Fixed a bug with filtering on Race/Caste
Edit - Made a few changes to handle player-forts better (including new event Site Retired)
Title: Re: World Viewer for DF 40.01!
Post by: cephalo on July 10, 2014, 01:56:49 pm
It's been a long time since I've used this utility. Are there some instructions anywhere? I have a 'load world' button which seems to be look for a map .bmp file, but when I give it one it doesn't like it. I don't remember how to get started looking at worlds.

EDIT: Ok, got it working. However, I'm finding it very crash-happy on Windows 7. I'd like to upload my files, but that's over 200MB. Is it working for you?

EDIT2: Looks like a null reference exception.
Title: Re: World Viewer for DF 40.01!
Post by: Robsoie on July 10, 2014, 03:08:35 pm
I've been trying the 32 bits version with 0.40.01.

I noticed inside the 32bits directory there's a folder named x64 for some reason and i launch the program i load the map i exported from DF, then the program load the XML (that i had exported too), and crash suddenly with "DFWV has encountered an error..."

I decide to rename the x64 subfolder into x86 and it does exactly the same thing.

It's a small world with less than 100 years of history, played a fortress that i retired and an adventurer on it
screenshot of the program state at the time of crash :

(http://i.imgur.com/igiBlGvs.jpg) (http://i.imgur.com/igiBlGv.jpg)
Title: Re: World Viewer for DF 40.01!
Post by: ORCACommander on July 10, 2014, 09:59:11 pm
I still plan on doing a 10k again bt am having trouble genning past 500 years without lock ups
Title: Re: World Viewer for DF 40.01!
Post by: Mason11987 on July 10, 2014, 10:47:36 pm
It's been a long time since I've used this utility. Are there some instructions anywhere? I have a 'load world' button which seems to be look for a map .bmp file, but when I give it one it doesn't like it. I don't remember how to get started looking at worlds.

EDIT: Ok, got it working. However, I'm finding it very crash-happy on Windows 7. I'd like to upload my files, but that's over 200MB. Is it working for you?

EDIT2: Looks like a null reference exception.

Just upload a new version and I properly included the readme this time. direct download link (http://dffd.wimbli.com/download.php?id=7851&f=World+Viewer+df40.01.zip).  Give that a shot, a few issues were reported on reddit that I addressed.  If you have the same errors there might be some additional info when that error comes up, if you have that it might help.

I've been able to load multiple GB files so that shouldn't be the issue.

I've been trying the 32 bits version with 0.40.01.

I noticed inside the 32bits directory there's a folder named x64 for some reason and i launch the program i load the map i exported from DF, then the program load the XML (that i had exported too), and crash suddenly with "DFWV has encountered an error..."

I decide to rename the x64 subfolder into x86 and it does exactly the same thing.

It's a small world with less than 100 years of history, played a fortress that i retired and an adventurer on it
screenshot of the program state at the time of crash :

(http://i.imgur.com/igiBlGvs.jpg) (http://i.imgur.com/igiBlGv.jpg)

I resolved an issue related to adventurer worlds (and the misplaced folder deal) (hadn't been able to test that enough this release) that might help if you want to redownload (http://dffd.wimbli.com/file.php?id=7851).  Let me know if your issues are resolved

I still plan on doing a 10k again bt am having trouble genning past 500 years without lock ups

Sounds good, I'm going to try to gen one myself to do some more testing.
Title: Re: World Viewer for DF 40.01!
Post by: eternaleye on July 11, 2014, 12:29:45 am
Hey, this works under Mono on Linux except for one teensy bug:

In MainForm.cs, on lines 102-105 and 132, you concatenate paths as if they were strings using '\\' (unescaping to '\') as the path separator. On Linux, the path separator is a _forward_ slash '/', and thus it looks for the _file_ parentdir\filename in those cases (and fails, unless you humor it and name a second copy of the files accordingly).

If you use Path.Combine( workingFolder, tempName + <per-file-suffix> ), it'll Just Work regardless of OS.
Title: Re: World Viewer for DF 40.01!
Post by: BigD145 on July 11, 2014, 01:21:04 am
01 files were causing crashes but 02 files seemed to work from the viewer downloaded july 9th. This was win7. Will try new version soon.
Title: Re: World Viewer for DF 40.01!
Post by: Kraetzin on July 11, 2014, 07:33:03 am
Great tool! This makes adventure mode 100x more playable. I've got one question though. Is it possible to filter sites on the map based on type? Mainly I'm after filtering the neutral sites, to only display ruins (ie. abandoned hamlets, hillocks, towns, forts etc). It'd be useful to be able to filter out a lot of the noise you get from the various other neutral sites.
Title: Re: World Viewer for DF 40.01!
Post by: Robsoie on July 11, 2014, 08:12:40 am
I resolved an issue related to adventurer worlds (and the misplaced folder deal) (hadn't been able to test that enough this release) that might help if you want to redownload (http://dffd.wimbli.com/file.php?id=7851).  Let me know if your issues are resolved
This new version is working for me, it managed to load the map then the xml apparently without a problem and no crashing.
Thank you.
Title: Re: World Viewer for DF 40.01!
Post by: cephalo on July 11, 2014, 09:56:58 am
I have this working as well. Great!

I have one little confusion to report so far. When someone marries an 'unknown creature', presumably a non-historical figure, they are listed as having married themselves. I guess that's ok, but it made me look it up just to figure that out.

Nah, nah, I get it now, it just has no target for the event. Makes sense now. Might be nicer to say that the target is unknown.
Title: Re: World Viewer for DF 40.01!
Post by: Mason11987 on July 11, 2014, 09:04:34 pm
Glad it's working well for you guys now. 

Made a few tweaks as suggested/requested:

Hey, this works under Mono on Linux except for one teensy bug:

In MainForm.cs, on lines 102-105 and 132, you concatenate paths as if they were strings using '\\' (unescaping to '\') as the path separator. On Linux, the path separator is a _forward_ slash '/', and thus it looks for the _file_ parentdir\filename in those cases (and fails, unless you humor it and name a second copy of the files accordingly).

If you use Path.Combine( workingFolder, tempName + <per-file-suffix> ), it'll Just Work regardless of OS.

Awesome.  Got it, and fixed.

Great tool! This makes adventure mode 100x more playable. I've got one question though. Is it possible to filter sites on the map based on type? Mainly I'm after filtering the neutral sites, to only display ruins (ie. abandoned hamlets, hillocks, towns, forts etc). It'd be useful to be able to filter out a lot of the noise you get from the various other neutral sites.

Cool idea, pretty sure I got it in this update.  Take a look and let me know what you think.  As an alternative, you can also filter to get that in the site list, and then click the coordinates to show that item on the map.

I have this working as well. Great!

I have one little confusion to report so far. When someone marries an 'unknown creature', presumably a non-historical figure, they are listed as having married themselves. I guess that's ok, but it made me look it up just to figure that out.

Nah, nah, I get it now, it just has no target for the event. Makes sense now. Might be nicer to say that the target is unknown.

Could you possibly send me a screenshot?  I'm not 100% sure what you're referring to.  There are some issues with the Add HF HF link events because it only gives you the two Historical figures, and no details about what type of event it is.
Title: Re: World Viewer Updated! (for 34.11)
Post by: Raul on July 14, 2014, 05:12:35 pm
...some story generation ideas I've had.
Please, do tell us more.
Title: Re: World Viewer for DF 40.01!
Post by: KillzEmAllGod on July 14, 2014, 07:12:13 pm
I have noticed this doesn't show elf populations right, Legends viewer is what i have been useing to find the real population of them if it doesn't crash.
Its pretty good though it needs to show Civ statics and something easier for finding megabeats, nercomancers and other fun stuff.
Theres alot of information that just needs to be displayed alot better.

Found out that goblin fortresses get 10k population pretty quick on any world they're on even with goblin maxage set to 30-50
I did like how they kept changing their leaders every few years if the demon died.
Title: Re: World Viewer for DF 40.01!
Post by: Mason11987 on July 14, 2014, 09:24:18 pm
I have noticed this doesn't show elf populations right, Legends viewer is what i have been useing to find the real population of them if it doesn't crash.
Its pretty good though it needs to show Civ statics and something easier for finding megabeats, nercomancers and other fun stuff.
Theres alot of information that just needs to be displayed alot better.

Found out that goblin fortresses get 10k population pretty quick on any world they're on even with goblin maxage set to 30-50
I did like how they kept changing their leaders every few years if the demon died.

So there are a few issues with populations.  Mostly around which numbers to use.

There are civ population numbers at the top of the sites file, there are individual site population numbers below that, and there are the list of historical figures.  At the moment I believe the numbers listed on the civilizations tab are based on the numbers at the top of the sites file, since from my understanding that's the broadest measure.

I'm not aware of exactly how Legends Viewer does it if it comes out with a different number.

I'll try to pin down necromancers and megabeasts more though, you can expect that to be a filter option in the next update.

If you have some specifics about layout I'd be glad to hear them.  I mostly play fortress mode so what is most interesting to someone who plays a lot of adventure mode might not be obvious to me.

...some story generation ideas I've had.
Please, do tell us more.

When I've genned worlds and used World Viewer I've tried to dig into some interesting stories about people who have existed, but finding out what makes someone or some event interesting is tricky.  Essentially I'm trying to work on a way of distilling the legends mode stories people develop by stumbling across an interesting thread and following it to find a story.  I'd like to be able to find these and get them up front.  The one thing I've found by testing and analyzing hundreds of worlds is how crazy the variety is, but "what's interesting" is complicated but ultimately that's what people really want out of something like this.  Giving them easy ways to navigate the data is one step, but giving some tips on where to look first would be better.
Title: Re: World Viewer for DF 40.01!
Post by: KillzEmAllGod on July 15, 2014, 06:25:12 am
If you have some specifics about layout I'd be glad to hear them.  I mostly play fortress mode so what is most interesting to someone who plays a lot of adventure mode might not be obvious to me.

Spoiler (click to show/hide)
Before.

You can see how it doesn't tell you much, With all the leaders its not clear about whos who and who is dead out of them.

Spoiler (click to show/hide)

If you had a layout like this for Civs it would be easier to get the details about them like their population, the type of sites they own and its pop.
You might want to have site history for sites they once had or owned at some point.
Leaders really needs to be redone to something like this so it makes more sense then alot of names as it was before. Goes With By rank or title then their number then the year they took it as well as race.

Wars is another thing you need to show as well as events such as the civ being attacked by a megabeast or something.
Thats just the Civ screen.

Sorry about that.
Title: Re: World Viewer for DF 40.01!
Post by: PeridexisErrant on July 15, 2014, 06:32:17 am
Looks like a nice set of improvements - but please stick large images in a spoiler, if you're loading them inline!
Title: Re: World Viewer for DF 40.01!
Post by: ORCACommander on July 15, 2014, 10:34:53 am
I think i may have found my genning problem. toady says there was a bug making millions of people. lets see how 40.04 goes when it comes out
Title: Re: World Viewer for DF 40.01!
Post by: Mason11987 on July 15, 2014, 04:04:45 pm
<snip>

I like your layout suggestions, all of that is definitely do-able and now on the short-list.  Thanks.

Edit -
Spoiler: Progress! (click to show/hide)
Title: Re: World Viewer for DF 40.01!
Post by: cephalo on July 15, 2014, 04:14:11 pm

I like your layout suggestions, all of that is definitely do-able and now on the short-list.  Thanks.

I'll definitely be watching out for the next version. Thanks for making this super useful tool.
Title: Re: World Viewer for DF 40.01!
Post by: KillzEmAllGod on July 16, 2014, 03:28:18 pm
<Progress!>
You able to use a different font and colour plus some more space for the race?
Title: Re: World Viewer for DF 40.01!
Post by: Mason11987 on July 16, 2014, 03:56:16 pm
<Progress!>
You able to use a different font and colour plus some more space for the race?

It looks like trying to do coloring has a performance impact with the listbox.  I'm probably going to just go with one color to highlight sections since I think using multiple colors for text is only helpful if that provides additional information (like blue/red for male/female HF in some places).  Since the same code will be used for all sorts of grouping (Region types, site Types, civilization races, historical event types, etc. etc.) trying to assign a variety of colors might not be useful.

Spoiler: A little more progress (click to show/hide)
Title: Re: World Viewer for DF 40.01+!
Post by: PeridexisErrant on July 16, 2014, 06:38:40 pm
Yessss.....   It's almost unbelievable how much more information that screen conveys just by adding the second and third columns. 
Title: Re: World Viewer for DF 40.01+!
Post by: Graknorke on July 16, 2014, 06:52:45 pm
Ooh, I am a big fan of third-party programs like these.
Not ones that help with gameplay, those are mostly superfluous. But the amount of information that creates the background to the world is staggering, and it's pretty much impossible to keep track of how everyone is related to everyone else by memory alone.
Title: Re: World Viewer for DF 40.01!
Post by: KillzEmAllGod on July 16, 2014, 08:04:30 pm
I didn't know what you were referring to by the numbers on your example, birth/death?

First one was if they were 1st or 49th queen just to give some order second one of when they took it so you get a feeling of how many years they held it for and when.

Could you do colour for the sites types?
At least do something so its easier to read the info then just all the same text and colour at some point.
Title: Re: World Viewer for DF 40.01+!
Post by: Mason11987 on July 16, 2014, 08:15:29 pm
Yessss.....   It's almost unbelievable how much more information that screen conveys just by adding the second and third columns.

Yeah I definitely think it's a great addition, I definitely focused more on other areas but these changes are definitely fleshing it out.  (I also added something for dealing with zip files!)


First one was if they were 1st or 49th queen just to give some order second one of when they took it so you get a feeling of how many years they held it for and when.

Could you do colour for the sites types?
At least do something so its easier to read the info then just all the same text and colour at some point.

Ah okay, they come in order by type.  But I think how long they served would be a good addition.

How bout this for the groupings along the left?

Spoiler: More Progress (click to show/hide)

Title: Re: World Viewer for DF 40.01+!
Post by: PeridexisErrant on July 16, 2014, 08:59:49 pm
Legends Viewer just updated for 0.40, but I think I might not go back :D

Only suggestion:  use the ISO standard for dates; YYYY-MM-DD
Title: Re: World Viewer for DF 40.01+!
Post by: Mason11987 on July 16, 2014, 09:52:30 pm
Updated, here's a direct link (http://dffd.wimbli.com/download.php?id=7851&f=World+Viewer+df40.0X.zip)

(Note: It doesn't behave quite right if you try to group by two fields at once... so don't try that for now)
Title: Re: World Viewer for DF 40.01+!
Post by: KillzEmAllGod on July 16, 2014, 10:19:43 pm
How bout this for the groupings along the left?
Looks much better though i would like a different font or it being bold to stand out from the other text there.

For the site one could you get it so you can sort it by type and population and something to point out where the Leader is ruling from.
Just need to add in small thing for population by top 5 sentients could do a more detailed population count under Enity population for everything other then intelligent.

The leader one just needs to also say their gender maybe merge the race with it so its like Female Human or male human instead of humans.

Spoiler (click to show/hide)
Title: Re: World Viewer for DF 40.01+!
Post by: Mason11987 on July 17, 2014, 08:44:33 am
Looks much better though i would like a different font or it being bold to stand out from the other text there.

For the site one could you get it so you can sort it by type and population and something to point out where the Leader is ruling from.
Just need to add in small thing for population by top 5 sentients could do a more detailed population count under Enity population for everything other then intelligent.

The leader one just needs to also say their gender maybe merge the race with it so its like Female Human or male human instead of humans.

I'll look at making it stand out more and sorting the site list on the civ tab.

I haven't actually collected the data for where he's ruling from, but that should be available from the sites file, so I'll get that displayed somewhere.    Edit - It looks like it's only available for some leaders.  The others can be assumed from Change HF State events, but some have no way of pinpointing where he's located.

I'm not sure what this is saying "Just need to add in small thing for population by top 5 sentients could do a more detailed population count under Enity population for everything other then intelligent."

I went ahead and dug into plurals/singulars a bit and fixed all that stuff with things like Dwarves showing up where Dwarf should and the reverse.  Actually a lot simpler then I thought since I didn't have to write my own code to pluralize/singularize things.
Title: Re: World Viewer for DF 40.01+!
Post by: KillzEmAllGod on July 17, 2014, 04:29:12 pm
I'm not sure what this is saying "Just need to add in small thing for population by top 5 sentients could do a more detailed population count under Enity population for everything other then intelligent."
Just somthing to show the Total poplation and the races that make up it up.
Title: Re: World Viewer for DF 40.0X!
Post by: PeridexisErrant on July 17, 2014, 11:53:56 pm
Quick question:  what would be in the zip file to include everything World Viewer can use? 

For Legends Viewer it's the XML, sites/pops txt, history txt, and a map; but I can add more with the porcessing script since additional files are just ignored without error. 
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 18, 2014, 05:39:55 am
I'm not sure what this is saying "Just need to add in small thing for population by top 5 sentients could do a more detailed population count under Enity population for everything other then intelligent."
Just somthing to show the Total poplation and the races that make up it up.

For entity populations, like on this screen?

Spoiler (click to show/hide)

Quick question:  what would be in the zip file to include everything World Viewer can use? 

For Legends Viewer it's the XML, sites/pops txt, history txt, and a map; but I can add more with the porcessing script since additional files are just ignored without error.

Same thing, additional stuff is just ignored.




Big news though!  I've begun work on a DFHack script that will do the normal export all, but will also export an additional file which should fill in all the gaps that the current xml file has.

I'm working with Parker147 (of Legends Viewer (http://www.bay12forums.com/smf/index.php?topic=72702.msg1788452#msg1788452)) who's sharing his own notes on where he's found the xml to be lacking.   It will be a bonus file that you can get if you have dfhack and world viewer will just read it and parse it if it finds it, but continue just fine if it doesn't.

Short term goals are to be able to fill every Unknown in every event, including names of structures, items (which do the killings most of the time) and more.
Long term goals are to dump anything else I find interesting into the same format in the extra file.  This depends on what's loaded into memory while in legends mode, which is a ton of stuff.

Right now I'm working with the previous version of DF since DFHack hasn't been updated, but by the time I get a script finished exporting the missing details I bet DFHack will be updated.
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 21, 2014, 07:59:22 pm
Here's a bit of a teaser for the new stuff I've been able to pull out of legends mode using DFHack:

Spoiler (click to show/hide)

Unfortunately underground regions overlap eachother so it just looks like a mess, but I'll find a way to include that as well.

For now this is only for the previous version of DF (as DFHack hasn't been updated) but the script should work fine whenever it is, and then I'll release an update to support the additional exported details.
Title: Re: World Viewer for DF 40.0X!
Post by: Raul on July 21, 2014, 08:07:46 pm
Here's a bit of a teaser for the new stuff I've been able to pull out of legends mode using DFHack:

Spoiler (click to show/hide)
That's... wow. I can't even find words to explain how sweet that is.
Title: Re: World Viewer for DF 40.0X!
Post by: BigD145 on July 22, 2014, 12:00:40 am
yeeeeessssssss
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 22, 2014, 07:56:39 am
Ah, and the underground regions can be mapped correctly as well.  I hadn't considered that there were multiple layers.  Turns out that there is a full map for every level of cavern (3 by default), one for magma, and one for underworld.  I haven't played enough adventure mode to know if those maps would be useful, but they'll be in there as well because I love how the map drawing turned out.  I figure if you want to map the underworld you'll just pick a depth.
Title: Re: World Viewer for DF 40.0X!
Post by: therahedwig on July 22, 2014, 08:17:49 am
Glad it's working well for you guys now. 

Made a few tweaks as suggested/requested:

Hey, this works under Mono on Linux except for one teensy bug:

In MainForm.cs, on lines 102-105 and 132, you concatenate paths as if they were strings using '\\' (unescaping to '\') as the path separator. On Linux, the path separator is a _forward_ slash '/', and thus it looks for the _file_ parentdir\filename in those cases (and fails, unless you humor it and name a second copy of the files accordingly).

If you use Path.Combine( workingFolder, tempName + <per-file-suffix> ), it'll Just Work regardless of OS.

Awesome.  Got it, and fixed.

Ah, uhm, when trying the 64bit version on Linux I get this error:

err:process:start_process L"Z:\\mnt\\6A0F06C454D4F476\\games\\df\\64bit\\DFWV.exe" doesn't have an entry point, it cannot be executed

May that perhaps be related?
(The 32bit version gives me this instead:
Spoiler (click to show/hide)
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 22, 2014, 09:21:03 am
Unfortunately, I have essentially zero experience with Linux, I was surprised it worked at all.  None of those errors are anything I'm familiar with.  It seems like it's not handling the config file correctly on the 32bit.

eternaleye may have some tips.  Not sure if he had to compile it himself on Linux but if so I'd gladly host his version in the OP as well.  I sent him a PM to see if he'll comment.
Title: Re: World Viewer for DF 40.0X!
Post by: cephalo on July 22, 2014, 09:34:54 am
Looking forward to the new maps! I'm wondering if it would be possible for us to change the tileset used to draw? I have a bit of trouble understanding the graphical tilesets as I play with largely clean ones.

EDIT: It would be very interesting to see how things like volcanism influences the underworld regions.
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 22, 2014, 11:42:06 am
Looking forward to the new maps! I'm wondering if it would be possible for us to change the tileset used to draw? I have a bit of trouble understanding the graphical tilesets as I play with largely clean ones.

EDIT: It would be very interesting to see how things like volcanism influences the underworld regions.

World viewer doesn't actually draw the tileset, it just draws on top of the maps exported from legends mode.  Those maps are based on the tilesets used at that time.  The particular map I showed was from a world I like to test on (It's actually a community map called The Museum).  The map was exported with a tileset in use.  If you exported with a different tileset you'd get a different map and worldviewer would draw on that.  So the main map (not the graphical ones) will look just like it does in-game.
Title: Re: World Viewer for DF 40.0X!
Post by: eternaleye on July 22, 2014, 01:26:38 pm
Ah, uhm, when trying the 64bit version on Linux I get this error:

err:process:start_process L"Z:\\mnt\\6A0F06C454D4F476\\games\\df\\64bit\\DFWV.exe" doesn't have an entry point, it cannot be executed

May that perhaps be related?
(The 32bit version gives me this instead:
Spoiler (click to show/hide)

At a guess, you're trying to run it in Wine? That's not necessary - It's a .NET program, which means it's compiled to a bytecode representation instead of native code (the bytecode is for the "Common Language Interpreter", or "CLI") which Microsoft in its infinite wisdom decided to embed in the PE format and name .exe like regular executables. Run it with "mono" instead, as "mono DFWV.exe"

C#/.NET is basically Java, with all the branding sharpied over - just like you'd run a java program with java -jar <file>, you run .NET programs with mono <file> (on Linux, at least)

As a result, as long as the programmer doesn't make _semantic_ mistakes affecting portability (see above regarding path separators) you can run the same compiled .NET program on any OS. So no, I didn't have to recompile :D
Title: Re: World Viewer for DF 40.0X!
Post by: therahedwig on July 22, 2014, 01:31:36 pm
Ah, uhm, when trying the 64bit version on Linux I get this error:

err:process:start_process L"Z:\\mnt\\6A0F06C454D4F476\\games\\df\\64bit\\DFWV.exe" doesn't have an entry point, it cannot be executed

May that perhaps be related?
(The 32bit version gives me this instead:
Spoiler (click to show/hide)

At a guess, you're trying to run it in Wine? That's not necessary - It's a .NET program, which means it's compiled to a bytecode representation instead of native code (the bytecode is for the "Common Language Interpreter", or "CLI") which Microsoft in its infinite wisdom decided to embed in the PE format and name .exe like regular executables. Run it with "mono" instead, as "mono DFWV.exe"

C#/.NET is basically Java, with all the branding sharpied over - just like you'd run a java program with java -jar <file>, you run .NET programs with mono <file> (on Linux, at least)
Ah, I see!
I've only been a year fulltime on Linux and hadn't had to use wine or mono-ran programs at all in that year.
It's only DF applications that need that, so thanks :)
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 23, 2014, 11:21:34 am
Ah, uhm, when trying the 64bit version on Linux I get this error:

err:process:start_process L"Z:\\mnt\\6A0F06C454D4F476\\games\\df\\64bit\\DFWV.exe" doesn't have an entry point, it cannot be executed

May that perhaps be related?
(The 32bit version gives me this instead:
Spoiler (click to show/hide)

At a guess, you're trying to run it in Wine? That's not necessary - It's a .NET program, which means it's compiled to a bytecode representation instead of native code (the bytecode is for the "Common Language Interpreter", or "CLI") which Microsoft in its infinite wisdom decided to embed in the PE format and name .exe like regular executables. Run it with "mono" instead, as "mono DFWV.exe"

C#/.NET is basically Java, with all the branding sharpied over - just like you'd run a java program with java -jar <file>, you run .NET programs with mono <file> (on Linux, at least)
Ah, I see!
I've only been a year fulltime on Linux and hadn't had to use wine or mono-ran programs at all in that year.
It's only DF applications that need that, so thanks :)

Just for future reference did this fix your problem?   If so I'll add it as a note in the OP for linux users.  Thanks!
Title: Re: World Viewer for DF 40.0X!
Post by: therahedwig on July 23, 2014, 11:34:40 am
Ah, uhm, when trying the 64bit version on Linux I get this error:

err:process:start_process L"Z:\\mnt\\6A0F06C454D4F476\\games\\df\\64bit\\DFWV.exe" doesn't have an entry point, it cannot be executed

May that perhaps be related?
(The 32bit version gives me this instead:
Spoiler (click to show/hide)

At a guess, you're trying to run it in Wine? That's not necessary - It's a .NET program, which means it's compiled to a bytecode representation instead of native code (the bytecode is for the "Common Language Interpreter", or "CLI") which Microsoft in its infinite wisdom decided to embed in the PE format and name .exe like regular executables. Run it with "mono" instead, as "mono DFWV.exe"

C#/.NET is basically Java, with all the branding sharpied over - just like you'd run a java program with java -jar <file>, you run .NET programs with mono <file> (on Linux, at least)
Ah, I see!
I've only been a year fulltime on Linux and hadn't had to use wine or mono-ran programs at all in that year.
It's only DF applications that need that, so thanks :)

Just for future reference did this fix your problem?   If so I'll add it as a note in the OP for linux users.  Thanks!
Yes it did :)
Title: Re: World Viewer for DF 40.0X!
Post by: ORCACommander on July 24, 2014, 12:18:35 am
I'll have a 129x129 10k year gen uploaded in the morning
Title: Re: World Viewer for DF 40.0X!
Post by: ORCACommander on July 24, 2014, 10:06:41 am
double posting  because i doubt an edit will make people notice

http://www.gamefront.com/files/?filepath=bloodreign//Dwarf-Fortress/df_40_04_10k-exports.rar
Title: Re: World Viewer for DF 40.0X!
Post by: cephalo on July 25, 2014, 10:37:05 am
I have a small request for this app. Would it be possible to count the populations of all historical figures according to species? It would be nice to know if those histfig pop numbers get out of whack.

For example, I believe in DF2014 that goblin histfigs, in spite of the high numbers of goblins in the abstracted population, tend to have very low populations compared to the other civilized histfigs. This leaves almost all the goblin dirty work in world gen to descendants of kidnap victims of other species.

Whether that's a bug or a feature is hard to say, but in any case it would be nice to have proof of whether these numbers are low, or just seem low.
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 25, 2014, 01:34:39 pm
I have a small request for this app. Would it be possible to count the populations of all historical figures according to species? It would be nice to know if those histfig pop numbers get out of whack.

For example, I believe in DF2014 that goblin histfigs, in spite of the high numbers of goblins in the abstracted population, tend to have very low populations compared to the other civilized histfigs. This leaves almost all the goblin dirty work in world gen to descendants of kidnap victims of other species.

Whether that's a bug or a feature is hard to say, but in any case it would be nice to have proof of whether these numbers are low, or just seem low.

If I'm not mistaken that's covered by the same update that made this work with the new DF version:

Spoiler (click to show/hide)
Title: Re: World Viewer for DF 40.0X!
Post by: cephalo on July 28, 2014, 09:44:02 am

If I'm not mistaken that's covered by the same update that made this work with the new DF version:


Is it possible to see how many of those histfigs are dead?

It doesn't happen every time, but in many of my worlds, when I look up a goblin name like 'amxu', I find that goblins are outnumbered 10 to 1 compared to elves, humans and dwarves. Before I commit to a world, it would be nice to make sure it's nice and balanced.

EDIT: Also, what is the official way to know what version of world viewer I'm using/downloading? Do you just update the file version number on DFFD? Checking on the exe file properties just says 1.0 while in DFFD it's given as 1.6.

EDIT: Ok, one more thing! World viewer crashes on this world generated in 40.05 (http://dffd.wimbli.com/file.php?id=9186). Maybe something changed?
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 28, 2014, 04:04:26 pm

If I'm not mistaken that's covered by the same update that made this work with the new DF version:


Is it possible to see how many of those histfigs are dead?

It doesn't happen every time, but in many of my worlds, when I look up a goblin name like 'amxu', I find that goblins are outnumbered 10 to 1 compared to elves, humans and dwarves. Before I commit to a world, it would be nice to make sure it's nice and balanced.

Not easily, The best you could do is filter by race, and then by dead or not using the filter button at the bottom of the historical figure list, but that only gives you a list, not a count.  Since there are a million types of things people could want to filter on I'll work on a way to easily see how many objects are in the list when you add a filter.

EDIT: Also, what is the official way to know what version of world viewer I'm using/downloading? Do you just update the file version number on DFFD? Checking on the exe file properties just says 1.0 while in DFFD it's given as 1.6.

I'll make sure it's included in a visible place on the next update, and is consistent.  Thanks for the heads up.  For now just referring to the version number on DFFD is best.


EDIT: Ok, one more thing! World viewer crashes on this world generated in 40.05 (http://dffd.wimbli.com/file.php?id=9186). Maybe something changed?

Thanks, I'll check this out ASAP.

Title: Re: World Viewer for DF 40.0X!
Post by: PeridexisErrant on July 28, 2014, 04:23:06 pm
I'm betting it's some of the many raw and language changes in 40.05 causing the error.
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 28, 2014, 07:46:55 pm
I'm betting it's some of the many raw and language changes in 40.05 causing the error.

Nope, I don't really deal with anything from raws or language.  It was actually one of those situations where in an effort to coax a bit more out of the data I made an assumption that isn't always true.

Details: Turns out that sometimes there can be Abduction event collections where the abduction is done by a historical figure who was culled from the xml export.  I mistakenly assumed that wouldn't happen and in several dozen worlds it never did.

Put the version in the title (and made it correctly end up in file properties).  I'm holding out release until there is a version of dfhack for v40 that I can test my new bonus xml script against.  And I'm also trying to make sure the export incorporates all the new details.  Speaking of it.  These are the things I've been able to pull from dfhack:

Title: Re: World Viewer for DF 40.0X!
Post by: PeridexisErrant on July 28, 2014, 09:59:32 pm
Having lost an unspecified bet, I'll replace the nonexistent forfeit with excitement about the new information - it looks great!
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on July 30, 2014, 01:11:32 pm
Gathering extra details about world constructions turned out good as well!

This is the 10k year world for the previous version that ORCA was kind enough to share.

Think lines are roads, curves are bridges, Dotted lines are tunnels.  The colors are the color of the civ who built the site.

Spoiler (click to show/hide)

Title: Re: World Viewer for DF 40.0X!
Post by: Parker147 on July 30, 2014, 08:00:16 pm
That is awesome. I can't wait to start working with this new stuff.
Title: Re: World Viewer for DF 40.0X!
Post by: Lewa263 on August 04, 2014, 02:58:06 pm
In the Races tab, are the populations listed only for historical figures, or is it adding up the population from each site? Because it usually lists there being 0 trolls, but looking at individual sites there are tens of thousands.
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on August 04, 2014, 03:19:26 pm
In the Races tab, are the populations listed only for historical figures, or is it adding up the population from each site? Because it usually lists there being 0 trolls, but looking at individual sites there are tens of thousands.

It actually only uses the counts from the world_sites_and_pops.txt file. 

Which means it shouldn't be showing zero for trolls in most cases...

Ha!  Looks like a bug.  And one parsing the world sites file no less, that was one of the first things I made.  Can't believe I never noticed that!

Essentially everything under "Underground Animal Populations (Including Undead)" is ignored.  Thanks for reporting this.  I'll have this fixed in my next version for sure.

Title: Re: World Viewer for DF 40.0X!
Post by: Beowulf on August 07, 2014, 02:51:44 pm
Awesome program, really like it a lot!

Am I the only one that had it crash while playing back the timeline (so a good deal into it), though?
Should I upload the necessary world files for that (created with 0.40.06; also I'm on mono)?

The bridges and stuff you showed here: http://www.bay12forums.com/smf/index.php?topic=128932.msg4426698#msg4426698
They do not require extra info from dfhack, or do they (think not, otherwise this "for DF 40.0X" thread would not be entirely appropriate for it atm, eh)?
Is that only available in your current build version?
Title: Re: World Viewer for DF 40.0X!
Post by: therahedwig on August 08, 2014, 09:18:33 am
Dumb question, but is it possible to figure out the Emblem of a given entity from the legends xml? I know that they have them ingame, but outside of Engravings and the Historical maps (and invader's armour)I can't seem to find them anywhere...
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on August 08, 2014, 06:00:16 pm
Dumb question, but is it possible to figure out the Emblem of a given entity from the legends xml? I know that they have them ingame, but outside of Engravings and the Historical maps (and invader's armour)I can't seem to find them anywhere...

Hmmm, not sure what you mean by emblem, where do you see it in-game?

Awesome program, really like it a lot!

Am I the only one that had it crash while playing back the timeline (so a good deal into it), though?
Should I upload the necessary world files for that (created with 0.40.06; also I'm on mono)?

The bridges and stuff you showed here: http://www.bay12forums.com/smf/index.php?topic=128932.msg4426698#msg4426698
They do not require extra info from dfhack, or do they (think not, otherwise this "for DF 40.0X" thread would not be entirely appropriate for it atm, eh)?
Is that only available in your current build version?

Please do upload the files it would help me find if it's a bug, I've never had the timeline crash but because of how many factors are involved I could have certainly overlooked something.

The bridges in the first post don't require the DFHack details, but they're formed only from connecting the starting and ending site, which isn't quite as accurate.

The more accurate roads in this picture (http://www.bay12forums.com/smf/index.php?topic=128932.msg5525622#msg5525622) require the detaisl from DFHack, and they're not in the build you have available, since DFHack doesn't work yet with the latest version
Title: Re: World Viewer for DF 40.0X!
Post by: therahedwig on August 10, 2014, 08:35:06 am
Dumb question, but is it possible to figure out the Emblem of a given entity from the legends xml? I know that they have them ingame, but outside of Engravings and the Historical maps (and invader's armour)I can't seem to find them anywhere...

Hmmm, not sure what you mean by emblem, where do you see it in-game?


These:

(http://img.ie/krmt0.png)
(Image spliced together from Historical Maps in legends mode, and two engravings from fort mode, which, of course, also show up in legends mode.)
As you can see, the symbol for Gizzard stones(which I did not know at all DF had, but okay) is the same circley symbol as used for ant-nests. It's used in the historical map to mark territory for that civ.

I was wondering if it's possible for world-viewer to give us the text-description of said symbol somehow, though I understand this may require looking through the existing art.
Title: Re: World Viewer for DF 40.0X!
Post by: Beowulf on August 10, 2014, 08:56:59 am
Please do upload the files it would help me find if it's a bug, I've never had the timeline crash but because of how many factors are involved I could have certainly overlooked something.

Hmm, it has done it on another world, too (I'll first try it on Windows to make sure it is not caused by using mono which I suspect it is; I'd have to see where to upload [possibly would end up being dffd] to anyway).

Thanks for the answers and once again for the sweet program.
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on August 11, 2014, 07:39:33 am
Dumb question, but is it possible to figure out the Emblem of a given entity from the legends xml? I know that they have them ingame, but outside of Engravings and the Historical maps (and invader's armour)I can't seem to find them anywhere...

Hmmm, not sure what you mean by emblem, where do you see it in-game?


These:

(http://img.ie/krmt0.png)
(Image spliced together from Historical Maps in legends mode, and two engravings from fort mode, which, of course, also show up in legends mode.)
As you can see, the symbol for Gizzard stones(which I did not know at all DF had, but okay) is the same circley symbol as used for ant-nests. It's used in the historical map to mark territory for that civ.

I was wondering if it's possible for world-viewer to give us the text-description of said symbol somehow, though I understand this may require looking through the existing art.

Got it.  It's not available with the normal export from legends mode. I'll take some time to see if I can include it as part of the extra DFHack output.  That will be available publicly once DFHack is updated to the latest version, and I can confirm my script works for it.  Thanks for clarifying. I don't use the historical maps from legends mode often, because they aren't included in the export, although the timeline feature was built to help with that.

That also reminded me that I'd like to be able to draw entity territories on the map, if that data exists, which it should.
Title: Re: World Viewer for DF 40.0X!
Post by: therahedwig on August 11, 2014, 07:57:51 am
Awesome! :P

Well, the historical maps do have a 'civ/site'-toggle, but no clue what it does, it doesn't seem to work for me...
Title: Re: World Viewer for DF 40.0X!
Post by: Lightningy on August 14, 2014, 03:48:05 pm
Mason11987 could you tell me what texture pack you're using
Title: Re: World Viewer for DF 40.0X!
Post by: 1v0ry_k1ng on August 15, 2014, 04:04:44 am
Would it be possible to upload this to a mirror site? I can't access dffd  :(
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on August 15, 2014, 05:19:14 pm
Updated!

First, for those who care, the source is now available in a proper repository on bitbucket (https://bitbucket.org/Mason11987/world-viewer).  I'm still getting familiar with how that all works, but this should be better, and will let people follow along with my development if they're inclined.  It's also a place to report issues in addition to this thread.  That also means the world viewer download is significantly smaller, so that's cool!

World Viewer 2.0  - Now including DFHack script for extra details! (http://dffd.wimbli.com/file.php?id=7851) - (If DFFD doens't work for you, you can use this dropbox link (https://www.dropbox.com/s/f5lw5gq437mrh2b/World%20Viewer%20df40.0X.zip))

New Screenshots!  Updated with all the new details (http://imgur.com/a/ISGwl#0)

Changes

*It's recommended to use DFHack 0.40.08-r2 (http://www.bay12forums.com/smf/index.php?topic=139553.0) or later.  It seems like r1 has some bugs.  It worked for me with r1, but r2 is apparently much more stable.
Title: Re: World Viewer for DF 40.0X!
Post by: Raul on August 15, 2014, 05:27:35 pm
Check out the screenshots below
Don't you mean above? :P
Title: Re: World Viewer for DF 40.0X!
Post by: Mason11987 on August 15, 2014, 05:30:56 pm
Check out the screenshots below
Don't you mean above? :P

ha, changed my mind on where the screenshots should go mid-post.  Thanks ;)
Title: Re: World Viewer for DF 40.0X! - Now with DFHack Integration!
Post by: Robsoie on August 18, 2014, 06:43:12 am
On 40.09 i got this after trying to load a world :
(http://i.imgur.com/1d75aS9.jpg)

of course the 5 files actually exist :
(http://i.imgur.com/sWfcDPks.jpg) (http://i.imgur.com/sWfcDPk.jpg)
On 40.08 i had no such problem, though obviously i was playing with a different world at the time

I then downloaded the new World Viewer 2.0 , but same message about the 5 files not being found.

Legends Viewer 1.13.19 (http://www.bay12forums.com/smf/index.php?topic=72702.msg5530851#msg5530851) can load the xml without problems so i'm not sure why World Viewer does not seem to do.

In case it has to do with something in one of those legends files, here are the 5 files :
http://dffd.wimbli.com/file.php?id=9429
Title: Re: World Viewer for DF 40.0X! - Now with DFHack Integration!
Post by: Mason11987 on August 18, 2014, 08:42:29 am
Yup, this is due to the file name changes in the most recent version.  I had intended to put out a release to address it before Toady implemented it but life got in the way.

You can wait for that update (should be today hopefully) or you can rename the files and it'll work, here's them renamed:

(http://i.imgur.com/TojmfJY.png?1)

Most are straightforward changes (remove the date) but the main map file is in a format:

world_map-regionname-year-seconds.bmp

where "regionname" in your case is region1, year is 44, and seconds i put as 1000 because it doesn't matter in the current WV version, but a number must be there for it to work.

If you export the other maps their format is going to be:

world_graphic-SYMBOL-regionname-year--seconds.bmp

Where SYMBOL is the map symbol for that map (dip, el, nob, str, trd, etc.) that symbol is included in the new DF version names so it should be simple to change them.  Note the two hyphens between year and seconds.

Legends viewer let's you pick the files one by one if they aren't found, World Viewer doesn't, so it's less resilient to changes like this (although the file name for these exports has never actually changed, so I never thought it would :P).

Thought I had another day, damn toady and his speed :).
Title: Re: World Viewer for DF 40.0X! - Now with DFHack Integration!
Post by: Rose on August 19, 2014, 03:08:54 am
Note: the double dash before seconds cannot be relied on. It only exists there if you export legends directly after generating a world, before using it.

The second dash is a minus sign, and doesn't appear once the world has been played.

I found this out when doing isoword.
Title: Re: World Viewer for DF 40.0X! - Now with DFHack Integration!
Post by: Lewa263 on August 19, 2014, 05:09:06 pm
Old age has caste listed, with that slight indent to show it is describing the old age itself. The caste appears to always be female.
Title: Re: World Viewer for DF 40.0X! - Now with DFHack Integration!
Post by: PeridexisErrant on August 19, 2014, 06:30:59 pm
I went through last night and updated my legends exports processor (https://github.com/PeridexisErrant/LegendsExportsProcessor).

The new files follow a more consistent pattern:  "regionX-YYYY-MM-DD-$item"

The year, month, and day fields are zero-padded, so it'll always be the same string length.  For site maps, the $item is "site_map-$number".  The region map tokens are mostly the same as before, though the combined biome+elevation map has been renamed from "world_graphic" to "detailed".  "world_map" (tileset map) remains the same.

The only exceptions to the pattern are "regionX-world_gen_param.txt", which does not change with date, and the various color key text ($thing_color_key.txt) files which are region-independent. 

Title: Re: World Viewer for DF 40.0X! - Now with DFHack Integration!
Post by: Mason11987 on August 20, 2014, 11:33:33 am
I went through last night and updated my legends exports processor (https://github.com/PeridexisErrant/LegendsExportsProcessor).

The new files follow a more consistent pattern:  "regionX-YYYY-MM-DD-$item"

The year, month, and day fields are zero-padded, so it'll always be the same string length.  For site maps, the $item is "site_map-$number".  The region map tokens are mostly the same as before, though the combined biome+elevation map has been renamed from "world_graphic" to "detailed".  "world_map" (tileset map) remains the same.

The only exceptions to the pattern are "regionX-world_gen_param.txt", which does not change with date, and the various color key text ($thing_color_key.txt) files which are region-independent.

Thanks for the details.  I'll make sure to do some detailed testing on this to check when work slows down.

@Lewa, I'll take a look at that, that's a strange bug.

@Japa, The code currently doesn't rely on the second dash, but I didn't remember what circumstance led me to not rely on it, so thanks for the heads up.  This should all be moot soon though.  But World Viewer will support both file formats for a bit at least.
Title: Re: World Viewer for DF 40.08! - Now with DFHack Integration!
Post by: Robsoie on August 20, 2014, 12:32:50 pm
Thanks for the explanation, i changed the file names according to your instructions and it worked.
The "panic time" after noticing the error message shows how much i became used to World Viewer for my legends exploration :D
Title: Re: World Viewer for DF 40.08! - Now with DFHack Integration!
Post by: Lewa263 on August 20, 2014, 01:54:45 pm
Oh, I realize I never specified where I saw old age. It's in the death event for historical figures who died of old age. You may have been able to figure that out, but reading my earlier post, it was pretty vague.
Title: Re: World Viewer for DF 40.0X! - Now with DFHack Integration!
Post by: PeridexisErrant on August 23, 2014, 08:39:55 pm
Yup, this is due to the file name changes in the most recent version.  I had intended to put out a release to address it before Toady implemented it but life got in the way.

You can wait for that update (should be today hopefully) or you can rename the files and it'll work
<bump>
*the update shudders and begins to move*
Any timeline on a fix?  I'd like to put WV back in the pack soon  :)
Title: Re: World Viewer for DF 40.08! - Now with DFHack Integration!
Post by: MrPalmer on September 10, 2014, 03:35:17 pm
I made this to copy the files to the old format, and WorldViewer (and old version satelite map maker too) worked fine for me. Your results may vary.

I've been fighting months with Carpal Tunnel Syndrome (surgery suggested) in both wrists, so the script below is sloppy and unpolished. It works for me. It is a Python  (http://www.python.org/)script to copy a new format region legends/maps to a new directory in old format. Yes, I know I could have made it shorter/efficient; I worked on it off and on (using copy/paste, replace, etc for repeating parts). Normally I would have made something more efficient, user friendly, error-proof, and documented. Im embarrassed to release it, but maybe someone can use it (or better, improve it).

Change the `mypath` variable below "### CHANGE THIS TO YOUR PATH ###" to be your path to your file. It SHOULD work on linux and mac, since I think I used cross platform functions. I release the hideous monster...

exports_new2old.py
Spoiler: exports_new2old.py (click to show/hide)
Title: Re: World Viewer for DF 40.08! - Now with DFHack Integration!
Post by: Saiko Kila on September 19, 2014, 01:55:02 am
I made this to copy the files to the old format, and WorldViewer (and old version satelite map maker too) worked fine for me. Your results may vary.
[...]

exports_new2old.py
Spoiler: exports_new2old.py (click to show/hide)

Thank you, that was helpful.
Title: Re: World Viewer for DF 40 - Now with DFHack Integration!
Post by: Mason11987 on December 18, 2014, 03:24:58 pm
ahhhhh, so it's been a pretty crazy fall for me!  Starting out the first day of vacation nicely though!

So World Viewer is now (months late) properly handling the files exported from DF as of the new version 2.0.1 (http://dffd.wimbli.com/file.php?id=7851)

There is a LITTLE bit more interesting in there though, I am now including the site maps that are exported easily via dfhack (with the provided exportlegendsWV script), as well as the four different map color legends, pic below.

Spoiler (click to show/hide)

I have a bit of a tweak for that coming, I'd like to be able to tell you what you're hovering over based on the color of the pixel.  But the UI logistics weren't working so this is a good start.

Please give it a shot and let me know what you think.


I've missed quite a few changes to DF in the last few months so it's possible the few world gens I tested didn't incorporate them, anyone with a well-developed adventurer world, one with a couple forts, or a long-running fort in the most recent version would be helpful to test.  Thanks!
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: Rose on December 19, 2014, 02:36:08 am
Wait, we can read that info?

Oh my, this changes everything.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: Scoops Novel on January 28, 2015, 06:02:43 am
What exactly do i need to plug into this? I've used the legends export processor in the starter pack but everything i've tried in user generated content says no basic map image. exportlegendsWV is unrecognised by dfhack as well.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: Mason11987 on January 30, 2015, 08:15:24 pm
What exactly do i need to plug into this? I've used the legends export processor in the starter pack but everything i've tried in user generated content says no basic map image. exportlegendsWV is unrecognised by dfhack as well.

The legends export processor is for legends viewer, it doesn't apply to World Viewer.

If you're using the starter pack you'll need to copy/paste the ExportLegendsWV.lua file into the dfhack folder:


So you can find it here: (Starter Pack Folder)\LNP\Utilities\World Viewer df40 (v2.01)\exportlegendsWV.lua
And you can put it in this folder: (Starter Pack Folder)\Dwarf Fortress 0.40.23\hack\scripts

That should work, let me know if you have any issues.

Alternatively, if you're not using the starter pack you can just go to legends mode and export all the files (press 'd', then 'x' to get the important ones).  Then you can navigate to the DF folder to select the files.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: lethosor on January 30, 2015, 09:08:03 pm
It would be nice to mention that this (or at least the download on DFFD) is Windows-only.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: Scoops Novel on January 31, 2015, 04:45:11 pm
TY Mason11987, I'll try it

YAY
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: PeridexisErrant on February 01, 2015, 07:36:50 pm
I'm just working on the long-overdue config for World Viewer in the starter pack, and have a couple of suggestions:  include a script to automatically launch the correct (32/64 bit) version, and some upgrades to exportlegendsWV.lua.

Spoiler: DFWV.bat (click to show/hide)

exportlegendsWV.lua has a couple of issues, and a few ways it could be upgraded.
I'm happy to attempt these fixes and upgrades, if you're OK with it, and get it into the standard DFHack library in one of the forms I listed.
Spoiler: some code changes (click to show/hide)
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: cephalo on February 03, 2015, 10:51:51 pm
I am having a crash during the world loading, and I have a strong deja vu like I've had this exact crash a long time ago last time I used WV. Maybe I did something wrong.

I used the exportlegendsWV script to export all that stuff with dfhack. But it's not loading in the 64 bit version. This is a new large world.

EDIT: Both 64 and 32 crash after the 'Historical Event Collections Processing' message.

Here is the save file that crashed WV. (http://dffd.bay12games.com/file.php?id=10543)

Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: Eldrick Tobin on February 05, 2015, 07:14:10 pm
Is there a more up to date exportlegendswv.lua file around that I've not seen the posting about? Searching on "assumed_identities" in this thread produced nothing. Yay for github.

I've fixed it myself, but I thought maybe I just missed something. *later... I need to his POST faster* Woo. Completed operations. Yay for corrections.


Items changed:
*PE's changes implemented
*As I had a full 'exportlegends all' I changed the dfhack_run line from "exportlegends info" to "exportlegends" so it wouldn't rehash old ground.
*assumed_identities->identities
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: Mason11987 on February 07, 2015, 05:48:02 pm
I'm just working on the long-overdue config for World Viewer in the starter pack, and have a couple of suggestions:  include a script to automatically launch the correct (32/64 bit) version, and some upgrades to exportlegendsWV.lua.

Spoiler: DFWV.bat (click to show/hide)

exportlegendsWV.lua has a couple of issues, and a few ways it could be upgraded.
  • xml filename should contain the date of the world as of export, since it contains info that changes over time
  • DFHack code should use spaces instead of tabs (style issue only)
  • This could be integrated with exportlegends itself and included in standard DFHack
  • Or even better, changed to run whenever the xml is exported in any way
I'm happy to attempt these fixes and upgrades, if you're OK with it, and get it into the standard DFHack library in one of the forms I listed.
Spoiler: some code changes (click to show/hide)

Thanks PeridexisErrant.  I'll include that .bat file in my next release.  I'm trying to sort out the site map stuff before I do so.

I saw you included it in exportlegends.lua.  I wouldn't mind if exportlegends.lua includes the modifications I made by default, but I really made them for World Viewer, so they may not be something everyone would want in that script.  For now I'll include an exportlegends.lua script in releases, which can replace the one that comes with dfHack, and I'll change the spaces/tabs as well.

I didn't include the date because I didn't need it and it would have been complicated to make sure it was right and there.



I am having a crash during the world loading, and I have a strong deja vu like I've had this exact crash a long time ago last time I used WV. Maybe I did something wrong.

I used the exportlegendsWV script to export all that stuff with dfhack. But it's not loading in the 64 bit version. This is a new large world.

EDIT: Both 64 and 32 crash after the 'Historical Event Collections Processing' message.

Here is the save file that crashed WV. (http://dffd.bay12games.com/file.php?id=10543)



Got it, it was due to a previously unseen situation where a Site conquering event collection could not have a defender civ listed.  That's weird.  Don't know if that's a df bug or intentional, ha.

I'll post an update when I have the fix out for that.



Is there a more up to date exportlegendswv.lua file around that I've not seen the posting about? Searching on "assumed_identities" in this thread produced nothing. Yay for github.

I've fixed it myself, but I thought maybe I just missed something. *later... I need to his POST faster* Woo. Completed operations. Yay for corrections.


Items changed:
*PE's changes implemented
*As I had a full 'exportlegends all' I changed the dfhack_run line from "exportlegends info" to "exportlegends" so it wouldn't rehash old ground.
*assumed_identities->identities

Could you clarify the assumed identities change you made?
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 12-18]
Post by: Mason11987 on February 10, 2015, 07:58:12 am
Just put out a new update.

A few changes

Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: PeridexisErrant on February 10, 2015, 06:16:51 pm
Awesome! 

About the massive upgrades to exportlegends - do you mind if I get it into the standard DFHack library?  (I'm assuming it's "finished" for now, and you don't mind me removing the print(date)) 

And I assume the filename is now stable, so I can build support for it into the PyLNP's legends export processor?
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: Mason11987 on February 11, 2015, 02:15:56 pm
Awesome! 

About the massive upgrades to exportlegends - do you mind if I get it into the standard DFHack library?  (I'm assuming it's "finished" for now, and you don't mind me removing the print(date)) 

And I assume the filename is now stable, so I can build support for it into the PyLNP's legends export processor?

I don't mind at all.  And thanks for that catch on print(date).  I was trying to deduce why your lua date code wasn't getting noticed by my code as the proper date and it drove me crazy for a while until I realized one was an en-dash and the other an em-dash or a hyphen, or whatever, ha.

I won't mess with the filename although I don't really know about the workings of the legends export processor.

I'm sure I'll add to the exporter in general.  It's really tuned to be exactly what I want out of it, as easily as I could get it.  But each release I find new things to add.  I have no objection with the current version being released with dfhack at all if it can be used by others, but I definitely foresee myself adding to it in the future.  For example, I haven't used DFHack at all to tackle any extra historical figure data that might be interesting, as an example "Night Creature" doesn't show up in DF Legends, the actual creature name does.  I'm using that now in the "Changed Creature Type" event, but I imagine I'll want to incorporate proper night creature names for all Historical Figures at some point, if it won't balloon the file size too much.  So add it to DFHack, but I'll include it in my releases as well and just make a note if I edited it in the release notes.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: PeridexisErrant on February 11, 2015, 05:16:32 pm
Cool.  I'll stick that in, and can update pull requests when something changes.  Just make sure that new WV releases can handle older versions of the legends_plus file!
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: Robsoie on February 28, 2015, 09:41:36 pm
Hello,
I've been exploring a few worlds with the current version and i've got a lot of fun.
I noticed a very minor problem, " humans " are called " humen " apparently in that latest version, i'm not an english speaker but i doubt " humen " is correct.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: Mason11987 on March 01, 2015, 08:57:04 pm
Hello,
I've been exploring a few worlds with the current version and i've got a lot of fun.
I noticed a very minor problem, " humans " are called " humen " apparently in that latest version, i'm not an english speaker but i doubt " humen " is correct.

yeah... I'm using a tool to automatically convert names from singular to plural and reverse and it's not perfect.  I'm working on an alternative way of handling that though with the dfhack export.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: Mason11987 on March 05, 2015, 04:07:47 pm
This brought me down the rabbit hole.

So I'm pulling the race names out through the raws directly now (through DFhack) so that humen/humans thing is resolved.  That led me to the problem of forgotten beasts, and I fixed an earlier assumption about race names, so now they're distinct.  I also wanted to be able to differentiate them so now every race (well every caste within each race) now has a description as well:

Spoiler (click to show/hide)

As you might notice this has the population wrong (there's gotta be one right?), so I went digging into the population info, and realized that the info from the world-sites file is lacking.  So I dug into some raw data about that.  Turns out population is stored based on region and race, so now regions have their population listed as well:

Spoiler (click to show/hide)

There is some weirdness around the "Unnumbered" things, but this seems pretty accurate.  Last things to do is rectify the castes from the original XML with my more detailed castes, and handle total populations and I'll do an update.  Good bug report!
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: Foffy123 on June 22, 2015, 02:35:59 pm
This brought me down the rabbit hole.

So I'm pulling the race names out through the raws directly now (through DFhack) so that humen/humans thing is resolved.  That led me to the problem of forgotten beasts, and I fixed an earlier assumption about race names, so now they're distinct.  I also wanted to be able to differentiate them so now every race (well every caste within each race) now has a description as well:

Spoiler (click to show/hide)

As you might notice this has the population wrong (there's gotta be one right?), so I went digging into the population info, and realized that the info from the world-sites file is lacking.  So I dug into some raw data about that.  Turns out population is stored based on region and race, so now regions have their population listed as well:

Spoiler (click to show/hide)

There is some weirdness around the "Unnumbered" things, but this seems pretty accurate.  Last things to do is rectify the castes from the original XML with my more detailed castes, and handle total populations and I'll do an update.  Good bug report!
Any progess updates? Looking forward to a new version.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 2-10]
Post by: Mason11987 on October 09, 2015, 10:06:58 am
New Version! 2.1! Download here (http://dffd.bay12games.com/download.php?id=7851&f=World+Viewer+df40.zip)

Most of this will only apply if you're utilizing DFHack to export your legends


There's also some prep work for Geo Biomes (layers of stone/veins/etc.) in there.  Unfortunately as far as I can tell there's no way to associate specific Geo Biomes with tiles on the map without embarking there.  I also have a "Historical Figure Travels" feature in the map mode, but it's still in an early stage as well.  Feel free to offer any ideas on either of those!

It also looks like there is a lot of opportunity with the "open-legends" dfhack script.

A few easily accessible details I can pull for future versions:

* Armies
* Units - Extensive details on all of your dwarves on the current embark
* Engraving details of your fort
* Vehicles
* Incidents
* Crimes and Punishments
* Adamantine tube locations
* All Announcements, reports
* Buildings built
* Constructions built
* Items in the fort
* Stockpiles
* Plants
* Squads
* All map blocks
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Tilogour on October 10, 2015, 03:54:40 am
It's not working for me. Probably because you are not supporting Perfect World utility.
Save (http://dffd.bay12games.com/file.php?id=11198)
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Mason11987 on October 10, 2015, 06:09:27 am
Actually it was a bug related to save files having a hyphen in them.  Ha.   Updated version here (http://dffd.bay12games.com/download.php?id=7851&f=World+Viewer+df40.zip)

Fixed and tested with your save, which uncovered a few long-standing bugs so I fixed those too.  Loads perfectly now, cool map!
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Tilogour on October 10, 2015, 07:26:32 am
Will you add kobolds in future relase? I don't see any kobolds in my civ list
BTW: I think it's a really cool utility : )
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Mason11987 on October 10, 2015, 08:07:18 am
There are kobolds in your civ list, but they aren't considered "full civs" as far as the tool goes.  That's because they don't have leadership or worship gods (at least without mods).

You can see all civs if you turn off the "IsFullCiv" filter by clicking "Filter" underneath the Civ list, click "isFull" in the box on the left, and then "Del" followed by "Apply".

Let me know if you have any other questions.

Edit - Made a few more bug fixes to the script to handle open-legends fine as well as regular legends mode.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Tilogour on October 12, 2015, 05:52:28 am
My suggestions:
1. Possibility to zoom in/out map.
2. Possibility to get more information about battles if someone want to click it on map (I mean coord).
3. Coord square showed differently on map. (If i want to see coord of historical figures and I click coord there I can't see where it is exactly. Of course I can read coords, but showing it on map somehow will be helpful) Edit: I see my curson IS on coord after I click that, but still showing this square different on the map could be useful.
I don't know if it's a big problem to do this or not.
4.If I have my curson near site and civ it should view more information about creatures in it (below type of site)

BTW: I found another bug (my previous posted save) - > If I turn on showing sites and civs on map and click coord 34.12 or 36.12 i have error "The given key was not present in the dictionary" (I have exported new legends, but it's from the same world, so previous save should also have this bug)
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Shonai_Dweller on October 20, 2015, 05:38:04 pm
Can I use World Viewer to check the historical population of a site? I've got a site with 19 Dwarves (the last of their civ) and I want to see if they're in decline or if this is an improvement over a hundred years ago.

Closest I can trace the history so far is through the 5 historical figures who settled here 150 years ago, but they all died over the following 5 years leaving a pretty big dark age.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: PeridexisErrant on November 27, 2015, 09:59:50 am
Two questions:
 - Is the included lua script fully replaced by this one? (https://github.com/DFHack/dfhack/blob/master/scripts/exportlegends.lua)  If so, can it be taken out, and any further work go on DFHack directly? (I'm happy to facilitate this)
 - Can we get separate files for 32 and 64 bit, so PyLNP doesn't get the filenames mixed up?
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Das123 on December 02, 2015, 02:02:00 am
I was looking at World Viewer to handle the legends in a YouTube series I'm starting soon but I keep running into issues with it. I'm using PeridexisErrant's Starter Pack 40.24_r20.

If I process the files using DFHack 'exportlegends' and go to the game root I can load in the files OK.

If I then run 'Process Legends Exports' from the Starter Pack and then open the zip file up nothing happens but an archive folder is generated with the files in the ZIP expanded. If I try to use the expanded files I get an error message saying 'Couldn't find basic map image.' I think the fact this is now a PNG rather than a BMP is the issue.

If I change the save folder name then it simply won't work no matter what I do. It seems to need the 'region[number]' format. I get errors such as 'Files cannot be found...' and then instructions to look at the txt file.

If I go back to the original folder settings, process the images, and then copy all the BMP files etc into another area it will open them up. If I replace the large BMPs with PNG files then it can no longer find the basic map image again.

I don't think the program knows how to play with the changes made by the starter pack yet it says it accepts both ZIP and PNG files. But when testing it only seems to work if it uses BMP files.

Thanks

Das
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Mason11987 on December 03, 2015, 11:56:28 pm
Lots of new events and event collections.  Interesting stuff.  Should be even more interesting when I can mess around with DFHack.  I have tomorrow off so I'll get an update that works with the regular xml export, and then look at the status of dfhack.

Can I use World Viewer to check the historical population of a site? I've got a site with 19 Dwarves (the last of their civ) and I want to see if they're in decline or if this is an improvement over a hundred years ago.

Closest I can trace the history so far is through the 5 historical figures who settled here 150 years ago, but they all died over the following 5 years leaving a pretty big dark age.

I'll think about it, but I don't think there is an easy way of doing that.

I was looking at World Viewer to handle the legends in a YouTube series I'm starting soon but I keep running into issues with it. I'm using PeridexisErrant's Starter Pack 40.24_r20.

If I process the files using DFHack 'exportlegends' and go to the game root I can load in the files OK.

If I then run 'Process Legends Exports' from the Starter Pack and then open the zip file up nothing happens but an archive folder is generated with the files in the ZIP expanded. If I try to use the expanded files I get an error message saying 'Couldn't find basic map image.' I think the fact this is now a PNG rather than a BMP is the issue.

If I change the save folder name then it simply won't work no matter what I do. It seems to need the 'region[number]' format. I get errors such as 'Files cannot be found...' and then instructions to look at the txt file.

If I go back to the original folder settings, process the images, and then copy all the BMP files etc into another area it will open them up. If I replace the large BMPs with PNG files then it can no longer find the basic map image again.

I don't think the program knows how to play with the changes made by the starter pack yet it says it accepts both ZIP and PNG files. But when testing it only seems to work if it uses BMP files.

Thanks

Das

So a few things.  As far as I know PeridexisErrant's pack doesn't have DFHack, if it doesn't have it I wouldn't try to force it. 

The process exports isn't used by WorldViewer.

I just started working on handling the 40_24 changes, I should have a good update tomorrow.  Is there a working DFHack with the newer version?



Two questions:
 - Is the included lua script fully replaced by this one? (https://github.com/DFHack/dfhack/blob/master/scripts/exportlegends.lua)  If so, can it be taken out, and any further work go on DFHack directly? (I'm happy to facilitate this)
 - Can we get separate files for 32 and 64 bit, so PyLNP doesn't get the filenames mixed up?

I'll have to check the lua script with DFHack, but I haven't changed it since my last update.  So if the one in the github isn't the one in the download than there must be a change to the github one which I haven't tested yet.

separate files for what?  the executable?
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: PeridexisErrant on December 04, 2015, 01:16:23 am
Lots of new events and event collections.  Interesting stuff.  Should be even more interesting when I can mess around with DFHack.  I have tomorrow off so I'll get an update that works with the regular xml export, and then look at the status of dfhack.

So a few things.  As far as I know PeridexisErrant's pack doesn't have DFHack, if it doesn't have it I wouldn't try to force it.  The process exports isn't used by WorldViewer.

I just started working on handling the 40_24 changes, I should have a good update tomorrow.  Is there a working DFHack with the newer version?
I'm looking forward to the update!

My pack usually has DFHack, but as it's not available for 0.42.xx (and probably won't be for a few weeks) the 'unstable' version doesn't have it yet.

It would be nice if World Viewer handled images in .png format and archives with any name.
@Das123; we do our best but not everything is compatible.  Try Legends Viewer; it's more liberal in what it accepts.


Two questions:
 - Is the included lua script fully replaced by this one? (https://github.com/DFHack/dfhack/blob/master/scripts/exportlegends.lua)  If so, can it be taken out, and any further work go on DFHack directly? (I'm happy to facilitate this)
 - Can we get separate files for 32 and 64 bit, so PyLNP doesn't get the filenames mixed up?

I'll have to check the lua script with DFHack, but I haven't changed it since my last update.  So if the one in the github isn't the one in the download than there must be a change to the github one which I haven't tested yet.

separate files for what?  the executable?
[/quote]
The lua scripts are very similar; the one I linked is the latest version of exportlegends.  It includes your extra XML with every export.  Stripping all the trailing and leading space and empty lines makes it pretty easy to tell.  If at all possible, it would be great to just use the version that comes with DFHack - easy to install and so on.

Currently there's one download, with subfolders for 32bit and 64bit.  It might be helpful to host these as standalone options, so users can only download the one they want.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Mason11987 on December 04, 2015, 10:02:17 am
I'm looking forward to the update!

My pack usually has DFHack, but as it's not available for 0.42.xx (and probably won't be for a few weeks) the 'unstable' version doesn't have it yet.

It would be nice if World Viewer handled images in .png format and archives with any name.
@Das123; we do our best but not everything is compatible.  Try Legends Viewer; it's more liberal in what it accepts.

The lua scripts are very similar; the one I linked is the latest version of exportlegends.  It includes your extra XML with every export.  Stripping all the trailing and leading space and empty lines makes it pretty easy to tell.  If at all possible, it would be great to just use the version that comes with DFHack - easy to install and so on.

Currently there's one download, with subfolders for 32bit and 64bit.  It might be helpful to host these as standalone options, so users can only download the one they want.

Yeah, that's fine and what I thought.  World Viewer works without DFHack (you just don't get all the extra details), when it comes out I'll test against it.

It looks like the code allows for PNGs, but maybe it's broken somehow.  Probably if the filename is off from what it expects.   I'll test it.  I'll allow archives of any name, that shouldn't be an issue.  Edit: Looks like both of these work.  Just re-saved my world-map as a .PNG and added it and the other files to a Test.7z and it worked fine.

Whenever I release it I'll include a .lua I know works, just in case the exportlegends is changed in the mean time.  But in general I expect the one with DFHack to work.  I'll modify my readme to refer to that.

I'll separate them out to two separate dffd posts.
Title: Re: World Viewer for DF 40! - Now with DFHack Integration! [Updated 10-9]
Post by: Das123 on December 04, 2015, 12:13:59 pm
My pack usually has DFHack, but as it's not available for 0.42.xx (and probably won't be for a few weeks) the 'unstable' version doesn't have it yet.

It would be nice if World Viewer handled images in .png format and archives with any name.
@Das123; we do our best but not everything is compatible.  Try Legends Viewer; it's more liberal in what it accepts.

Yes, relied on Legend Viewer for the start of this current LP but was running World Viewer in the background to cross reference info. :)
Recording this series with 40.24 version. First video in the series is here (https://www.youtube.com/watch?v=-XL0fESpGBY&feature=youtu.be) but World Viewer isn't shown in the intro.

Agree that PNG support would be good - and also not as stringent naming of the folders.
Title: Re: World Viewer for DF 42! - Now with DFHack Integration! [Updated 12-10]
Post by: Mason11987 on December 10, 2015, 08:50:55 pm
Upgraded to support df 42.xx!


Download on DFFD (http://www.bay12forums.com/smf/index.php?topic=128932.0) - 64 bit version here (http://dffd.bay12games.com/file.php?id=11454) (use the first link if you aren't sure which you need)  Source is on BitBucket (https://bitbucket.org/Mason11987/world-viewer).

Changes

Not anything world shattering here, but once dfhack is updated I'll update to support it and then put out my work on open-legends, assuming that still works.
Title: Re: World Viewer for DF 42! - Now with DFHack Integration! [Updated 12-10]
Post by: PeridexisErrant on December 10, 2015, 11:17:04 pm
Very, very nice work.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Shonai_Dweller on December 11, 2015, 06:58:54 pm
Having trouble seeing the whole window on my laptop monitor (1366x768) and as there's no scroll bars I'm losing the right side and the bottom.
Don't think the previous release was like this.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Mason11987 on December 11, 2015, 07:24:01 pm
Having trouble seeing the whole window on my laptop monitor (1366x768) and as there's no scroll bars I'm losing the right side and the bottom.
Don't think the previous release was like this.

Ah sorry, I expanded the default size, probably shouldn't have done that.

You can use Windows and the Up arrow to maximize the window, that should work.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Shonai_Dweller on December 11, 2015, 11:46:33 pm
Ah, thanks for the reply but that didn't work. I'm still missing the right side and bottom of the screen (even after removing windows toolbar).
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: thedonkified on December 13, 2015, 06:18:29 pm
I have a question. What does it mean when the population of a race is unnumbered?
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Mason11987 on December 13, 2015, 10:57:14 pm
I have a question. What does it mean when the population of a race is unnumbered?

It means that the game reports them as being "unnumbered".  Essentially this means you could never kill all of them, and they aren't individually tracked or counted.  This applies to things like ants and bugs, and   demons and so on.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: thedonkified on December 14, 2015, 08:22:46 am
Oh okay. Thanks!
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: TheFlame52 on December 16, 2015, 11:57:51 am
Does this version work for 0.34.11 saves by any chance? I have something I want to check out.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: PeridexisErrant on December 16, 2015, 05:54:07 pm
You'll need to export the XML and some maps from legends mode, but it should be compatible with them.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: TheFlame52 on December 16, 2015, 06:59:59 pm
I know how to use it, I just wanted to see if I could find out how many descendants Yonali had.

EDIT: The download link is broken. It just opens the forum topic in another tab.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Mason11987 on December 17, 2015, 02:12:08 pm
I know how to use it, I just wanted to see if I could find out how many descendants Yonali had.

EDIT: The download link is broken. It just opens the forum topic in another tab.

Sorry, fixed:

http://dffd.bay12games.com/file.php?id=7851

I don't know if it works with .34.  I've changed so much since then.  If it doesn't work for you post the save here and I'll check it out.

I know when I started working on this you couldn't get as much info, and the parent/child relationship was one of them, so it may work but nothing might show up there.  That's just a shortcoming of early version sof XML.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: TheFlame52 on December 17, 2015, 02:24:20 pm
Well it doesn't matter anyway, because I seem to have lost Yonali's world!
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: KillzEmAllGod on December 17, 2015, 09:54:55 pm
Nice that you have kept going with it.
Some suggestions for a few things.


Anyway to get musical forms and other things out of the xml?
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Mason11987 on December 17, 2015, 11:44:45 pm
Nice that you have kept going with it.
Some suggestions for a few things.

  • Gods should be sorted by Civs like Races for Civs. Also needs their gender and race.
  • Civ screen would be nice to know if they're dead.
  • Leaders needs something to sort it out as well instead of alphabetical. Might do that for Civs the ranks as well as descending in order from the first one to the last maybe even put their reign in there. Either that or a more complex view.
  • Sites, good to group them by type.
  • Race should be sorted into groups, Civilization Dwarf, elf, human etc. Then animal people might also do it so they're sorted into playable and unplayable for adventure.
  • Entity might also be good to sort them by their types. There's also an error when selecting religion groups

Anyway to get musical forms and other things out of the xml?

Thanks!


Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: KillzEmAllGod on December 18, 2015, 04:35:07 am
I think animal people just need to be apart of a Civ's population. Would also be helpful to display a Civ's total population of sentient members, could just do the number and then % if space is an issue. Not that clear when a different entity type starts. The filter is also a bit confusing, pretty hard to tell what it does or how to use it correctly. Also might be good to have something to explain things like why some HF names are red and others blue, that appears to be gender but its rather confusing mainly due to going through links and not going back to the first one.

Its great to have all this information its just sometimes hard to tell what's what in some of the areas. 
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Mason11987 on December 18, 2015, 11:01:45 am
I think animal people just need to be apart of a Civ's population. Would also be helpful to display a Civ's total population of sentient members, could just do the number and then % if space is an issue. Not that clear when a different entity type starts. The filter is also a bit confusing, pretty hard to tell what it does or how to use it correctly. Also might be good to have something to explain things like why some HF names are red and others blue, that appears to be gender but its rather confusing mainly due to going through links and not going back to the first one.

Its great to have all this information its just sometimes hard to tell what's what in some of the areas.

I'm not sure I follow by "I think animal people just need to be apart of a Civ's population."  Animal Men are listed individually as separate civilizations in the world_history export, that's why they get their own listing in the Civ Tab.

So how would you determine total population of sentient members?  I could start getting the total population by examining the population of all the sites that are part of the civ.  But the problem with sentient is that I'd most likely have to make some assumptions of what race is sentient or not, maybe not if I can explore DFHack further.  For example, I don't have any place in the code where dwarves/elves/humans/goblings/kobolds are hardcoded in as unique, because a mod can easily alter that and it's intended to adapt to that.  It's a tricky problem and I'm open to suggestions.  A "Total population" count on the civ screen is absolutely do-able though.


Regarding entity types, do you have your filter set up like this?

(http://i.imgur.com/UcocADy.png)

If so that puts a red word dividing items in the list like this:

Spoiler (click to show/hide)

I agree the filter screen is tricky.  I will add some explanatory text and tool tips to the next version.

Blue and Red are gender yes.  I'm not sure how to explain that in the app in an effective way.  If you follow a blue HF link though it'll show Male, and same with red and female.  If there are places (other than the main HF listing) that don't have blue/red I could add them to keep consistency.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: KillzEmAllGod on December 21, 2015, 10:00:14 pm
That filter screen isn't friendly and using the greater then or = to doesn't help either, its powerful though if it was easier to use and had more options.
The feature to save filter settings would also be handy and to just click on them to change it to a different one would be good.
Could do it so there's a bigger filter (easier and more clear wording and context) and a compact one (Pretty much as it is).

The picture does seem like it should be a default setting for that screen.
The red arrows at the top might want to be more clear that they're forward and back.
Might just want to put a Question mark in that can be clicked to explain things or have so when we hover over thing it brings up a info box to explain it.

For Races it might just be good so there's the option to group them by Civilization, then man then creatures, mega beasts as well? Looks to be the case for dwarf, elf, goblin and humans already.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: PeridexisErrant on January 25, 2016, 11:16:22 pm
PSA:  exportlegends.lua is not only available again in DFHack, but has several contributors adding further information to the legends_plus xml export.

@Mason 11987 - I strongly recommend ensuring that the DFHack version has all your changes, then ceasing to distribute it to avoid users downgrading their installed copy.  Further development can happen in the DFHack repo!
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Mason11987 on February 06, 2016, 12:42:31 pm
PSA:  exportlegends.lua is not only available again in DFHack, but has several contributors adding further information to the legends_plus xml export.

@Mason 11987 - I strongly recommend ensuring that the DFHack version has all your changes, then ceasing to distribute it to avoid users downgrading their installed copy.  Further development can happen in the DFHack repo!

Thanks,  I had a nearly done version but I was waiting on open-legends to work and then life got in the way.  I'm currently merging my changes in with the dfhack version.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: KillzEmAllGod on February 27, 2016, 12:29:05 am
Anyone to see if the civs has followers of megabeasts?
I would Also request that it check the followers if they're alive or dead.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: TheFlame52 on February 27, 2016, 07:17:50 pm
Being able to see the followers of gods/others would be cool.
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: wuphonsreach on March 11, 2016, 07:04:56 am
dfHack for 42.06 is on alpha-2, so maybe legends export now works again?
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: lethosor on March 13, 2016, 12:33:33 pm
Was it not working with DFHack at some point?
Title: Re: World Viewer for DF 42! [Updated 12-10]
Post by: Mason11987 on June 18, 2016, 06:53:09 pm
Release 2.2.0 is now out:

Download (Windows): on DFFD (http://dffd.bay12games.com/file.php?id=7851) - 64 bit version here (http://dffd.bay12games.com/file.php?id=11454) (use the first link if you aren't sure which you need)  Source is on BitBucket (https://bitbucket.org/Mason11987/world-viewer).

To install, ensure you copy the exportlegends script to the DFHack/scripts folder, and overwrite the one that's located there.


Some pictures (http://imgur.com/a/Z1rMZ)

This has some pretty massive changes in it.  Hopefully it brings some more interesting things out to investigate, especially on long-running forts.
Title: Re: World Viewer for DF 43! [Updated 06-18]
Post by: KillzEmAllGod on April 05, 2017, 10:10:15 pm
Still around Mason?
Title: Re: World Viewer for DF 43! [Updated 06-18]
Post by: Mason11987 on April 30, 2017, 09:25:16 pm
Still around Mason?

Yup! checking in occasionally waiting for a new release.
Title: Re: World Viewer for DF 43! [Updated 06-18]
Post by: TheFlame52 on May 01, 2017, 08:40:09 am
I'd just like to say that this is a very useful utility. It complements Legends Viewer well. I especially like it because it lets me see how many worshipers gods have and how many descendants a historical figure has.
Title: Re: World Viewer for DF 43! [Updated 06-18]
Post by: Vaftrudner on July 05, 2017, 02:53:56 pm
I just wanted to say thank you for this! I just came back to DF after a long time away and wanted to start back up again with something special - your tool helped me find the perfect civ to start in, a decimated dwarven civilization with a grand total of 62 dwarves, at war with an elven Civ of some 5000 pointy eared bastards who have ravaged our civilization in a war currently ongoing for 8 years. We, the last survivors, shall build a fort in the middle of the ocean and drown them! Also the main deity is a bobcat.
Title: Re: World Viewer for DF 44! [Updated 11-25-17]
Post by: Mason11987 on November 25, 2017, 03:31:56 pm
Hi all!

Just released version 2.4.2 for df 44.  Download (http://dffd.bay12games.com/file.php?id=11454) (32-bit (http://dffd.bay12games.com/file.php?id=7851))

This mostly incorporated parsing for all the new items:


It should load whatever world now, but there's a bit left to do to flesh out the new stuff:


I've ran this through about 20 different size/length world, but if anyone want to throw a world or two add it to make sure it handles it well that'd be helpful.  Thanks!

Edit: Version 2.4.2
Title: Re: World Viewer for DF 44! [Updated 11-25-17]
Post by: TheFlame52 on November 26, 2017, 05:44:20 pm
Thank you!
Title: Re: World Viewer for DF 44! [Updated 11-25-17]
Post by: Mason11987 on November 28, 2017, 09:51:50 pm
I'm working on something to try to write a little bit of a story out of a civilization's history, like I have for viewing historical figures (https://i.imgur.com/ubEaDr5.png?1).  Lists of data and stats are good, but sometimes words are nice.

This is an example of something I'm automatically generating now when you view a civ.  Considering the entire swath of data available for me about civs, all their wars/battles/leaders/members/sites/events, let me know what you think would be a cool addition to this:

(https://i.imgur.com/I36iyyI.png?1)

Title: Re: World Viewer for DF 44! [Updated 11-25-17]
Post by: Mason11987 on December 16, 2017, 10:08:32 pm
Hi all!

Just released version 2.5.0 for df 44.  Download (http://dffd.bay12games.com/file.php?id=11454) (32-bit (http://dffd.bay12games.com/file.php?id=7851))

This version reincorporates support for additional data from DFHack.  Included is a replacement exportlegends.lua file for the one that comes with DFHack.

Gives details about:

Title: Re: World Viewer for DF 44! [Updated 11-25-17]
Post by: lethosor on December 16, 2017, 10:39:39 pm
This version reincorporates support for additional data from DFHack.  Included is a replacement exportlegends.lua file for the one that comes with DFHack.
What does it change? Is it something that should be incorporated into the DFHack version?
Title: Re: World Viewer for DF 44! [Updated 12-16-17]
Post by: Karater on December 17, 2017, 06:14:09 am
just hat a look at your changes to exportlegends.lua. can you check export_sites() it seems to write the closing tag for underground_regions.
the populations you added for regions and underground_regions are in some kind of list. Wouldn't it be better to export them as separate xml elements "<population><race></race><count></count><type></type></population>?
Title: Re: World Viewer for DF 44! [Updated 11-25-17]
Post by: Kromtec on December 17, 2017, 10:37:33 am
What does it change? Is it something that should be incorporated into the DFHack version?

WrittenContent and the three ArtForms are now in the vanilla legends.xml since 0.44.01+. But they are at the end of the xml, instead of just before the HistoricalEvents as in the legends_plus.xml.
I already opened a pull request (https://github.com/DFHack/scripts/pull/36) a week ago to bring them in the same order as in the vanilla legends.xml.
It would really help to process the two xml files parallel when they have the same order.

Additionally it would be really awesome if you could find a way to get the used and current identities of historical figures into legends_plus.xml.
There are <current_identity_id> and <used_identity_id> nodes, but this ids do not point to the strings that are used in legends mode. This strings are nowhere in the legends.xml file.

Title: Re: World Viewer for DF 44! [Updated 12-16-17]
Post by: Mason11987 on December 18, 2017, 07:23:38 pm
Thanks guys for the input!

@Karater

You're right on the underground regions tag, thanks for the catch.  my processing ignores extraneous close tags so I missed it.

I used a list format in a couple places, like river coordinates, historical figures in an entity, and a few other places.  I used them in a few cases to try to reduce the excessive size of the file that gets generated, for race populations it's probably not as big of an impact as a few other spots.  I'm fine going another route though.

@lothosor,

Honestly, I think the changes I made should be incorporated there, but I didn't want to do it myself right away, especially since I had changed so much in terms of structure to make it easier to manage, and removing/adding/changing tags.  Also, I couldn't easily pull down the dfhack code to make a pull request, I'm probably doing it wrong though.

@Kromtec,

I'll change the ordering in mine, and I'll look into the identities, I think I saw something on that while trawling through the new stuff in this version.

Title: Re: World Viewer for DF 44! [Updated 11-25-17]
Post by: Heretic on October 18, 2019, 02:33:52 am
Hi all!

Just released version 2.5.0 for df 44.  Download (http://dffd.bay12games.com/file.php?id=11454) (32-bit (http://dffd.bay12games.com/file.php?id=7851))

This version reincorporates support for additional data from DFHack.  Included is a replacement exportlegends.lua file for the one that comes with DFHack.

Gives details about:

  • Buildings
  • Site Buildings
I'm heavily interested in getting this info in separate XML for auto generating signatures for map based on isoworld export. Can somebody help me?