Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[LeoCean]

Are you still getting the graphical FPS went way down with the new tileset problem you had before after you started using 5.47 twbt?

[TheBloke]

Leo: Yes, I see the same lower GFPS / higher CPU with TWBT 5.47 as I did with 5.45.   

More importantly, I can now also easily repeat the crash.  It is usually happening within 5 minutes of loading my current save, with both TWBT 5.45 and 5.47.

It also happens both with and without "twbt filesize 18 18".

I now have Visual Studio 2010 installed so I was able to capture some more details.  I will post them in this thread as it's the new Spacefox that is triggering the crash.  I can also re-post to the TWBT thread, if mifki wants me to?

I can trigger the crash quickly (within a minute or so), simply by rapid scrolling - either horizontally or up/down Z levels but horizontal triggers it quicker.   If I don't do rapid scrolling it will still eventually crash, but takes longer (but never more than 10 minutes.)   However only certain Z levels will trigger a crash - ones showing only rough stone and stairways definitely don't crash, and I've also not yet had a crash in an outside area.

It appears to only crash when unpaused, never when paused.  However screen activity while paused ramps up the CPU, and once it ramps up it stays ramped up - and will then crash very quickly after the game is unpaused.

I therefore believe it's probably some kind of resource leak, and that this appears to be related to particular objects/tiles.

At the end of this post I have given the details I have of the two types of crash I get.  Whenever it crashes, it is always one or other of those two types.

Here is an example of how I can trigger the crash, and showing the variations on its effects - paused v unpaused, effect of scrolling on CPU usage, signs of resource leaking, etc.

• I load DF and load my save.  In this example I have "twbt filesize 18 18" enabled, so I have a smaller viewport than normal.
• I am viewing an area at  Z level -4: a few workshops, several stockpiles, lots of dwarves, lots of objects
• GFPS shows at 40 (my configured GFPS cap), and overall DF CPU usage is 9.5%
• I also check Z level -3: even more stuff than -4, but CPU is lower (7.5%), GFPS still 40;  I also check Z level -5, a nearly empty level (only rough stone, stairs and four wall sections): GFPS still 40, CPU down to 5%
• (Still paused) On level -4: I wait a while, watching, confirming GFPS and CPU usage are steady - CPU never goes above 9.5%
• (Still paused) I scroll around madly for a minute or so; GFPS drops as low as 7, CPU goes up to around 15% (meaning the graphics thread is at near 100% of one core)
• (Still paused) I stop scrolling; CPU stays at 15%, it does not drop back to the original 9.5%.  GFPS goes up to only 17, not the original 40
• (Still paused) I go down one Z level to -5, the nearly empty level.  CPU drops to 5% (same as it was before the rapid scrolling), GFPS back to cap of 40
• (Still paused) I go back up to original Z level -4; CPU jumps up to 15% again, GFPS down to 17 again
• (Still paused) I go up another Z level to -3, a very busy level: CPU still at 15%, GFPS still at 17
• I unpause and:

• If I unpause while looking at Z levels -4 or -3, the overall CPU will jump to as high as 22% (my normal DF + TWBT level is 14-15%) and will then always crash within 5 - 30 seconds (tested 5 times)
• If I unpause while looking at Z level -5, the CPU goes only to my normal unpaused level (14-15%) and the game will not crash

From this I conclude:

• The final trigger for the crash is something that only happens while unpaused
• But the crash is set up by any graphical updating, including while paused
• A lot of rapid screen activity causes CPU to rise and then not go down, and puts TWBT in a state where it will crash soon; some kind of resource leak?
• But this high CPU, and the final crash, seems to be related to particular assets/objects being displayed at the time
• Because rapid screen activity caused high CPU to continue even after activity stopped, but only viewing certain Z levels: levels -4 and -3 in this example.  The "quiet" Z level -5, which only has rough stone and some stairways, had normal CPU before and after rapid scrolling, and then did not crash when I unpaused.
• But both Z level -4 and -3 continued with high CPU after rapid scrolling stopped, and then crashed if I was viewing these levels after unpause
• Both -4 and -3 have workshops, stockpiles, dwarves, and lots of other objects that level -5 did not.
• So maybe there is a buffer/cache associated with certain object/building (types)?  And this buffer is getting filled every time I scroll, such that the CPU rises and the game crashes - but only if/when those particular objects are visible on screen?

Screenshots of the three fort areas described above
Click images to see in original resolution

Spoiler: Z Level -4, the "original" area: 10% CPU to start with (highest of all), and the area in which I did the mad scrolling (click to show/hide)

Spoiler: Z Level -5, the "empty" area: 5% CPU to start (lowest of all), and remained low CPU/high GFPS after the rapid scrolling; does not crash (click to show/hide)

Spoiler: Z Level -3, the other "busy" area: 7.5% CPU to start, lower than Level -3 despite having more "stuff", but also high (15%) CPU after scrolling stopped, and crashes (click to show/hide)

I hope that all makes sense and it's maybe some help in pinpointing the issue, if it's not obvious from the crash details.

I don't yet have the TWBT code downloaded/compiled so I can't debug inside TWBT.  I will be setting that up shortly and will then try to get more detail on where the crash is happening.  I can also upload my save game + settings if that's needed.  But hopefully mifki can find the issue before then

Crash type 1: "A heap has been corrupted"

Spoiler: Screenshot with stack trace (click to show/hide)

Spoiler: Stack trace text and autos (click to show/hide)

[/list]

Code: [Select]

ntdll.dll!_RtlReportCriticalFailure@8()  + 0x83 bytes
  ntdll.dll!_RtlpHeapHandleError@4()  + 0x1c bytes
  ntdll.dll!_RtlpLogHeapFailure@24()  + 0x9f bytes
  ntdll.dll!RtlpFreeHeap()  + 0x4e07c bytes
  ntdll.dll!_RtlpFreeHeapInternal@12()  + 0x27e bytes
  ntdll.dll!_RtlFreeHeap@12()  + 0x2c bytes
> msvcr100.dll!free(void * pBlock=0x35ccb008)  Line 51
  twbt.plug.dll!57a91f21()
  [Frames below may be incorrect and/or missing, no symbols loaded for twbt.plug.dll]
  twbt.plug.dll!57a92308()
  twbt.plug.dll!57a93d68()
  twbt.plug.dll!57a92a03()
  gdi32.dll!_GetAPI@12()  + 0x28 bytes
  KernelBase.dll!_LoadLibraryExW@12()  + 0x149 bytes
  KernelBase.dll!_WaitForSingleObjectEx@12()  + 0xb6 bytes
  ntdll.dll!_ZwReleaseSemaphore@12()  + 0xc bytes
  SDLreal.dll!6813e46a()
  Dwarf Fortress.exe!002c721d()
  Dwarf Fortress.exe!002c4f70()
  Dwarf Fortress.exe!002c3689()
  Dwarf Fortress.exe!002c5580()
  Dwarf Fortress.exe!002c5a05()
  Dwarf Fortress.exe!002c32de()
  Dwarf Fortress.exe!002c37b2()
  kernel32.dll!@BaseThreadInitThunk@12()  + 0x24 bytes
  ntdll.dll!__RtlUserThreadStart()  + 0x2f bytes
  ntdll.dll!__RtlUserThreadStart@8()  + 0x1b bytes


_crtheap 0x03970000 void *
pBlock 0x35ccb008 void *


Crash type 2: "Access violation reading location.."

Spoiler: Screenshot with stack trace (click to show/hide)

Spoiler: Stack trace text (click to show/hide)

Code: [Select]

> twbt.plug.dll!57a92081()
  [Frames below may be incorrect and/or missing, no symbols loaded for twbt.plug.dll]
  twbt.plug.dll!57a92308()
  twbt.plug.dll!57a93d68()
  twbt.plug.dll!57a92a03()
  gdi32.dll!_GetAPI@12()  + 0x28 bytes
  KernelBase.dll!_LoadLibraryExW@12()  + 0x149 bytes
  KernelBase.dll!_WaitForSingleObjectEx@12()  + 0xb6 bytes
  KernelBase.dll!_ReleaseSemaphore@12()  + 0x14 bytes


[LeoCean]

Can you post your save? I'd like to see if it crashes for me also, it should your computer is way better. But I also have crashdumps enabled so mifiki could take a look at the crashdump maybe..

Just from your screenshots I can see that, that fort is going to be laggy anyways, you have way too much open space. It seems you dig out mutliple layers and you also have most of your stuff on one layer more or less, but that's not a big deal lag wise. But all that open space is what dwarves will constantly use up memory with as they calculate their pathing, so honestly its better to do mutli layer bases and not have such huge open places and if you do mine out a area underground close it off so that your dwarves won't use memory/ resources on those areas (walls and potentially locked doors). Also you use such large stone stockpiles, I'd learn quantum stockpiling but everyone has an opinion on that. I'm not saying you have to only use quantum stockpiles because that is inefficient I use a feeder and a fed on my quantum stockpiles.

It's possible it's just the design of your fort but it may not be. It could be twbt it causing the end prematurely.

[TheBloke]

OK, this Google Drive folder contains a RAR of the save, and all my DFHack init files.  I've already confirmed the save loads Ok with DFHack disabled, so it should be fine to use without all the same DFHack init files/same plugins enabled, but I include them for reference and in case my options are contributing to the crash at all.  Although 99% of the init files are identical to PyLNP's default config: I've only changed a couple of mousequery options and added a couple of new keybindings.

Google Drive's UI is a bit annoying.   When you click TWBT-Crash-Save, it will open it like a folder, not download it.  Once you've clicked it, a download icon appears in the black bar at the top - although sometimes this bar disappears, so you might need to move your mouse cursor to the top of the window to see it!  Dumb.

The init files are loose in the folder and also RAR'd in the file DFHack-Init-Files.rar for easy downloading in one go.

[TheBloke]

Regarding lag:  I am not sure how much the open areas are contributing to CPU FPS, because I have experimented with blocking off large areas of open space with new walls/doors and also using traffic rules to do the same, and it made almost no difference.   I have spent many hours working on FPS, and have found a number of things that help quite a bit (having fewer animals; setting all Traffic designations to a weight of 1, ie disaabling traffic;  not opening certain Cavern areas; killing all running water) but open spaces has never been one of them.  Maybe 1-2 FPS reduced, but I could not really notice it in my testing.

However, regardless of that, that is only CPU FPS.  It definitely should not contribute to GFPS.  When I use the original Spacefox (or any other tileset), my CPU FPS and GPU FPS are always the same - ie CPU FPS is the limiting factor.  And most importantly, the graphical calculations use almost no CPU (maybe 1.5% of my total, ie total DF CPU % = 14%, with 12.5% being CPU calcs and 1.5% being the graphical thread.)   This has been very consistent over 100+ hours I have played on this same for with Spacefox. 

With the new Spacefox, the GFPS is often be a lot lower than my CPU FPS, and also graphical CPU % goes much higher.  And there is a direct correlation between sudden high CPU usage and imminent crash.

My CPU FPS is certainly not great - usually a max of 30 - but I have been playing this fortress for 100+ hours and it very very rarely crashes (four times in 80-100 hours).  With the new Spacefox, it now crashes repeatably within minutes.  So it is not the open space that is causing TWBT to crash in general

I'm pretty sure there's some kind of resource allocation bug in TWBT, perhaps triggered by the greater number of overrides or perhaps by some specific combination of overrides.  And it's definitely related to what objects/tiles are being shown on screen at a time.  Maybe it even requires a specific combination of objects to trigger, which would explain why I see it and you haven't yet.

[TheBloke]

I just realised I should probably include my DF init files as well, I forgot they are not included in the save. 

I added a new RAR file to the shared folder, init.rar - this is all my DF init files, including colors.txt and overrides.txt from SpaceFox Overrides.

[TheBloke]

One more thing on lag: I am sure you are right that in general open spaces doesn't help.  But all my bedrooms, dining rooms and meeting areas are on Z level -3, and I have basically nothing below Z level -4 except stones (with no empty stockpiles to store them in at the moment, except for Gold and Platinum.) 

So the dwarves never path down there unless they have a job, and this is only a small minority.

Even then, I don't see significant FPS drops when, for example, I open a new stockpile and the dwarves start picking up large numbers of stones from lower Z levels.

Admittedly I haven't tried closing off the large open spaces in the last few game years.  I tried it about half way through the game, using Traffic Designations to set all lower levels to Restricted.  I also used Traffic on my busy levels to define High Priority corridors, and to set as low priority dead ends, empty rooms, etc.  All stuff that the Wiki recommended.  None of it seemed to help much at all. 

Then later I realised I got a much bigger FPS boost by simply turning off Traffic completely (setting all values to 1).  That increased my FPS by about 25%, so right now Traffic is basically disabled.  But since I did that I haven't haven't then tried blocking off the lower levels.

You are right that it is something I should review again.  But I think the most important factor is that I'm not yet really using most of the open levels - my 250 dwarves are 90% on levels Z-2, Z-3 and Z-4, and so 99% of pathing only has to consider those spaces.  It's only a few dwarves that go higher or lower - 15 x miners go below Z-4, and I have a few butchers, tanners, animal trainers, and hunters who go on levels Z-1, Z0 and Z+1. 

[LeoCean]

Well that was kinda useless for me to test, there's many things that could cause that large of lag in that fortress and turning off twbt made no difference for me. In 40.24 if trees grow up in the areas you dig out after the cavern areas spread to your fortress it's instant crash, when they become full size in those small areas. It'd have been nice for toady to just push out a 40.25 build with cutting of tree saplings. Even if saplings grow into floors or walls it crashes. Hmm I wonder if that's moddable. Probably is. You don't have that much running water so I can't see that effecting it much, I only play with 150ish dwarves so can't really say much about 250 but less is better.

I don't quite know how dwarve pathing works but I'm pretty sure I read they calculate everywhere they can possibly go, how often they do that I have no idea. It's essentially why 100-180ish is usually the cap most people probably play on. But I'm getting 7(7) fps on your map so I'm not surprised that yours can handle more. I can't test if it's related to twbt though, 7fps is sloowwww.. I couldn't cause it to crash on -3 or whereever the workshops are but I had like 5(5).. Same with printmode standard so idk. That said I only used your save file so maybe it has to do with the other files but with only 5(5) it's out of my hands. It's kindof why I play with a modest modish/accelerated build of spacefox, since the game has way to many materials for me without simplifying it (as I don't care about different types of wood and what not with the same properties).

[TheBloke]

OK thanks for trying.

If you can only get 7 FPS then I am not that surprised you couldn't get it to crash.  From what I've seen it seems that the crash gets built up to over a little time, resulting from a build up of screen updates/activities that eventually push it over the edge.  And it only happens when the game is unpaused, so it's definitely triggered by something that happens when CPU calculation is going on.

Putting that together makes me think that it would need a higher amount of CPU FPS to trigger it - at 5 FPS and 5 GFPS, you're probably not getting enough graphical updates in a short enough amount of time to overload it in the right way.  And/or, there's not enough CPU calculating going on while unpaused to trigger the final crash.

Today/tomorrow I will get TWBT compiled locally and then I will be able to debug inside twbt.plug.dll and see exactly what it's doing when it's crashing.  Then hopefully I can submit a more precise bug report for mifki.

As an aside, I thought I had it bad only getting 20-30 FPS on this current game   I can see why you wouldn't want large open areas if you can only get 5-7 from this save.  That sucks man.  Although you didn't copy my init files - I get 20-30 FPS with Weather and Temperature disabled, and you probably have them turned on.  Although I haven't actually measured how much FPS difference turning those off makes. 

Also, my FPS varies a lot according to what I'm looking at.  If I look at outside (Z0 and Z+1) or some of my busy levels (Z-2, -3, -4), I only get around 20-22 FPS.  If I'm looking at levels with fewer buildings/dwarves, then it goes up to 30.  I don't quite get why looking at different levels affects the CPU FPS, I would have thought it'd only affect GFPS.  But I guess the game does more CPU calcs when you can actually see the dwarves moving about.  Maybe when they're out of sight, it uses a quicker/simpler pathing algorithm.

I really feel it's about time Toady made some effort at doing multi-threading so the game can run a lot faster on dual/quad core systems.  It's an astonishing, amazing game, which makes it all the more sad that so many users can't enjoy it to the full.  It makes me sad that I have to turn off key games features like weather and temperature to get a halfway playable game.  And even with those turned off, I still have to limit myself.  For example, in this current save if I dig into the caverns at a certain place, FPS drops in half, to about 12-14.  So I can't even explore the whole caverns without making the game crawl.  I can't enjoy playing when it's consistently less than 20 FPS, it's just too slow and laggy.

It's still an amazing game but it's very sad it has to be crippled in this way.  I'd love to get a more modern PC but I can't just drop the  it would take to get me a suitable upgrade from this one.   My current one was very high end when I bought it, but that was more than 5 years ago.  I have managed to keep it relevant by overclocking it heavily (from 2.8 to 4.2ghz) and that's probably why I get so much higher FPS than you.  But it can't OC any further and I'm sure it's well behind more modern CPUs on single thread performance, despite the reasonably high clock speed.

It just makes me sad, seeing slow FPS and also seeing the game using only 14% of my total CPU (I have quad core with hyperthreading, which shows as 8 logical CPUs, so 100% of one thread = 12.5% + a few % for the graphics which are the only processing that happens in a second thread.)

Maybe the next version will have some performance improvements.  I've seen no mention of them though, so I kind of doubt it.  Just lots more features to make the game even slower and/or need to be turned off to make it playable

Anyway thanks for looking into it Leo.  Hopefully mifki has enough info to try and fix it, or if it now I will get TWBT compiled and find out where specifically it's crashing.

[TheBloke]

Hey Leo, could you let me know what accelerating mods you use?  I'm just getting ready to start a new game.  Well I already started a new one actually, but only played it an hour and now am thinking I should have used the opportunity to put in some mods.  So I might start again.

Like you, I really don't care about having 100 different material types that all do the same thing.  In fact I find it kind of irritating having so many different types of stone, and every time I look them up they're all value 1, no special properties.  I just find that tedious.

You mentioned modest mod, which I've heard of - is that all you use?  Or can you recommend any others?  I was also considering Masterwork DF which I understand also limits the amounts of materials, maybe even more?  Like only having "leather" instead of every individual kind of leather, etc.

I'm going to take your advice on this new game and not dig out so many big wide open areas where possible.  And I will also investigate quantum stockpiling - I don't really care if it's a cheat or whatever.

So any advice you can give as to how you accelerate your game would be much appreciated.   When I started the new game last night I was getting 100/40 FPS (both matching their configured FPS cap) and it was so nice!  But I know that is going to deteriorate fast as I keep playing.  So anything I can do to keep it high as long as possible will be great.

[TheBloke]

Just to add that I've installed Modest Mod + Accelerated.  But would be interested to hear if you use anything additional to those, Leo?

[LeoCean]

I don't use that, I made a version back in 40.08 for personal use and was gonna upload it but had errors so I didn't. It's still not perfect  . I've downloaded that version though to see what they do differently though, but it's pretty different.

[konni]

Hi guys,
does anyone have a 20x20 version of Spacefox? Or 24x24?

[Max™]

This is an older version of most of it and Phoebus at 24x24. At some point when I get better at preserving colors I'll try to scale Spacefox up so Dibujors lovely animals are included.

Edit: fixed link.

[LeoCean]

You can just use twbt tilesize 24 24 in extradfhack.init. Paste that line there. It will scale it all to 24x24 and should still look pretty decent.