Bay 12 Games Forum

Dwarf Fortress => DF Bug Reports => Topic started by: Beedub on April 04, 2010, 12:06:48 pm

Title: [0.31.01] Periodic Lockups
Post by: Beedub on April 04, 2010, 12:06:48 pm
Hi hello.  Long time reader first time poster.  Been enjoying the new version immensely, however, I do have one problem I cannot find an explanation for.  I'm using the new mayday tileset, but the problem is that every so often during normal dorfing activities the client will completely lock up for a period ranging from a split second to several seconds.  The consistency seems to be random, and I cannot manage to replicate it on demand.  I'm on a relatively quiet area, the only major feature (that I know of) is a small brook.

I'm getting 100fps quite easily 99% of the time, except when these mini lockups occur, whereupon it will drop to zero for the duration.  I'm running windowed mode at 800*600.  I've talked a few other people that have similar problems, but other friends I know that play say they have nothing like this occurring.  I'm slightly at a loss here.

System specs are something like; Intel QuadCore, 8gigs of ram, GTX285, Windows 7 Ultimate.

Any ideas?
Title: Re: [0.31.01] Periodic Lockups
Post by: Captain Xenon on April 04, 2010, 12:32:41 pm
ive seen this as well, where the game stops responding for a few seconds. its very annoying, but when i made a new fort things were fine again. so it seems to be very dependant on something specific to a fort.
Title: Re: [0.31.01] Periodic Lockups
Post by: Beedub on April 04, 2010, 01:16:07 pm
I should also add; I'm not sure if it's related to my above issue, but I have seasonal autosave on, and whenever the season rolls over and it attempts to save, it takes quite a while to save the game, and it actually puts the client into a "not responding" state for a bit.  I can't remember it doing that in 40d.
Title: Re: [0.31.01] Periodic Lockups
Post by: Neskiairti on April 08, 2010, 03:49:16 pm
Ive actually got the same problem. both the save freeze and the mini lockups, some times as much as 10 seconds happening every 3 or 4 seconds.. or as few as half a second once a minute or so. (or anywhere else along those two spectrum)

the save freeze can last up to 3 minutes some times. AT the very least you might think about running the data save in a separate thread and allowing the rest of the game to just sit idle (yet responding) waiting for that thread to finish.

specs:
Phenom II X4 955 clocked to 3.6ghz
4GB 800Mhz DDR2
GeForce GT240 1gb
3ghz Sata 7500RPM HDD
Windows 7 Pro 64Bit
Title: Re: [0.31.01] Periodic Lockups
Post by: Wyrframe on April 08, 2010, 04:50:53 pm
Ditto. Full framerate, but every 20 seconds there's a 1-second complete freeze. Zero disk activity, and the CPUs aren't breaking 30% load.

Core i7 4x 2.8Ghz
6GB 1333hz DDR3
GeForce GTS 360M 1GB
Windows 7 Home Premium 64-bit
Title: Re: [0.31.01] Periodic Lockups
Post by: Helix Marine on April 10, 2010, 09:27:46 pm
Having the same thing happening, makes the game almost completely unplayable.

Edit: it has happened on both of my forts in version [0.31.01] and [0.31.02]
Title: Re: [0.31.01] Periodic Lockups
Post by: askovdk on April 11, 2010, 07:20:23 pm
It's probaly not that, but 'mini lockups' may be related to re-path finding for everyone at once.
I get a mini lockup every time I raise or lower a draw bridge or flood gate. My guess is that all paths are recalculated as the action may have invalidated a former set path.
Title: Re: [0.31.01] Periodic Lockups
Post by: Helix Marine on April 11, 2010, 09:48:14 pm
I have 3 drawbridges and it doesnt lag at all when i open/close them.
Title: Re: [0.31.01] Periodic Lockups
Post by: Kuraudo on April 12, 2010, 10:21:08 am
Same problem, although it's not nearly enough to drag me away from this game.

As a previous poster mentioned, I too notice it without fail when drawbridges and floodgates are open/closed. 

At every other time I'd say it hits every 100 steps or so - it's fairly regular.
Title: Re: [0.31.01] Periodic Lockups
Post by: Helix Marine on April 12, 2010, 07:37:45 pm
Quote from: Kuraudo on April 12, 2010, 10:21:08 am
Same problem, although it's not nearly enough to drag me away from this game.

As a previous poster mentioned, I too notice it without fail when drawbridges and floodgates are open/closed. 

At every other time I'd say it hits every 100 steps or so - it's fairly regular.

I said it doesn't lag with drawbridges for me or floodgates and doors.
Title: Re: [0.31.01] Periodic Lockups
Post by: Gauphastus on April 13, 2010, 08:40:26 am
Just had the game do one final big lockup on me. It's just not budging now.
I also notice my memory use is at about 700000 K. Must be a leak somewhere.

EDIT: Then again, I am on a pretty large map.. and I think I read that large embarks cause issues at the moment.
Nevermind that then.

Still, on the topic of the constant regular, but brief lockups, I just.. can't figure out what might be causing it.
Title: Re: [0.31.01] Periodic Lockups
Post by: Helix Marine on April 13, 2010, 05:57:51 pm
Has anyone tried making a new fort in 31.03 yet? i loaded a old game and it still locks up but not as bad.
Title: Re: [0.31.01] Periodic Lockups
Post by: Gauphastus on April 13, 2010, 07:32:25 pm
Yeah, I've got a new fort. Doesn't seem to matter if your fort is big or small either. What you guys are describing seems really steady, almost rhythmic.
No more pathing errors in the error log anymore either. Do you guys have anything?
I've just got two errors from Mayday's graphics set over and over again.

Code: [Select]
Unrecognized Creature Texture Token: GUARD
Unrecognized Creature Texture Token: ROYALGUARD
Relevant? No?
You guys experiencing this locking up, do you get these errors?
Title: Re: [0.31.01] Periodic Lockups
Post by: Helix Marine on April 14, 2010, 05:51:29 pm
Heres my error log.

Spoiler (click to show/hide)

Bug seems to be caused by the well for me.

Edit: I removed the well and game ran at 80-95 FPS with no lockups afterwards
Title: Re: [0.31.01] Periodic Lockups
Post by: Kishmond on April 14, 2010, 10:52:31 pm
Ever since 40d and on every version since it stops responding every time it saves or loads. Now it has started crashing frequently on such occasions.

Windows Vista 64-bit, AMD Phenom II quad core 2.8 ghz, 4 gig RAM

Does everyone here have a 64-bit OS?
Title: Re: [0.31.01] Periodic Lockups
Post by: Gauphastus on April 15, 2010, 02:18:19 am
I do indeed.
I'll just put my specs in my signature for the time being.
Title: Re: [0.31.01] Periodic Lockups
Post by: Kishmond on April 16, 2010, 01:17:41 pm
Well, imagine that.

After upgrading to windows 7 64-bit with the exact same hardware and the exact same saved game it's no longer hanging on save or load.

EDIT: It seems I spoke to soon. It's started up again. I wonder what could be causing it that it didn't do it the first few times.

EDIT2: Wait, I just realized what the difference was.

EDIT3: Nevermind, didn't help.
Title: Re: [0.31.01] Periodic Lockups
Post by: n00bs4uce on April 17, 2010, 01:40:25 am
I can confirm that I am having these lockups in .03 version. Might be the mayday pack, because that's the only common denominator I see.
Title: Re: [0.31.01] Periodic Lockups
Post by: Lord Byte on April 17, 2010, 04:20:57 pm
I've had the same thing, mayday pack and I found one other factor :) It happens EVERY time it starts freezing / thawing! In my fort that's twice per change (once when parts near rocks freeze, once more when everything except stuff inside freezes, and then the other way round around spring).

Only temporary lockups I've had!
Title: Re: [0.31.01] Periodic Lockups
Post by: Hyndis on April 18, 2010, 01:46:59 pm
Definitely seeing this with changes in weather. Looks like as soon as the river unthaws it needs to do some calculations, then it resumes.
Title: Re: [0.31.01] Periodic Lockups
Post by: Levi on April 26, 2010, 08:44:30 pm
I also had this problem (and have a 64bit OS).  I fixed it by removing my well.  Strange, but it worked.
Title: Re: [0.31.01] Periodic Lockups
Post by: BubbaBrown on April 27, 2010, 12:35:17 am
Ditto on the random lockups.   Core i7 920, 6GB ram, Intel solid state hard drive, Radeon HD 4890, and Windows 7 x64.  The way the CPU usage is and other behavior makes me think that there are some near deadlocks occurring.  Might try to run the game with the affinity set to a single core to see if it makes a difference.

EDIT:  Set the dwarfort.exe process to a single core affinity.  These lockups seem to have all the symptoms of deadlocks:  Programs slows down, but CPU usage also drops.  Wasn't as easy to track before since Windows will try to have threads do work on as many processors.  I don't get lockups as much as massive slowdowns:  Dropping from 100 to 32 then back.  And this is on a new game with a 4x4 embark.  So, multi-core systems might be at a disadvantage for this game.
Title: Re: [0.31.01] Periodic Lockups
Post by: Jay on April 29, 2010, 06:26:27 pm
Quote from: BubbaBrown on April 27, 2010, 12:35:17 am
So, multi-core systems might be at a disadvantage for this game.
Well, that was a huge generality that's entirely untrue.

Seriously.  Just because you're having trouble does not mean everybody with one equal variable does too.
64-bit Windows 7 Home Premium, i7-860 OC @ 4.00 GHz (x4) here, constant 100 fps on a 4x4 embark for at least 3 years.
No Mayday pack, no seasonal autosave.
Title: Re: [0.31.01] Periodic Lockups
Post by: BubbaBrown on April 29, 2010, 08:00:20 pm
Quote from: jaybud4 on April 29, 2010, 06:26:27 pm
Well, that was a huge generality that's entirely untrue.

Seriously.  Just because you're having trouble does not mean everybody with one equal variable does too.
64-bit Windows 7 Home Premium, i7-860 OC @ 4.00 GHz (x4) here, constant 100 fps on a 4x4 embark for at least 3 years.
No Mayday pack, no seasonal autosave.

I did say, "might be".  It was based off the observation of strange behaviors that are similar, symptomatically, to issues I run into when debugging multi-threaded, multi-process, or cluster distributed applications.  There are many applications that are threaded, but have issues with Windows dividing up the processing across other processing units.  This is because they weren't designed to handle asymmetric processing issues that arise out this.  There's a big difference between work being done concurrently and consecutively.  Also, I noticed a significant reduction of complete freeze ups that trip Windows's detection of hung processes and saw far more slow-downs and bog-downs instead.

All I did to cause this change was to assign the dwarfort.exe affinity to a single idle core.  This also allowed me to watch a rollercoaster effect when it comes to CPU usage that coincided with bog downs.  Game bogged down, but CPU usage dropped.  Beforehand, many of the these would have become complete freezes.

The visibility of these issues can also vary because of the "race condition" that sometimes causes them.  You may not being seeing anything obvious, because of the speed your computer is presently running at.  The race condition, presently, is being won in your favor because the game is processing things in a correct enough order to avoid any issues.  You might experience completely different behaviors if there were other tasks running on the system hogging up resources enough to upset the tricky balance and order.  This is one of the bigger pains in parallel processing:  Glitch reproduction across dissimilar platforms.

So...   I am making a generalization, technically, but it is made with considerable consideration making use of specific experiences and specialized education.  Again, it is made without being able to see the source code and practices in use.  And the statement was meant to be a suggestion for the basis of exploration into possible causes for an issue rather than a direct implication of the cause for an issue. 
Title: Re: [0.31.01] Periodic Lockups
Post by: Firehound on April 30, 2010, 05:02:35 am
IIRC DF is not dual-quad-multi-core compatible yet. it still thinks there is exactly one core. And refuses to believe otherwise.

In other more related news, I believe I have also suffered this issue on a fortress on a 6x6. Almost every ten minutes it'd freeze up, I figure something to do with pathfinding or some underground creatures(Had peirced underground 1)
Title: Re: [0.31.01] Periodic Lockups
Post by: manbeef on May 01, 2010, 10:14:41 pm
So I've just confirmed that removing the well stops the lockup problem for me.  Basically the game becomes unplayable once I reach a certain size of fortress, and I guess it makes sense that that point is when I build a well.  If other people are having this problem, try removing your well.  The second that it is deconstructed the lag stops for me.  Then try to rebuild it.  As soon as it was finished building the lockups returned.  I'm running Win 7 64-bit/Mayday.