Print

Please login or register.
1 Hour 1 Day 1 Week 1 Month Forever
Login with username, password and session length

[zilpin]

...My understanding of multithreading tells me that it should be pretty easily applicable in principle to the way the game works, but I know absolutely nothing of its internals and so can't speak on the subject reliably like he can.

Multithreading is never pretty or easy in C/C++.
Even separating the pathfinding from the rendering from the user input results in unforeseen consequences.
Damnable race conditions always come up.

But have no fear, for one day The Almighty Toad shall write Slaves to Armok Chapter III ... in APL ... and it shall be good and pure.

(he is a mathematician, after all)

[Telgin]

Okay, you got me. 

No, multithreading anything manually is an absolute pain even with what seems to be fairly trivial cases.  I know.  I've tried.

What I meant though is that supposedly Toady doesn't know of a good way to even apply multithreading to the game to make it faster, which seems strange to me.  Maybe he's right, since I don't have the code, but it at least seems to me that the theory behind it shouldn't be hard to apply.  Taking the game update step and letting two threads perform updates on half of the object set each, for example.

No, it's not literally that simple (or really even close), but multithreading seems like it should be blatantly applicable to it.  I understand the issues with data dependencies between the threads (i.e. two dwarves decide to move onto the same spot from different threads, who gets there first?), but it's not an insurmountable problem.


Anyway, to get myself back on topic, I'll mention that one thing that sort of disappoints me about DF right now is the fact that I have a soldier with a rotten pancreas and she really couldn't care less.  I'm very much looking forward to improvements in the way body parts are handled (these are coming, right?).

[kaenneth]

Multithreading at the smallest level is HARD.

Say you have a list of objects that need something done to them.

So you divide up the list to hand out to other threads.

Problem 1) divvying up the list requires looping through the list and doing something to them... which is exactly the problem you were trying to solve.
Problem 2) if the something being done requires modifing the objects, threads running on different cores may continually cause cache invalidations, making it several time slower.
Problem 3) inserting/deleting from the list while iterating becomes near impossible (not a big problem for DF I think, as it builds up a history, without reusing entries), however there is always the chance that 2 threads will want to append an item at the same time. (I solve that in my code by having each thread fill a queue with pending inserts/deletes to be processed by the main thread at the end of each cycle... lame, but it works)

Best way (I think) around most of these problems is to have each thread 'own' some portion of the objects, instead of trying to maintain a single list that needs to be split and re-merged each frame. you still have to deal with cross-thread interactions, but that can be reduced by intelligent partitioning of the objects (like every few thousand frames assign all the objects on the left side of the map to thread a, and on the right to thread b.)

anyway, it's on my mind since I'm trying to work out a threaded simulation engine for my own project.

[Andir]

I think DF would fair better with a "client/server" type of multi-threading.  Where the server spawns processes that control a subset of creatures/water mechanics/etc.

EX: Server needs group of elephants, spawn controller, wait for connection, send local data to client, wait for return information on what the elephants are doing with their turn.  Let server resolve discrepancies [Elephant trampled dwarf, tell dwarf controller one died].  It could become a liability though if the client crashed.  The elephants would have to be passed onto another client or would just vanish.  They've done turn based clients like this for multi-player turn based games (MOO2?, GalCiv...) and adapting it to wait for responses from sub-clients is easy enough.  You can even set timeouts to force split long running clients into more fine grained subsets.  (That group of mobs is taking too long to path?  Split into two processes dividing the creatures up.  [though, it's probably a pathing issue because mob group controllers should have a local cache of paths...])

It would also make it trivial to add player clients where someone can connect to your base and walk around.   (The server would only wait a set time for them so one client doesn't prevent the server from continuing.  Only the "God Client" would be able to pause the simulation.)

[Rockphed]

Problem 1) divvying up the list requires looping through the list and doing something to them... which is exactly the problem you were trying to solve.

No it doesn't.  Most lists are traversable in both directions, or are really easy to divide in half.  With a binary search tree, you can just go down and spawn* a new thread to traverse each lower tree until you reach however many threads you need.  With linked lists, you just need to be able to start at both ends and possibly in the middle, of course then you need to check which items have been done already.  In short, how easy it is to divide a list in half, or thirds, or fourths, or 162nds depends on how the list is organized.  If you need to traverse the entire list to divide it in half, you need a better list.

Not that I have any clue about HOW to multithread.  My computer programming skills never got near there.  But I suspect that the other problems are more relevant to the issue.

[Jerg]

Only 1 thing turns me off about DF.

Migrational waves are too frequent and with too many.

I don't mind ending up with very populous fortresses, but I'd like the game much more if each wave only has say 3 dwarves minimum and 8 dwarves maximum, and only occur once a year.

[Areyar]

Only 1 thing turns me off about DF.

Migrational waves are too frequent and with too many.

I don't mind ending up with very populous fortresses, but I'd like the game much more if each wave only has say 3 dwarves minimum and 8 dwarves maximum, and only occur once a year.

'tis more a personal preference this.
I think this can be partially attained by lowering the maximum allowed fortress population in the raws to below the current pop and increasing this to slightly above the current pop once every few years or so. The (last) migrant wave size does not get smaller IIRC, but at least you could manage the frequency of them occuring.

[Telgin]

More control over migrant waves in general would be nice.  In my current community fort, I'm trying an above ground embark while still attempting to give every migrant their own room.  This quickly turned out to be impossible, as you'd need an army of carpenters or masons to get bedrooms built as fast as migrants arrive, even with a paltry fort wealth.  My first migrant wave was about size 3 I think.  Everything after it was about 12.  Ridiculous, if I could have kept that at 3 it would have been perfectly fine.

Funny thing is, I turned pop cap down, but I still received two migrant waves past the 50 mark.  Presumably they were on the way already.  Frustrating, that.

Then there's the fact that I get useless migrants.  I have no intention of ever needing dyers in this fort.  Yet, I have a grand master dyer and a proficient dyer.  I can just repurpose them, but some control over the immigrating skills would be nice.

Of course, this has all been discussed at length in suggestion threads many times.  I'd still like to see it someday though.

[daggaz]

This game is AWESOME, and the complexity is very much part of what makes it special and draws players like me to it.   That said (and we start with the obvious of course):

1. ASCII (and im an old MUD player).  Get an official tileset.  Make it easy to change ingame (see minecraft) when people get that far. 
2. UI.  Needs mouse support.  Menus need to be consistent and more clearly organised.  They need to be more consolidated and easier to get thru. 
3.  World creation..  make more obvious toggles, like "volcanos" or "aquifiers". Aquifiers themselves are horrendous as they stand now, they spam the whole map and are a horrendous trouble to even experienced players, and you have to edit game files to fix them.  I had to google to find out what "embark sites" really meant.
4. General unit oversight.. one of the most overwhelming things for new players is there is NO (obvious) way to quickly see how things are running.. you have the limited jobs menu, the messy units menu, and 3rd party dwarf the rapist, which in itself is a good start but not quite up to par.  Just finding your guys on the map can be horrifying, especially if you are trying to dig thru the alerts menus.
5. Military.  Sort out uniforms, training, and all the assorted bugs.  Make it easier and more transparent/obvious how to control them.  This is major.
6.  General bugs and game balance.  FPS issues and justice, for two big examples. 

At this point, I would say it is a good time to stop pushing forward, take a big breath, and go back in to the core of the game and clean up shop.  Streamline it, make it nice and im sorry to say, but more undwarfy.  Leave the dwarfiness to the actual dwarves (which i wouldnt change for the world).    If it doesnt get the knots and kinks out at this point, it probably never well, and you will end up with a bunch of wonderful ideas getting lost and broken on the tangled mess in the middle. 
 

[MrWiggles]

Well, according to Toady, he hasn't reached the core of the game. So, there no 'core' to go back to yet, and make shop.

[Jerg]

Only 1 thing turns me off about DF.

Migrational waves are too frequent and with too many.

I don't mind ending up with very populous fortresses, but I'd like the game much more if each wave only has say 3 dwarves minimum and 8 dwarves maximum, and only occur once a year.

'tis more a personal preference this.
I think this can be partially attained by lowering the maximum allowed fortress population in the raws to below the current pop and increasing this to slightly above the current pop once every few years or so. The (last) migrant wave size does not get smaller IIRC, but at least you could manage the frequency of them occuring.

That's the thing though, I don't mind having no cap on the population max; in fact, I want it that way. Instead, it'd be awesome if there is an option to control approximately how large and how frequent migrant waves are.

[zilpin]

That's the thing though, I don't mind having no cap on the population max; in fact, I want it that way. Instead, it'd be awesome if there is an option to control approximately how large and how frequent migrant waves are.

+1.
Even just a simple toggle to disable migrations, with no fine-grained control, would be nice.
Within 10 years, you could very well have a Lord of the Flies fortress.  With a Pig Skull totem & all.
(I still prefer bugfixes be done first).

[Kogut]

Even just a simple toggle to disable migrations, with no fine-grained control, would be nice

Done, and called population cap (except 2 forced waves).

[zwei]

Even just a simple toggle to disable migrations, with no fine-grained control, would be nice

Done, and called population cap (except 2 forced waves).

Which is not really same thing that ingame option that would be:

a) Toogleable without restarting game
and
b) Savegame specific

[KFK]

At this point, I would say it is a good time to stop pushing forward, take a big breath, and go back in to the core of the game and clean up shop.  Streamline it, make it nice and im sorry to say, but more undwarfy.  Leave the dwarfiness to the actual dwarves (which i wouldnt change for the world).    If it doesnt get the knots and kinks out at this point, it probably never well, and you will end up with a bunch of wonderful ideas getting lost and broken on the tangled mess in the middle.

And the longer it waits the harder it will be. Knowing that is probably what turns me off DF the most: the whole development focus strikes me as backwards.