Bay 12 Games Forum

GavJ, go ahead and write a larger project using multi-threading. It seems to me you got some theoretical ideas about threading, but never had the "joy" to work through a complexer project utilizing it, and seeing how tons of semaphores a) eat up your coding time, b) eat up your debugging time and c) even eat up cpu time gained first place.

Nobody says there isn't some value gained, but you're definietly overestimating it. Unless you'd go some completely different like SIMD.

Like I told you from bening, which has been rephrased a few times by others. At the end you'll have to have one main thread that collects all the computed data and applies it. It be the big semaphore all stuff will have to go through. This will be a bottle neck. It will be faster than currently, but with 8 cores, definitely not 8 times faster. If you'd leave that main thread away, you'll have no predictable race conditions and chaos. If you have it, you'll have the pathing diamond shape. Which is the worst kind of shape.

I imagine with classical cores, you'll get twice as fast as currently or something like that. Not much more, you'll hit a limit.

This whole thread discussing threading is pointless, if you want to, you're free to write a DF-clone using multithreading. Thats all that will ever get out of it.

The DF code suffers much bigger performance from much more trivial things anyway, like fat dwarves, or the container-temperature flickering and the such. Sometimes I got the idea, it does a lot of binary searches, where hashtables would be applicatable.

Honestly, how exactly its working on base level I'm no expert in, I've doing most of my coding on levels where these things are abstraced away, so take it with a grain of salt.

On a classical computer (desktop) there is only one system bus, so when another core wants to use it, it has to wait to get free. Cores have their own caches tough, so if the memory is in the cache, it doesn't have to use the bus. There is also prefetching going on, if the bus is idle, which is a guess-work by the CPU what memory might be used next. etc. There has been a lot of research into that. One core can block the whole bus if it does nothing but truely true random memory access. But in practice, a) access is not random but focused on pages that get cached, b) the core wants to do some calculations before it needs new memory again, which gives another core air to use the bus.

You can't just multiply bus access. A RAM chip only has so much wires.

Honestly as coder, I personally hate classical multi-threaded code. Simply because its a nightmare to debug when you have a race condition, a lock up, or randomly destroyed memory due to a failed mutex. Around the millennium when it was the cool thing to do, I also did some multi-threaded stuff, nowadays I avoid it like hell.

Its correct that many things that take some process load, like path finding, can be computer in parallel with classical multithreading from one tick to the next. However, you'll need one master thread to collect all computations and execute all the changes on the main data tree which then will become a bottle neck. Otherwise if you allow all threads to apply changes on the main data tree, welcome to race condition and mutex nightmare. With mutexing you have the classic problem, make a few, and you'll gain little benefit from multithreading since one thread will hold large pieces of the tree while others wait for it, or go fine grained, and lose a lot of the speed benefits on the load created by mutexes.

I agree that it is impossible to add something like this on an existing code base. If it would have been to be planned from start. Anyway, I agree with putnam. Maybe you get +100% or so speed. But then you'll hit a limit with classical multi-threading, no matter how many cores you go.

I didn't follow CPU-developments of late, but I got the idea that also here, the multi-coreing hits a limit, since unless you're creating something like a VirtualMachine host for lots of VMs, many have learned, you can only take useful advantage with a few cores, while more hardly get anything new. I think we have to accept we're hitting a limit not only on linear speed, but also on parallelism with core in Van-Neumann design.

One thing that might be big is being able to utilize the GPU for DF calculation. I googled a bit, there are people that use GPU for path finding. BTW: Even on supercomputers, most of the top hundret nowadays go vector-based, GPU. There is less and less classical calculating in the upper area.


If I'd had the time to design DF from base, I'd go multiprocess with a database design. Have one main process (the database) that holds the world but does not compute anything, and a few processes for stuff that communicate with the main process via a socket what they'd like to change from tick to tick and getting the changes from the world as answer. Creatures, stuff and areas can be assigned to processes.

Toady has laid out arcs of future development;

There are a few possibilities what exactly would become and determining which is the most likely is more due to personal experience.

However, one I can tell you for sure. No human will dedicate his or her life to follow the arc laid out by toady. Toadys arc dies with toady, thats for sure.

If someone else wants to dedicate a hugh effort into, you can be sure that will be to fuel his or her own ideas.

Anyway, to my experiences in events like tis, the project at large is more likely to die. You cannot expect a coding community to start from zero to full should the event arise, such a thing needs years of caring to build up. Small fixes, chaotic add-ons, yes. But no more a dedicated project.

My opinion tough.

And I believe it might be a better idea to restart from scratch anyway, than continuing on a decade old code base which nevertheless how awesome the game as idea is, isn't the most efficient in execution in a number of areas, albeit looking into some stuff like the world creation code wouldn't be bad to be able to quickly boot up a clone project to a playable state.

I mean there are a sure a few of us who'd like to code a DF-like from scratch, but alas, all of them do not have the time to.

Anyway, DF has already inspired a whole generation of games, albeit none is similar enough to be considered a replacement candidate.

I don't get all that focus on traders.

After you traded for your anvil if you didn't embark with one, got all the available seeds and got some yarn/silk cloth you might not get easy without for moods, maybe a few bars of steel to get the leggings forging/smelting process going,
there is nothing interesting coming from caravans anymore. Except elves bringing some interesting beasts and they don't have wagons anyway. Maybe wood if you are in a scarce area and didn't want to go into caverns or make a tree farm with large patched out soil or mudded rock.

I care for the living in fortress mode. When I restart playing I even had to swear to myself not to go too emphatic, I remember getting a hurt king appearing on the map. Nothing I could do to save him.

Adventurer mode, however, somehow gets the worst out of me. I mean hammering some poor animals leg for training purposes for hours, even already ever bone in it is broken and it must be flat as a atom layer. After the adventurer got too tired, finally hammered its head a few times to put an end to it.