Well if it really comes down to it if performance gets bad enough i can fully see the community helping make dwarf fortress run fast working together there is a tight community here and if toady needs help I can see many of the community working with him to fix it me included.
A successful game ends because the simulation slows to the extent that it's like watching paint dry. What other game ends for that reason?
Also, performance is relative. A lot of newer DF players express opinions from time to time that 20 fps is "too slow for them", where a lot of older players have learnt to put up with as low as 5 fps. I suppose this all hinges on how "fast" you feel the game should be.
It's entirely possible, but it's also entirely the wrong time in the game's life to do so.
...
Besides, I for one would MUCH rather Toady worked on shiny new things and getting rid of bugs then optimising it so I can have giant fortresses.
Also, performance is relative. A lot of newer DF players express opinions from time to time that 20 fps is "too slow for them", where a lot of older players have learnt to put up with as low as 5 fps. I suppose this all hinges on how "fast" you feel the game should be.
That is just excuses, from a player point of view. The game should be faster, a lot faster. Nothing is gained by playing in slow motion at 15-25 fps. If it is possible to convert all the ancient code into a more optimal running code is the question.
Going multi-thread would not only be a colossal time investment on Toady's part, it's not actually a guaranteed performance boost. Multi-threading done right can be very powerful, but done wrong it can actually slow your program to a crawl if you even manage to keep it stable, and it's specifically dreaded in game programming. Multi-threading is typically only useful if your program has a lot of cache misses slowing it down, which DF probably has, but switching over is such a monumentally complicated, time-consuming and error-prone task that you could have played dozens of games over the years it could take Toady to manage making DF multithreaded without breaking the program.
If you're upset with poor performance, just let computer science advance and hold off playing the game until technology can support a faster DF without requiring unreasonably drastic changes to the program. Heck, within the next decade we could be seeing true commercial quantum computing, memristors or some other crazy invention.
Technically, Dwarf Fortress is doomed. It won't survive the heat death of the Universe. But this isn't particularly relevant to performance issues.
This is not how hardware scales over time, and I won't speculate on radically different architecture with an uncertain future.
I wouldn't go that far,just 30 seconds of research would show you Toady is likely more then capable of it:Going multi-thread would not only be a colossal time investment on Toady's part, it's not actually a guaranteed performance boost. Multi-threading done right can be very powerful, but done wrong it can actually slow your program to a crawl if you even manage to keep it stable, and it's specifically dreaded in game programming. Multi-threading is typically only useful if your program has a lot of cache misses slowing it down, which DF probably has, but switching over is such a monumentally complicated, time-consuming and error-prone task that you could have played dozens of games over the years it could take Toady to manage making DF multithreaded without breaking the program.
I didn't suggest Toady take care of it himself because I know that he simply cannot.
Adams earned a degree in mathematics at the University of Washington.[2] He applied for his doctorate at Stanford University, completing it in 2005 with a dissertation titled "Flat Chains in Banach Spaces", which was published in The Journal of Geometric Analysis. During his first year at Stanford, he was under pressure, the professional environment and competitiveness affected him negatively. He cited the conflict between studying mathematics and developing video games as the reason. This stressful situation left him depressed and he admitted to having a brief encounter with drugs.[8]
In 2006, he started his post doctorate in Texas A&M, which was his goal since his undergraduate days. He decided to leave during the first year due to the increasingly stressful situation[3] and is said to have broken down in the head of department's office. He left in the same year after receiving a stipend, to devote his full attention to developing Dwarf Fortress and other games, which was until then only a hobby. He said, "At the end of a math problem, you have a paper and maybe you publish it, and the paper can be a building block for the edifice of mathematics, but to me that’s not so important. But working on a problem and having a game when you’re done? That’s pretty damn cool."[2]
I wouldn't go that far,just 30 seconds of research would show you Toady is likely more then capable of it:Going multi-thread would not only be a colossal time investment on Toady's part, it's not actually a guaranteed performance boost. Multi-threading done right can be very powerful, but done wrong it can actually slow your program to a crawl if you even manage to keep it stable, and it's specifically dreaded in game programming. Multi-threading is typically only useful if your program has a lot of cache misses slowing it down, which DF probably has, but switching over is such a monumentally complicated, time-consuming and error-prone task that you could have played dozens of games over the years it could take Toady to manage making DF multithreaded without breaking the program.
I didn't suggest Toady take care of it himself because I know that he simply cannot.Quote from: WikipediaAdams earned a degree in mathematics at the University of Washington.[2] He applied for his doctorate at Stanford University, completing it in 2005 with a dissertation titled "Flat Chains in Banach Spaces", which was published in The Journal of Geometric Analysis. During his first year at Stanford, he was under pressure, the professional environment and competitiveness affected him negatively. He cited the conflict between studying mathematics and developing video games as the reason. This stressful situation left him depressed and he admitted to having a brief encounter with drugs.[8]
In 2006, he started his post doctorate in Texas A&M, which was his goal since his undergraduate days. He decided to leave during the first year due to the increasingly stressful situation[3] and is said to have broken down in the head of department's office. He left in the same year after receiving a stipend, to devote his full attention to developing Dwarf Fortress and other games, which was until then only a hobby. He said, "At the end of a math problem, you have a paper and maybe you publish it, and the paper can be a building block for the edifice of mathematics, but to me that’s not so important. But working on a problem and having a game when you’re done? That’s pretty damn cool."[2]
other then that,everybody else beat me to the point.
(Maybe, this final update before speedup cleaning will finally allow Masterwork to catch up ;) )Given how much fun new stuff has come out in v42, I am doubtful that there will be much interest in Masterwork by the time Toady decides he needs to optimize. However, I've never used Masterwork so I might very well, much like OP, be talking about something I know nothing about.
The problem here is that you want reassurance that something will be done, specifically with a nice timeline so that you feel your anxiety lessen.
You're not gonna get that.
What you are gonna get is "this is toady's baby, he'll inevitably work on fps issues as it starts bothering them". Does this mean that maybe he'll spend a whole year rewriting fps blockers? Probably. Will this kill player enthusiasm for Dwarf Fortress? Well, considering he's done 3 big updates that took a full year already, I somehow doubt this.
DF has been in development for a long time. The community is used to living the slow life and is not gonna topple over when toady isn't releasing new updates every week. (Maybe, this final update before speedup cleaning will finally allow Masterwork to catch up ;) )
It is a specialists job. Not every programmer can do everything equally well, especially not if one to a degree is self taught.
Given how much fun new stuff has come out in v42, I am doubtful that there will be much interest in Masterwork by the time Toady decides he needs to optimize. However, I've never used Masterwork so I might very well, much like OP, be talking about something I know nothing about.
To quote scripture, premature optimization is the root of all evil.
Toady could very easily optimize a lot of things right now on that front, that could give in principle enormous speed up benefits.. Like a factor of 10 or more. Pathing for instance is a major compute hog, and could be ameliorated considerably.
The problem is, these sorts of optimizations are so specific that it's unlikely they will continue giving the same benefits if something changes drastically (Pathing for instance will radically change if you implement a rail system) and in fact could make things worse.
You really want the optimizations right before the final release.
some of the most vocal fans suggest FPS death doesn't really matterFor the record, nobody actually thinks this (and if they do they're wrong) but it isn't a priority while the game is in active development.
~~~Please don't bicker
It's entirely possible, but it's also entirely the wrong time in the game's life to do so.
...
Besides, I for one would MUCH rather Toady worked on shiny new things and getting rid of bugs then optimising it so I can have giant fortresses.
DF is in continuous development, new features are added and will be until the end of time, there will therefore be no end point when the game is done and can be optimized. I would rather that Toady spend a year restructuring the code base now, and then continue to expand it than the current slow walk towards total fps dead.
Toady has made 2 year releases before, he can do it again too.Except he doesn't want to, because he didn't like it, the fans didn't like it, and the donations dropped significantly over that period. Toady has said many times that he wants to make smaller updates with a shorter cycle.
Actually he has a well laid out plan for the game (ON NOTECARDS believe it or not) and the 42, means its 42% doneThat's not how version numbers work. 0.42 means Toady has made forty-two major versions of the game thus far. It could finish at 0.85, or go on to 0.1337. It is not a percentage.
You are wrong, his version scheme is based on 100 and it is also based on core components 100 core components is version 1.0 . http://dwarffortresswiki.org/index.php/DF2014:Version_number http://dwarffortresswiki.org/index.php/v0.34_Talk:Version_numberToady has made 2 year releases before, he can do it again too.Except he doesn't want to, because he didn't like it, the fans didn't like it, and the donations dropped significantly over that period. Toady has said many times that he wants to make smaller updates with a shorter cycle.Actually he has a well laid out plan for the game (ON NOTECARDS believe it or not) and the 42, means its 42% doneThat's not how version numbers work. 0.42 means Toady has made forty-two major versions of the game thus far. It could finish at 0.85, or go on to 0.1337. It is not a percentage.
Toady also said that they need make a new list of what needs to go in before it qualifies for v1.0.0 status so they can better estimate that "percentage" number for future versions (I think this was in one of the DFTalk podcasts).Since this quote toady did that. and it is now in fact approximating percentages.
This technology will come to processing speed in no time: http://www.pcgamer.com/experimental-5d-data-storage-could-store-360tb-of-games-for-138-billion-years/ (http://www.pcgamer.com/experimental-5d-data-storage-could-store-360tb-of-games-for-138-billion-years/)That's only for data storage, think of it as a high-capacity DVD/Blu-Ray. It won't speed up the processing of the data. It means that we might be able to store an entire galaxy of generated DF worlds with millions of years of history onto one single disk. Enjoy the lag! =)
I don't care if improvements come from the most brilliant implementation of multithreading in gaming history or some other means. I just know something dramatic must be done, and that doesn't involve waiting for hardware to improve in a magical way that makes an inefficient engine behave otherwise. I hope the options, however daunting, are being considered.
... "this is toady's baby, he'll inevitably work on fps issues as it starts bothering them". Does this mean that maybe he'll spend a whole year rewriting fps blockers? Probably.
A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.Hey, yeah, why don't we remove the primary feature that makes DF so great? The thing that none of the DF clones actually even try to copy? The part of the game the Toady actually really, truly enjoys making, and part of what makes it a game worth featuring in the museum of modern art? That'll really help development along.
A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.Hey, yeah, why don't we remove the primary feature that makes DF so great? The thing that none of the DF clones actually even try to copy? The part of the game the Toady actually really, truly enjoys making, and part of what makes it a game worth featuring in the museum of modern art? That'll really help development along.
:V
Oh, no, I am fully aware of that. I just have two questions for you.A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.Hey, yeah, why don't we remove the primary feature that makes DF so great? The thing that none of the DF clones actually even try to copy? The part of the game the Toady actually really, truly enjoys making, and part of what makes it a game worth featuring in the museum of modern art? That'll really help development along.
:V
I don't think you understand what I mean, or else have you maybe just started playing this version?
In a not so ancient past, when you started the game the rest of the world froze, whereas now the game simulate a lot of stuff that at the moment have next to no effect on anything related to your game at all.
I made this thread (I'm hardly the first) because the issue should be brought up frequently.
Oh, no, I am fully aware of that. I just have two questions for you.A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.Hey, yeah, why don't we remove the primary feature that makes DF so great? The thing that none of the DF clones actually even try to copy? The part of the game the Toady actually really, truly enjoys making, and part of what makes it a game worth featuring in the museum of modern art? That'll really help development along.
:V
I don't think you understand what I mean, or else have you maybe just started playing this version?
In a not so ancient past, when you started the game the rest of the world froze, whereas now the game simulate a lot of stuff that at the moment have next to no effect on anything related to your game at all.
1) Now that armies and visitors move on the world map, how is anyone going to arrive at your fort if they are not allowed to move in the world? 2) When we're able to send armies out from our fortresses, a feature that is in the works, how exactly are they going to be able to move if the rest of the world isn't active?
The active world is actually the most important part of the game for me. That there is a living world outside my fortress makes everything a lot better than when the world simply stopped. Dwarf Fortress as a project is 100% about that feature being expanded as opposed to eliminated.
Here's a thought: Why not pitch a "Turn Continuous World-Gen Off" feature on the Suggestions Forum? I mean, you can turn off temperature and whatnot, so it's not too far out.Oh, no, I am fully aware of that. I just have two questions for you.A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.Hey, yeah, why don't we remove the primary feature that makes DF so great? The thing that none of the DF clones actually even try to copy? The part of the game the Toady actually really, truly enjoys making, and part of what makes it a game worth featuring in the museum of modern art? That'll really help development along.
:V
I don't think you understand what I mean, or else have you maybe just started playing this version?
In a not so ancient past, when you started the game the rest of the world froze, whereas now the game simulate a lot of stuff that at the moment have next to no effect on anything related to your game at all.
1) Now that armies and visitors move on the world map, how is anyone going to arrive at your fort if they are not allowed to move in the world? 2) When we're able to send armies out from our fortresses, a feature that is in the works, how exactly are they going to be able to move if the rest of the world isn't active?
The active world is actually the most important part of the game for me. That there is a living world outside my fortress makes everything a lot better than when the world simply stopped. Dwarf Fortress as a project is 100% about that feature being expanded as opposed to eliminated.
1) Like they did before. They just spawn (and thereby eliminate a lot of problems, with enemies that never show up)
2) That will be a nice thing, but it is also something that very well could be two or more years out into the future. If it is even added. I don't think ideas of what can be done should be given much weight in regard to current features, especially not in a game like DF with a very slow development (one guy).
Because:Here's a thought: Why not pitch a "Turn Continuous World-Gen Off" feature on the Suggestions Forum? I mean, you can turn off temperature and whatnot, so it's not too far out.Oh, no, I am fully aware of that. I just have two questions for you.A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.Hey, yeah, why don't we remove the primary feature that makes DF so great? The thing that none of the DF clones actually even try to copy? The part of the game the Toady actually really, truly enjoys making, and part of what makes it a game worth featuring in the museum of modern art? That'll really help development along.
:V
I don't think you understand what I mean, or else have you maybe just started playing this version?
In a not so ancient past, when you started the game the rest of the world froze, whereas now the game simulate a lot of stuff that at the moment have next to no effect on anything related to your game at all.
1) Now that armies and visitors move on the world map, how is anyone going to arrive at your fort if they are not allowed to move in the world? 2) When we're able to send armies out from our fortresses, a feature that is in the works, how exactly are they going to be able to move if the rest of the world isn't active?
The active world is actually the most important part of the game for me. That there is a living world outside my fortress makes everything a lot better than when the world simply stopped. Dwarf Fortress as a project is 100% about that feature being expanded as opposed to eliminated.
1) Like they did before. They just spawn (and thereby eliminate a lot of problems, with enemies that never show up)
2) That will be a nice thing, but it is also something that very well could be two or more years out into the future. If it is even added. I don't think ideas of what can be done should be given much weight in regard to current features, especially not in a game like DF with a very slow development (one guy).
Depends on world/lagginess of fort. Obviously. Also, object testing arena? You CAN turn off world activation for an experiment, by not having a world.And weather. And tree growth(?). And jobs/wealth/HFS/wild animals/every other fort thing. All the world info that still needed to be tracked before the activation. It's never just the activation.
words
Technically, Dwarf Fortress is doomed. It won't survive the heat death of the Universe. But this isn't particularly relevant to performance issues.
A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.
A entire other option is to remove some of the features designed over the years, like the dynamic world, in principle it is a very cool feature, but when playing a fort. Do one really care if the rest of the world is changing? As I remember it, that feature did cost something around 20% of the fps.
Yeah, the dynamic world is basically the whole point of the game. It doesn't do much right now, but as development continues we'll be interacting with the world as a whole in very meaningful ways, and vice versa.
Doesn't anyone remember how Z levels killed this game?!
Another thought I just had:Hah! Hahahahaha!
Toady's job is to create the best fantasy world generator and simulator. Intel's job is to create a computer that can run it. Right now, despite having over 100,000 employees, assets over $100 billion, and revenue over $55 billion... Intel is the one falling behind on their end of the process :)
Another thought I just had:
Toady's job is to create the best fantasy world generator and simulator. Intel's job is to create a computer that can run it. Right now, despite having over 100,000 employees, assets over $100 billion, and revenue over $55 billion... Intel is the one falling behind on their end of the process :)
Coming up next: How is this thread not technically doomed?Perpetual motion
(the other 57.94% of the game)
Heh, oh yeah that makes sense. The versioning information.I think we are more than 42% of the way there. Some of the major features (the .01s) are bigger than others, and world activation was a really big one. Another big one or two coming near then end will be cleaning up the interface.
We should make this the general chat thread.I like corn chips.
Makes sense...Yeah, but a lot of the principles apply to things that aren't DF.
The other option I see is that he never gets around to optimization and DF just grinds to, well, not really a halt, but to the point where it's no longer able to do what he wants.No, not at all. When .40 was released the world simulation was really really slow, so Toady spent a few days fixing it up to run faster. Now it's good enough that it has very little impact on gameplay. Toady is capable of optimising when he wants to.
Multithreading is basically untenable for DF mechanics-wise. 90% of the stuff the game does requires things to be done in order, i.e. one thing at a time. It's like multithreading Minecraft - you can put chunk loading, rendering, and all these other things in separate threads, but you can't split up the core game (i.e. where most of DF's lag comes from) without buggering the core mechanics.Either that, or put things like AI(include pathing) in other threads. Problem is you'll never get the same result even if you start the game with exactly the same RNG seed.
That doesn't sound like a problem at all.Multithreading is basically untenable for DF mechanics-wise. 90% of the stuff the game does requires things to be done in order, i.e. one thing at a time. It's like multithreading Minecraft - you can put chunk loading, rendering, and all these other things in separate threads, but you can't split up the core game (i.e. where most of DF's lag comes from) without buggering the core mechanics.Either that, or put things like AI(include pathing) in other threads. Problem is you'll never get the same result even if you start the game with exactly the same RNG seed.
Pathing doesn't use random seeds anyway.What I was trying to say is you don't have the same result even if you stard with the same save.
I could see AI/pathing stuff working if done multi-threadedly, in a fairly simplistic way. Trying to do asynchronous AI updates in DF would be nightmarish.
Makes sense...Yeah, but a lot of the principles apply to things that aren't DF.
1) CPUs will continue to get faster; ferex. AMD is working on a new line of processors optimising single-core performance. Moore's law may be dead, but we will see improvements until the end of time. That's how technology works. It's not something to rely upon, but saying technology won't improve is very strange.
3) You can of course put off optimisation until the code is feature-complete and then gut the entire thing to make it faster. It's not necessarily the best tactic, but it's the one Toady's following (presumably because the bulk of the engine was written ~10 years ago and Toady admits he was not the programmer then that he is now).
Multithreading is basically untenable for DF mechanics-wise. 90% of the stuff the game does requires things to be done in order, i.e. one thing at a time. It's like multithreading Minecraft - you can put chunk loading, rendering, and all these other things in separate threads, but you can't split up the core game (i.e. where most of DF's lag comes from) without buggering the core mechanics.
There's only so much low-hanging fruit he can optimize. Every time it will take more effort to get less gain. Eventually he'll be spending a month to get a 1% gain. There's only so much he can get without dealing with the elephant in the room.QuoteThe other option I see is that he never gets around to optimization and DF just grinds to, well, not really a halt, but to the point where it's no longer able to do what he wants.No, not at all. When .40 was released the world simulation was really really slow, so Toady spent a few days fixing it up to run faster. Now it's good enough that it has very little impact on gameplay. Toady is capable of optimising when he wants to.
No. Once you get transistors so small that the charge from one starts causing the one next to it to fire (which is where they are now), you can't make them any smaller. If they can't get any smaller, they can't really get much faster.That's the same logic that says we'll never be able to go from Moscow to Beijing in a day or two because horses can only gallop so fast.
The short answer is that I don't know how to do it [muktithreading], it'd probably break things and take forever, but there are definitely places where it would help.Should it be done? Probably. Is it going to happen? Not any time soon.
No, it's the logic that says we'll never travel from Moscow to Beijing in less than 19.31 milliseconds.QuoteNo. Once you get transistors so small that the charge from one starts causing the one next to it to fire (which is where they are now), you can't make them any smaller. If they can't get any smaller, they can't really get much faster.That's the same logic that says we'll never be able to go from Moscow to Beijing in a day or two because horses can only gallop so fast.
Anyway, re: multithreading:Exactly. He knows he needs to do it and doesn't know how, so he needs to learn. That's what I said.Quote from: Toady on the reddit AMAThe short answer is that I don't know how to do it [muktithreading], it'd probably break things and take forever, but there are definitely places where it would help.Should it be done? Probably. Is it going to happen? Not any time soon.
sorry i just tried implementing a liquid system using multithreading as a proof of concept and adding a print statement to the start of the chunk processing function makes it run fasterYour code is haunted.
i'm... gonna... just sorta... cry? a little bit
Does it mean on a slower pc, it drains slower? In DF, your miner has a high chance to survive than on a fast pc?sorry i just tried implementing a liquid system using multithreading as a proof of concept and adding a print statement to the start of the chunk processing function makes it run fasterYour code is haunted.
i'm... gonna... just sorta... cry? a little bit
Anyway, Terraria does a multithreaded fluid thing, which is pretty cool. Means you can drain an ocean and only the water will lag, not the rest of the game.
Dig a tunnel. Build a larger/smarter CPU. Lateral thinking. Improvement doesn't have to be limitless, just enough to serve our purposes.No, it's the logic that says we'll never travel from Moscow to Beijing in less than 19.31 milliseconds.QuoteNo. Once you get transistors so small that the charge from one starts causing the one next to it to fire (which is where they are now), you can't make them any smaller. If they can't get any smaller, they can't really get much faster.That's the same logic that says we'll never be able to go from Moscow to Beijing in a day or two because horses can only gallop so fast.
(That's 5,790 km from Moscow to Beijing divided by the speed of light.)
We can go faster than horses. We can't go faster than light. There are absolute limits in this universe. We're running up against them when it comes to semiconductors. Really. Science isn't this magical hat you can always pull one more rabbit out of. Eventually you simply hit a limit.
Multithreading is not as simple as setting a compiler flag, but it's also not an indivisible design foundation that must be all-or-nothing. Toady can start by making more formal interfaces to tasks so that they accept a state-of-the-world and return output (which may or may not be a copy of that data structure in a new state). At first everything would still be executed sequentially, but the "encapsulation" troubleshooting won't be commingled with timing troubleshooting. Once there are a handful of severable tasks, then he can worry about spawning threads and weaving the results back in.Quote
I agree 100%. This is why he needs to start working on it now, not just put it off until "the end." If he decides to redo the way mechanisms work, he should go ahead and redo it in a way that makes each operation that takes place in a tick be independent of the other operations in the same tick. So that some time in the future it will be possible to multithread it without another major rewrite.
If you say "I'm not going to optimize this for multithreading now, because I might rewrite it some time in the future" what you're really saying is "I will have to rewrite this in the future when I get around to multithreading."Of course, it makes sense to have a notion of how multithreading works before the start to identify the relatively low-hanging fruit.
Yeah, this is the tough part. It's a complicated thing, and trying to learn while implementing it on a major project like DF is likely to result in plenty of "learning experiences." That's why I think he should try a side project specifically so he can get experience at multithreading before trying to implement it in DF. A mentor who can coach him through the learning process would also be wonderful, but hard to come by. I'm not volunteering because I don't have enough experience in the subject to be able to help.
If I got stuck managing a software project similar to DF, something singlethreaded that really needed to be multithreaded, but had no one on the team with multithreading experience, I wouldn't enroll my programmers in multithreading courses. I'd try to hire someone with experience who can join the team and act as a guide and architect. But DF isn't being developed by a team, and that's not likely to change (and I'm not criticizing that, I respect Tarn for it). That does make it a lot harder to add new skills when they are required, though. But it doesn't mean those new skills shouldn't be acquired.
And as a programmer who is getting a little bit along in years, I can say from experience that it's easier to learn major new paradigms earlier than later. When I was in my 20s & 30s, picking up new languages and systems was easy, now that I'm in my 50s I have to work a lot harder at it. He can learn multithreading now, or he can wait 20 years and have to work a lot harder.
Or I suppose he can just hope he dies before he gets to the point where he has to deal with it, but I sure hope he's not taking that strategy. :)
The reason I said that multithreading "may or may not help" is because the updates that cause lag are already stuttered across time (growth checks every ten ticks, flow is not calculated for every tile every tick, temperature is only checked when certain events occur, etc.). Players may not see a performance boost from a multithreaded simulation, but the simulation itself ought to be smoother. Fortunately, this means that if some update cycles that take several ticks per run, it won't alter the fundamental nature of the game. Though it will probably affect edge cases like carving fortifications into warm stone.And there might be other things that could be checked less often, but aren't.
In my fort I had jobs for clothing queued up in the manager. When my cloth ran out, my announcements were filled with constant spam that dropped my FPS from 100 down to 20.
game...log... written to hard drive... as you play.
the game runs faster... with an ssd...
because...
SHIET
game...log... written to hard drive... as you play.Well I was talking about the job cancel message spam. Usually log has little to none effect against fps.
the game runs faster... with an ssd...
because...
SHIET
Spawn threads to calculate AI, pathing, fluids, etc. based on the last tickOh, I think I get what you mean. It is like putting AI in another thread other than putting Urist's AI and Tosid's AI in different threads.
Wait for each thread to finish
Update the world to match the results
rather than doing the parallel updates thing you guys think to seem would happen.
It's more that all the processing done each tick gets divided up so that one core isn't doing all the work. The way it generally works is that you have a thread "pool". Tasks are assigned to each thread, and once they finish their task they move on to the next one. All tasks done, everything syncs up, and you're done.This is what I was trying to say. Thanks.
Yes to all 4, and that multithreading also requires an entirely different perspective on how the game functions as a whole. Instead of things changing the map, you have things creating a stream of pending changes to the map that are then processed each tick. There's more to it, but generally it requires a complete restructure of thought.
More specifically:
1) Yes; more so as time increases and we reach worse and worse limits when it comes to individual processor improvements
2) Yep; it requires significant code restructuring, not just a few inserted lines here and there
3) It's more efficient than simply trying to optimize linear functions when done correctly, and will always be easier the earlier you implement
4) Yep; if Toady relies on, say, the water in the area being updated before he does action processing for combat, that'll be a big problem
So... can I just check I'm taking the right information out of this?
1. Modern computer speed gains are almost entirely dependent on multi-threading.
2. Multi-threading is different from most other optimising, which would be doing little tricks with the code done, rather than changing the way the computer runs DF.
3. Multi-threading as opposed to optimising the way the linear computing works would be more efficiently and easily implemented now as opposed to later.Multi-threading for *today's* computers of *today's code* would be more efficient done sooner. Given that we really don't know what the computers of tomorrow will bring, it's highly speculative to say that optimization for the computers of today would help for the computers around when DF is closing in on 1.0. How much effort to optimize for the new Pentium CPUs of 20+ years ago would still be valid today? Much of the code has been replaced, and the architecture looks completely different; that's at least the sort of difference we may be looking at.
4. Whether or not multi-threading could actually work really depends on how important the order of tasks completed is in DF: whether the computer can do all or any at once, or if there's a set order.
Consider:Fixed those links for you. The first is basically pointless because it lists CPUs in order of their single-thread performance, and all that tells us is that when you order CPUs by single-thread performance, it forms a gradually decreasing line. Shocking.
This chart of single-thread performance (http://www.cpubenchmark.net/singleThread.html)
This 2012 article on historical single-core performance (http://preshing.com/20120208/a-look-back-at-single-threaded-cpu-performance)
Honestly, the practical solution would to have saving sync the announcement file in RAM with the announcement file on the hard drive, because if the program crashes, you lose all progress since the last save...It's bad if you wanted to know what happened before the crash. The log prints **Loading Fortress** when you load the fortress back up, which helps weed out confusion.
GUYS THERE IS A MISCONCEPTIONIt's still much better than waiting for the first task to complete before even beginning the next. Only problem is you can't know what order things are being done in, which makes things damn near impossible to debug.
Multithreading DOES NOT afaik say, "do this until you're done, then merge it back in. We'll keep on doing stuff until you're done." It says, "everybody work on their own thing, then once every single thing is done we'll merge it all together and move on to the next tick."
Oh, I think I get what you mean. It is like putting AI in another thread other than putting Urist's AI and Tosid's AI in different threads. For example, Urist and Tosid both move towards the same tile. The result would be different if Tosid's pathing was done before Urist's and vice versa. But still, some thread may need results from another thread to complete its calculations, means it has to wait. Fluid first or pathing first may have different outcomes.It should be doable to put each creature's AI in its own thread, as long as you can handle the conflicts properly. Two dwarves try to grab the same sock? One of the tasks needs to be aborted when the threads rejoin. It's easy enough to solve thas issue, but there are all sorts of other ones you won't anticipate. That means lots of development time and headaches.
Guys there is a misconception"What is that?"
I love that phrase, it's my new sig once I get back on my comp.
Anyway! I was responding to people who were afraid that if fluid logic had its own thread, the game would go on without it, and on slower computers, water would flow slower. Multi threading is great, and I never meant my statement to disparage its utility.
This thread resulted in some excellent insights from khearn who could speak in more specific terms regarding the inevitable day of reckoning, and I hope Toady reads it at some point. Khearn's well-established ethos warded off most of the rowdy locals in denial as DEET repels bloodthirsty mosquitoes. I was not so protected, but I'm honored I was able to provide a platform for this dose of reality despite the protectionist furor it initially attracted. I've emerged unscathed.And you're being very agreeable about it.
"What is that?"And to the out-of-context thread she goes.
"It's a wall of water rushing down the passage right for us!"
"So why is it, you know, just sitting there?"
"CPU must not be able to keep up the updates. You know, more threads that cores."
"Wonder what is hogging up all the resources."
"Probably our AI. So don't stop thinking."
http://www.bay12forums.com/smf/index.php?topic=154172.0It might be the place with the most bang for the buck, but it may not be the best place to start for any number of spaghetti-code reasons that only Toady knows.
this topic was approached from every direction in that thread. im surprised how many people think you need to re-invent df for concurrency support.
the only thing that stands in the way between you and an unlimited, 100 fps dwarf fortress forever is refactored path-finding with multi-threading. path-finding is the only major bottleneck of end-game performance in df (i say this with confidence). re-writing only that single section, no matter how difficult, surely must be easier than re-writing the entire game. if anything, that would be a great start.
It's entirely possible, but it's also entirely the wrong time in the game's life to do so.On the contrary, optimisation should be done as early as possible.
all the time spent on the previous optimisation a waste
Making the game run better and faster is a job to wait until all the features are in, so there's less coding clashing going on. Once everything is in the game that Toady wants, that is when people should go back and look over it, find ways to make it faster, whatever. There's simply no point wasting time on something that could quite easily be rendered obsolete so quickly.
moving away from a tile based system early on would have helped so much but now the whole game's built around tiles.
It means objects can be located anywhere in space rather than being confined to a grid (Though it'd still be possible to keep the game looking exactly the same as it does now) which greatly simplifies physics calculations because there isn't the strange task of making physics apply to a tile-based game.moving away from a tile based system early on would have helped so much but now the whole game's built around tiles.
Could you explain why you think that's the case? I don't see why it would be such a boon. Surely it would only make things more complicated?
It's worth noting that DF's predecessor, Slaves to Armok, wasn't tile-based. It's not as if Toady hasn't tried doing things that way
It means objects can be located anywhere in space rather than being confined to a grid (Though it'd still be possible to keep the game looking exactly the same as it does now) which greatly simplifies physics calculations because there isn't the strange task of making physics apply to a tile-based game.moving away from a tile based system early on would have helped so much but now the whole game's built around tiles.
Could you explain why you think that's the case? I don't see why it would be such a boon. Surely it would only make things more complicated?
It's worth noting that DF's predecessor, Slaves to Armok, wasn't tile-based. It's not as if Toady hasn't tried doing things that way
5 4 7 5 1 2 3 4 9 1 5 4 3 4 9 7 5 1 2 3 4 3 5 7 1 2
import random
def bogosort(yourlist):
while not all(yourlist[i] <= yourlist[i+1] for i in xrange(len(yourlist)-1)):
random.shuffle(yourlist)
I'm not the most knowledgeable in this department, so have patience with my ignorance, but doesn't a tile-based system allows for Toady to "cheat" more easily than a mesh-based one? Most physical discrepancies in DF are invisible, which is beneficial not merely in terms of immersion, but also in regards to optimization. Surely not having to calculate each object's mesh (even rather primitive ones, per your proposal) frees up a fair amount of processing power for DF?It means objects can be located anywhere in space rather than being confined to a grid (Though it'd still be possible to keep the game looking exactly the same as it does now) which greatly simplifies physics calculations because there isn't the strange task of making physics apply to a tile-based game.moving away from a tile based system early on would have helped so much but now the whole game's built around tiles.
Could you explain why you think that's the case? I don't see why it would be such a boon. Surely it would only make things more complicated?
It's worth noting that DF's predecessor, Slaves to Armok, wasn't tile-based. It's not as if Toady hasn't tried doing things that way
It also means that terrain and structures can be mesh-based, which is better for simulating their destruction and can have their vertices used as pathing nodes (If a beeline to the target destination is not possible, the AI is definitely going to make a beeline towards a vertex).Agreed, but I'm not sure that's viable for something with the scale of DF. Look at the FPS-damage done merely by flooding or by smoke. Imagine the processing-strain resulting from simulating all the minuscule bits scattered by a cave-in.
It means objects can be located anywhere in space rather than being confined to a grid (Though it'd still be possible to keep the game looking exactly the same as it does now) which greatly simplifies physics calculations because there isn't the strange task of making physics apply to a tile-based game.moving away from a tile based system early on would have helped so much but now the whole game's built around tiles.
Could you explain why you think that's the case? I don't see why it would be such a boon. Surely it would only make things more complicated?
It's worth noting that DF's predecessor, Slaves to Armok, wasn't tile-based. It's not as if Toady hasn't tried doing things that way
That is completely off-topic, unless you're actually saying that easy to program implies easy to compute. If so, I want this list sorted by bogosort in the next 5 minutes:Code: [Select]5 4 7 5 1 2 3 4 9 1 5 4 3 4 9 7 5 1 2 3 4 3 5 7 1 2
Bogosort is very simple to program, heck, here's a quick implementation in python:Code: [Select]import random
def bogosort(yourlist):
while not all(yourlist[i] <= yourlist[i+1] for i in xrange(len(yourlist)-1)):
random.shuffle(yourlist)
def sort(i):
while True:
pass
def sort(i):
from time import wait
wait (9999999999)
return i.sort()
?
Yep,Fps would get bad fast with the scale of things in DF (especially after breaching hfs),im to tired to go into a bunch of details,so download Unity or UDK,and blender.Then add 100 plus low poly shapes (something with at least couple hundred triangles more then a cube[ A simple capsule shape has 960 traingles]) have them move around,then add your controls and try moving about,.That's not taking into account for fluid physics,terrain destruction,temperature,AI (which needs to be more robust to traverse 3D space then a simple grid as i recall),and all the fun stuff that now happens outside of the Fort.then there is the FPS issue,with 3d graphics low FPS can actually render the game impossible to play,or make the player feel sick,were in 2d graphics it makes the game feel more like a turn based game without making the player queasy-snip-I'm not the most knowledgeable in this department, so have patience with my ignorance, but doesn't a tile-based system allows for Toady to "cheat" more easily than a mesh-based one? Most physical discrepancies in DF are invisible, which is beneficial not merely in terms of immersion, but also in regards to optimization. Surely not having to calculate each object's mesh (even rather primitive ones, per your proposal) frees up a fair amount of processing power for DF?It also means that terrain and structures can be mesh-based, which is better for simulating their destruction and can have their vertices used as pathing nodes (If a beeline to the target destination is not possible, the AI is definitely going to make a beeline towards a vertex).Agreed, but I'm not sure that's viable for something with the scale of DF. Look at the FPS-damage done merely by flooding or by smoke. Imagine the processing-strain resulting from simulating all the minuscule bits scattered by a cave-in.
Just popping in to say that if DF has taught me anything, it's that everything is doomed.
Programming is hard.
import:pythonKnowledge
if false do find(pythonBooks) then
if boredom ~= 1 do
somethingElse
return "Sort your own damn list!"
end
end
Just popping in to say that if DF has taught me anything, it's that everything is doomed.
It is hardly relevant that there is a plan, it if the game end up not being playable.
And it is clearly in "continous development" and will of course stay so, since Toady like everyone else need an income to survive.
10/10, best necro.Just popping in to say that if DF has taught me anything, it's that everything is doomed.
sigged
It is hardly relevant that there is a plan, it if the game end up not being playable.
And it is clearly in "continous development" and will of course stay so, since Toady like everyone else need an income to survive.
It is hardly relevant that there is a plan, it if the game end up not being playable.ToadyOne and ThreeToe have said a few times that if donations dry up, they'll just go back to making games part time. In so while ToadyOne and ThreeToe apperciate the donation, it doesn't matter to them. Its not why they're making DF. And the Dev Plans havent been added to at all.
And it is clearly in "continous development" and will of course stay so, since Toady like everyone else need an income to survive.