Bay 12 Games Forum

Please login or register.

Login with username, password and session length
Advanced search  

Show Posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Messages - Blacken

Pages: 1 ... 35 36 [37] 38 39 ... 53
541
Other Games / Re: Firan MUX
« on: December 09, 2009, 11:55:44 pm »
Sorry, but any game that encourages roleplayed rape doesn't interest me.

Sooooo...... no HellMOO for you then?  :P
God no. My sanity is tenuous enough as it is.

542
General Discussion / Re: The Red Box.
« on: December 09, 2009, 11:37:54 pm »
No, I don't think your seeing what I said there. Basically thats the individual tax, not corporate.
And a sole proprietorship, grossing two hundred large a year? In the States at least, what isn't paid to employees is taxed as the owner's personal income.

Do you want to encourage him not to hire more workers by taking a larger chunk of his money? Perhaps just substituting his own labor in place of an employee's--working harder to keep the same take-home pay, while not employing someone else? I would hope not. There is a delicate balance to be struck, and erring on the side of "oh, they're rich, they can afford it" sounds rather like class envy. And as someone who very much aspires to be only marginally poorer than Bill Gates someday, that upsets me.

(That's another interesting point--higher taxes tend to drive down charitable donations. Interesting, no?)

543
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 11:26:29 pm »
EDIT: damn yooooooooooou, bombcaaaaaaaaaaaaaaaaaaaaaaaaaar!

I've got twelve copies of Firefox open, each multi-threaded enough to make even IBM cry; bound to more hyper-threaded CPUs than you could count even with 256 bit primes. You goin' down!
I know you're lying because Firefox multithreads like walruses mate.

Poorly.

544
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 11:15:40 pm »
Quote
There is no "CPU limit" on it.

There is a CPU limit. Basically it is what stops a single program from multiplying its processess until it eats up your computer.

The CPU Limit Remover is done at the same time as forcing it to a Single Core. (In fact it is the whole point to doing so)
This is also counterfactual. A program can spawn processes as often as it likes, so long as it doesn't hit a limit designed to stop processes from rabbiting and making a fork bomb.

The key thing that makes you very hard to take seriously is that we're not talking about processes, but threads. All affining a single-threaded program (or, since virtually every Windows application has at least 2-3 threads, a few-threaded program) to a single core ensures that you don't suffer the overhead of cache misses, register copyovers, etc. inherent from moving between cores (and avoids other stuff like busy-waiting on in-flight threads on whatever core the process is being dumped to). There is no magical "limit," there are just the algorithmic tendencies of the scheduler.

EDIT: damn yooooooooooou, bombcaaaaaaaaaaaaaaaaaaaaaaaaaar!

545
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 11:04:21 pm »
This would be why affining it to a single CPU often sees (sometimes significant) perf improvements.
This is sort of off topic (although given that the topic is apparently irredeemably flawed I guess that's all right) but are there any good utilities out there for facilitating or automating core affinity?
http://www.robpol86.com/index.php/ImageCFG

So... finding prime numbers sounds like it would have two fundamental stumbling blocks:

1) The basic code utilized by computers, having at its most basic level only a yes/no or odd/even, in other words opposite or mathematically speaking 0/1, +/- aspect is very limiting in this instance.

I realize this is the basis of computing, so please don't jump my case. I'm just pointing out that the way our most basic computer system exists disqualifies it to a large degree for this kind of work.
Correct. Computers are bad at certain things. (Who knew?)

The other reason why forcing DF to a single core also leads to better performance is because at the same time it also removes the CPU limits for Dwarf Fortress. (So instead of 30% it can go up to 100%)
This is counterfactual. An even-spread scheduler tends to do this. There is no "CPU limit" on it.

Is "counterfactual" polite enough for you, sir?
Has anyone brought up how switching the game to multithreading wouldn't have as much of an impact as it would seem?
Depends on what you could multithread. Water physics is probably not really parallelizable given its reliance on stateful data, but pathing almost certainly is (and given certain degenerate cases, would be a pretty big chunk to actually deal with). Given an omniscient pather like the current one, it should be relatively easy. Pathing already locks important stuff so other dwarves can't get to it, it seems like it would be a simple task to parallelize give the source code.

I still wonder how much impact pathfinding actually has, though (aside from the processor-crushing bug with pet-locked doors).  People take it for granted that 200-dwarf fortress crawls because of pathfinding, but others have recently shown that just having thousands of stones on the map can cause at least as large a performance hit, and those factors tend to go hand-in-hand in long-term fortresses.

Toady's going to be working through the top 10 ESV items after this release, along with improved sieges and Adv. Mode improvements, and "Improved (Speed Up) Pathfinder" is #4 on ESV.  So we could see some pathfinder work within the next couple months, depending on what he wants to do first.  I'm guessing that'll turn into more of a general performance item, which will hopefully include the use of a code profiler so that the relative importance of pathing, fluids, population, item count, etc. will become clearer.
He's probably iterating the stones somewhere he shouldn't be, or something similar. Obviously I don't have the code, but any big project can contain lots of similar pitfalls. The problem with pathfinding is that pathfinding is a problem you can't just magic away by writing better code or not doing something--you have to pathfind, and as such it's just One Of Those Things that, even if other stuff gets cleaned up, is going to remain ugly in terms of serial processing.

546
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 10:54:21 pm »

From what I understand, right now DF's current format only enables it to run on a single processor, failing to utilize the power of the dual- or quad-core processors most of our computers currently have. A good emulator (something along the lines of DosBOX) could unlock DF's potential.

Your understanding is flawed and I don't think I saw it corrected in any of the posts, though I admit there was a lot of tl;dr because I tire of these same old arguments easily.  Any modern operating system will distribute the workload of DF among all cores of your processor.  I have a quad core processor.  When I run DF, all four of my cores spike to somewhere between 25 and 30 percent usage.

I think you're misunderstanding something.

Your computer is not running DF on all four cores at once. It's bouncing it AROUND the cores, so that 1/4 of the time it's on one core, 1/4 of the time it's on another, etc.

Big, big difference.
This would be why affining it to a single CPU often sees (sometimes significant) perf improvements.

547
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 10:44:29 pm »
The thing is that if Toady were to knock off a chunk of the code (say, pathfinding) and pass it to the community for improvements like he did with the OpenGL code, then it would have to be made roughly independent of the rest of the program by him to such a degree that a lot of the work involved in multithreading it would probably already be done.

That would only hold for an asynchronous approach, right?  I'm not sure, but I suspect the pathfinding people had a synchronous approach in mind, so that they wouldn't have worry about the map changing while a path request is handled (although I guess that isn't a huge problem from a dwarves-aren't-omniscient POV).  I suppose I should go ask them.
As you say, the omniscient approach is much more easily made asynchronous (but still has to do weird and potentially ugly updates as far as if something that is known tries to update). To parallelize an omniscient-view pather, it would probably need to be synchronized.

(I'd love to see an ignorant pather, though. Fortress defense? "Don't let them see you!")

Has anyone brought up how switching the game to multithreading wouldn't have as much of an impact as it would seem?
Depends on what you could multithread. Water physics is probably not really parallelizable given its reliance on stateful data, but pathing almost certainly is (and given certain degenerate cases, would be a pretty big chunk to actually deal with). Given an omniscient pather like the current one, it should be relatively easy. Pathing already locks important stuff so other dwarves can't get to it, it seems like it would be a simple task to parallelize give the source code.
You're asking the wrong question, because if not for all those roadblocks--if Toady was interested in cooperating to the degree necessary to do anything significant--it could be done just by modifying the source code. There! Done! End of problem! But in the real world, the source code is not available [...]

It's not impossible that other sections of the source will get opened up, though.  The way people got the I/O code opened up was by demonstrating that they'd be able to make good use of it.
Certainly true, but this is pretty core functionality. I have a hard time seeing such a case evolving, in such a way that it can be modularized so Toady doesn't have to give away DF-code to get it to work.

548
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 10:39:06 pm »
Ah geez man you really, really need to chill out. No one is mad at you for shooting down ideas. Like I said: YOUR CONTRIBUTION IS CONSIDERABLE. But your tone is constantly offensive and you constantly go off-topic over your personal crusade.

Let's look at other ways to do it. I will present the idea to Toady of we have something useful, if we can't come up with anything then that's fine. Is it impossible to outsource processes much the way you outsource labor? When DF normally queries for closest resources every time the dwarves perform a job, causing massive processor delays, can't we have it query an updating separate database? We would basically be rewriting a small part of the program separately without the need to change the very thing you keep saying it is impossible to change.

Toady's cooperation is necessary. Rather than assuming we can't get that cooperation, let's just consider whether it is possible that we might be able to do something if we can. It's not going to kill us to consider the hypotheticals here in this thread.

For the last time, if you dont' want to play ball, then leave. This isn't the place for your crusade against your invisible enemies. If you want to continue to contribute without the sarcasm and off-topic defensiveness, then it would be nice to have your knowledge.

Damn you're stubborn about it.
You seem to be placing some kind of intent in my actions that is not there. I am pointing out things that are, to someone with experience and skills in this area, obvious. You are getting upset at this. That is something for you to work out, sir, not me. There is no crusade. There is me telling you what works, and me telling you what doesn't work, and you stuffing your fingers in your ears and going "LA LA LA BUT WHAT ABOUT IF WE HAD THIS STUFF THAT DOESN'T EXIST?". I am making a point of being about as polite as is possible here when I say that yeah, you could do it if you have all this stuff you don't actually have. But that doesn't make for any better discussion than "no, it cannot be done given the constraints of reality," now, does it?

Given cooperation from Toady on whatever necessary level, is it even possible to create an interface/utility that would do what I've suggested?
See what I mean? "If not for all the roadblocks in the real world, could something be done?"

You're asking the wrong question, because if not for all those roadblocks--if Toady was interested in cooperating to the degree necessary to do anything significant--it could be done just by modifying the source code. There! Done! End of problem! But in the real world, the source code is not available and as such modifying object code to parallelize application functionality (you can use whatever phrasing you want for it, it boils down to this) is a nontrivial task.



Basically, you're asking for the source, which Toady has repeatedly said he's not releasing. That's more the problem than any understanding of modern CPUs.

Though there is one way around that. If you can create an algorithm/equasion that can find Prime Numbers then you won't need the Source Code.
How does this relate? Why is prime-finding relevant to the discussion?

549
Life Advice / Re: I forgot every thing...Now I want to program again...
« on: December 09, 2009, 10:30:54 pm »
Those are an excellent place to start.

550
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 10:05:22 pm »
You don't get it, dude. What you're asking is like asking an architect how to make a thirty-mile bridge without any supports. Physics, whether on the bridge-building or the processor-architecture scale, does not work the way you think it does. You've been getting a "Whaaaaaaaaaaat?" reaction from people in-the-know (I'm not the only one in this conversation who's saying these things, I hope you've noticed) because it is not feasible.

If you have the money in your pocket to hire a dozen engineers from VMware to automatically parallelize object code, then by all means (just don't expect huge results)--but you don't, and this topic doesn't do anything. That's what I've been saying, and that's what other people have been saying, and that's what you don't want to hear. You keep saying you don't know anything about it, and people who do know about it are telling you it's not doable, and you get huffy and get mad! What the hell gives, man?

The way to solve this problem is to let Toady do it when he's ready. He has the code. It's his ballgame. Not yours, not mine. His.

I'm sorry that boundless optimism doesn't change physics and math. I really am. But getting pissed at me for pointing out the obvious is the wrong place to look.

551
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 09:59:26 pm »
It would be nice to see some reasonable and well-thought, well-supported, well-written responses. I asked for help and information...

So, Footkerchief did this. And subsequently increased my respect for him all the more. Thanks for the useful info, lookin it over now.
'Cause pointing out that OOE requires independent data sets (post #4, but that's OK, you are clearly reading what you think people are saying instead of what they are saying) and as such makes DF less than friendly to what you propose--oh yeah, that's totally not reasonable, well-thought, and well-supported. Except that it is. <_< >_> <_< >_> But recognizing that wouldn't let you play passive-aggressive games, so I guess we can't have that.

552
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 09:53:38 pm »
Asking a program to look at the compiled program (or even the source code) and do this for you relies on the same lack of understanding as asking someone "Why won't the computer do what I want?" - the computer never does what you want, only what you tell it to. Even the different parts of DF's code that could be run in parallel currently are not and will interface in ways that no program can magically figure out for you.

I think this is an oversimplification, if I'm remembering my Op Sys class correctly, which I'm not at all sure of.
Actually, he's pretty close. It is extremely difficult to figure out programmatically what code is serial (requires the results of previous code) and what code can be safely parallelized. A person can look at it and go "this, right here--this can be parallelized easily", but the heuristics to make a computer do so...not so much. There are techniques to do it, but (there be a pattern here!) they invariably require the source code.

(A pet interest of mine is compiler design, so I've been exposed to an unhealthy amount about this stuff. :) )

Splitting the processes DF does (if that is even necessary) might require access to code that has not previously been available to players, and we may need Toady's help. If that is so, then let's go there with it.

The point of this is to enable DF to utilize our computers' true potential without the need to rewrite the whole program. A program-specific emulator. Not a universal tool.

It seems obvious to me that I have already stated the possible need for access to the source code. You're right, I don't know a lot about what I am proposing. From what you say, access or at the very least close cooperation would be absolutely necessary. So be it, if this is in any way feasible with the source code then let's ask for it. If it's not possible either way then sure I'll forget it. But let's not dismiss it out of hand based on things that may or may not be the case (Toady/Threetoe never agreeing to such a thing).

I was actually looking for something to interpret the processes and feed DF information that it would normally need to manually cycle through. Not work the whole program on a multi-thread base. But most likely this would require non-existant output and non-existant input options. Outsourcing just the stocks database, causing DF to feed the info to a separate process and retrieve it from said process would cut the load tremendously, would it not?

DF can sequentially request info from the program rather than calculating it itself as it does now. The "emulator" would just be an interface. I do realize now that it might be the wrong term for such a program, but I dont' know a better one. The emulator would cause DF to believe xy was happening, when in fact z was performing the functions of xy separately. The same could be done with pathing, but that would be insanely problematic and likely to produce a headache just contemplating, but those are the two major drags besides job priority checks. Allowing us to set our own job priorities for dwarves and groups would probably cut lag in large forts by a ton as well. This problem could be circumvented by outsourcing the Jobs menu and feature to a separate program.

As I'm looking at it now, it definitely requires full cooperation from Toady. One thing it does not require is a rewrite of the program.
Really. Where are the credentials you're bringing to the table that say you know this? Do you tell your mechanic that no, he's wrong, just throw an extra set of wheels on it and it'll go faster? Because that's pretty much what you're doing here.

I've built monolithic single-threaded applications before and, uh, none of it's even remotely done this way, ever. The information-service architecture you're trying to suggest would almost certainly require even more restructuring than just manually parallelizing what's already there! Don't conflate this with stuff like the DF pathing library--that is being developed in a vacuum in hopes that Toady can later on integrate it himself (and they're doing fine work, I love reading that thread--I haven't posted there because algorithm grinding isn't my specialty, I don't have the head for it).

Quote
...No. Just...no

This isn't TVtropes, around here were support our ideas.
Really? You do? You, specifically, do? I couldn't tell from your half-redigested repetition of something that wasn't even relevant!

I'm not derailing this thread to indulge you and I won't reply to any silly attempts by you to rationalize what you said, but let's just go over this quickly. Vista "failed"--if an operating system that sold eight bajillion copies could be said to "fail"--because they put "ready for Vista" stickers on computers everyone knew weren't ready for it and, in doing so, poisoned the brand in the mind of Joe Public. It had nothing to do with diminishing performance returns from processor research. This is not the thread to debate this, so kindly don't. Just stop. Go make a "I think Vista failed because processors aren't getting faster no more!" thread if you really feel the need. Make sure you figure out why Win7 is a ridiculous success at the same time when it's pretty much Vista SP3, though.

And yes, I would say that Blacken's impatient tone of superiority is irritating. It would be nice to see some reasonable and well-thought, well-supported, well-written responses.
Then ask reasonable and well-thought, well-supported, well-written questions. You aren't dude. I can't be blamed for calling silliness what it is.

Quote
I asked for help and information, and got a smart-ass, "I'm tired of dealing with ignorant people" response. About half of what was said didn't even apply to my comments, but seemed more of a generic response to past irritations Blacken has experienced when dealing with irrational and ignorant people who demand the impossible.
With all due respect, I don't think you have the knowledge to determine what is relevant to your comments.

Look at everything that's been posted in this thread. I'm sorry that using blunt sentences isn't something you like, but everything I've said has been true. You are asking for what essentially amounts to the impossible. It sucks, everybody gets egg on their face once in a while--I'm sorry that your in-theory-great idea is practically not possible, but it isn't.

If we want reasonable and well-thought, well-supported, well-written responses: you start. Find some sources that suggest what you're talking about is even possible. Do some research before insisting that I'm wrong--you're asserting a positive, the burden of proof is on you.

I'll wait.

553
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 09:36:13 pm »
Here's a paper that touches on some of the techniques being explored for parallelizing single-threaded applications.  It's still a very active area of research, so it's not out of the question that DF will someday make good use of multiple cores.  But it won't be as simple as what you proposed.
Important to keep in mind regarding that research (which is interesting, I'd never seen the full paper before--thanks for the link!):

-It doesn't use current technology. I'm sure they could build a simplified prototype, but it wouldn't be fast enough for general purpose usage.

-It wouldn't work with x86 at all, because of some quirks in the architecture. I could see it as a POWER retrofit, but it'd be one of the biggest (and thus most expensive) changes made to date in x86, and millions of computers would still lack it.

-It still requires programs to be structured in a very specific way. There isn't a "free" speed increase. Code that requires dependencies will still require previous code to be finished first, and so for something like Dwarf Fortress you gain little.


I'm sure that DF will someday make good use of multiple cores--when Toady gets around to it. This isn't how it'll happen.

Ignoring "Single Coring" the game (Which apperantly works VERY well) the rates in which computers improve may destroy the lag itself.

Though I shouldn't bet on it since computers are finally slowing down. Hense one of the major reasons for the failure of the Vista.
...No. Just...no.

Quote
Note: By Single Coring, I am refering to some weird people who devoted an entire core to Dwarf Fortress
On a properly scheduled machine, this shouldn't matter. This is an OS limitation, and one that is progressively getting better (albeit slowly).

554
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 09:23:30 pm »
G-Flex is nicer than I am.

555
DF General Discussion / Re: We can end the lag forever!
« on: December 09, 2009, 09:17:04 pm »
Or possibly you know so much about it that you misunderstand what he's saying from assuming that he's even using the correct terms for what he's trying to say, due to him knowing little about it.

You're the only programmer whose responded thus far, I think it'd help to take a step back and look at this closely.  Even if it turns out he's wrong and you're right.
Sigh. Welp, let's look at what he's asking and see if it's ambiguous.

From what I understand, right now DF's current format only enables it to run on a single processor, failing to utilize the power of the dual- or quad-core processors most of our computers currently have. A good emulator (something along the lines of DosBOX) could unlock DF's potential.

This is clearly unambiguous in its intent. He doesn't know the solution (and neither does anyone else, that's the point), but his intent is clear. He is asking for a way to apply multiple instruction processors to a single (or near-single) instruction set, without touching the source code. He is essentially asking for a pet unicorn. Not even a pet pony--an out-and-out pet unicorn. There are companies spending millions of dollars on how to effectively predict code in this way. It is a decidedly nontrivial task, if it can even be done in a performant way. That's why everyone who needs it just rewrites the code.

At the risk of sounding like a jackass--it certainly would not be the first time--unless Toady wanted to give out the source code (or just do it himself) there is nothing to look at here, and it is rather silly for you to say "it'd help to take a step back" when it isn't technically feasible no matter how far you step back.

The code will eventually be rewritten to support them, I'm sure. Until then, get faster-clocked processors.

Pages: 1 ... 35 36 [37] 38 39 ... 53