1493
« on: December 10, 2009, 06:19:23 am »
The way I understand objects' impact on lag is as follows, and of course it is purely theoretical. But like any good theory, it is based on solid observational data. It is partially discounted by G-Flex's statement, but let's revise and correct it after it's been hypothecized:
DF, as we know, only assigns a certain amount of some jobs based on the total number of dwarves with that labor enabled. With other jobs, it places them in a que so that they are not constantly performing checks. Examples follow, and I will arbitrarily give them names in order to facilitate reference:
Type A
Hauling labors: if you have 20 dwarves with all hauling labors enabled and no others with any, you'll see up to 20 of each hauling labor in the job que. How labors are assigned and how frequently checks are performed would determine the impact these jobs have. For instance: When is the job assigned, is it a result of cycling through a dwarf's labors or list of total active labors every time a related (or unrelated) dwarf completes a job, are job objects selected based on dwarf location or destination in each instance, (etc.)? Dumping labors seem to bridge the gap between Types A and C, possibly sharing problems with either or both.
Type B
Workshop labors, which will go into the que but (supposedly) only be active and performing checks in order in each workshop, and only when it is active. The biggest drag here, and likely only a small one in sum, is that each time a dwarf takes up a workshop labor a check must be performed for the most appropriate material. Toady has not implemented a pre-task designating feature, overt or otherwise, in this one category alone for multiple obvious reasons. It's a good thing that objects are only "tasked" when a job becomes active.
Type C
Construction labors, which are qued from front to back as they're designated and only perform checks in order of their que. However, unlike workshop labors these jobs designate the material(s) as a fixed point and store that data most likely as a tag on the object. The possible sources of lag here should be self-evident once that is understood.
Type D
This can be used to describe any miscelaneous labors or labors which do not fit the existing arbitrary classifications.
So, that understood, it's easy to see that the number of dwarves, jobs and objects go hand in hand creating lag. If we can use this as a jumping-off point to provide suggestions and data to help the Adams brothers "fix" the lag, that would truly be wonderful.
Notes:
1) When viewed this way I don't think that you can come to a meaningful conclusion about the relevance of pathfinding vs object presence by doing the experiment suggested above, which was to test a few dwarves and a large number of objects in one fort, and a few objects and a large number of active dwarves in another fort.
2) While I believe the biggest drag is probably the tasking feature, it seems to be more of a stopgap preventing truly show-stopping lag than a problem. And other problems such as the one suggested by Blacken (multiple unnecessary iterations) are likely to exist concurrently.
3) Even pathfinding ties into this, as every time a dwarf begins a job and must find an object (often designating the object based on the dwarf's proximity) the game must check the distance of multiple objects. However this is done, whether as a check radiating outwards until a suitable object is found or as a check cycling through objects to find the closest, it's bound to be an obscene processing hog.
4) Other RTS and resource-dependant games preempt the task problem by counting the total number of a resource and allowing you to designate jobs out of that number, a secondary count being displayed to interface involving the number that is not currently designated to a task. Once the secondary number is exhausted, you can't designate more tasks for that resource. However, rather than designating and tagging specific objects the way DF does, materials are selected only when they are actually needed and then (if they have a physical presence, as they do in few enough games) they are moved to site (in our case, it would be by dwarven labor as now).
Toady could significantly benefit by following this procedure, but it would require rewriting DF's object processing from the ground up. This, readers, is what is called a critical flaw. And it's so imbedded that I would dare say it will not be changed before 1.0 is completed. As long as it remains, we will not be solving DF's lag problems. However I did (in my ignorance, as it turns out) think that we might have a chance of coming up with a method to alleviate lag if we put our heads together, and something to present to Toady which he might be willing to help along or do himself, as it must necessarily involve the source code.
So you see: while I might know little to nothing about programming, there's nothing wrong with my logical reasoning. Having examined the problem more carefully and with the help of enlightened individuals, it is obvious that we can't proceed. If this analysis of the labor system sticks and can be corrected and amended well enough, it should be passed on to Toady for consideration when he does finally tackle the speed issue.
