Hauling Improvements
- Being able to haul multiple small objects
- Having multiple dwarves involved with item hauling for a job
- Being able to move multiple objects with roughly the same destination at once
- Wheelbarrows to haul more objects than can be carried
- Minecarts
- Wooden, stone-carved and metal tracks
- Can be filled like stockpiles and moved between destinations
- Work animals to tow carts and haul objects
We have route from A to B. [for example single sock moved from A [battlefield] to B [trade depot]]It is not "the FPS drops by half", it is "the FPS drops to 1/n of old FPS" where n is quantify of dwarves. (or even "the FPS drops to 1/(n*n) of old FPS")
I'm going to have to ask you to explain where you came up with those numbers. The only way that sort of complexity would make sense is if the entire pathfinding routine would have to take place per every other pathfinding creature in the entire map, which makes absolutely no sense.
We want to detect routes likes A->X->Y->B, without huge additional cost.
To find routes like this it is required to make pathfinding A->B and A->X->Y->B for every possible pair of X->Y. [sock moved from X1 [battlefield] to Y1 [trade depot], sock moved from X2 [battlefield] to Y2 [trade depot], seed moved from X3 table to Y3 stockpile etc]
So with n jobs, single planned results in pathfinder running n times instead of one check.
Result: pathfinding running n^2 times for n jobs, instead of n times.
[I assumed that number of jobs is liner vs number of dwarves, but "where n is quantify of dwarves" can be changed to "where n is quantify of jobs"]
Coins have their coin batch index (year for dwarves) and the material, but there's also ownership, contaminants, temperature, damage, age, storage information, maker id, flags based on if your fort owns/made it, melt/dump/forbid/hide designations, job information... and probably more. Some of it can be scrapped or otherwise dealt with, but it's not a triviality if you want to keep track of things. Even if it checks all of these things and consolidates equal stacks (which would be rare -- or with age+maker id, virtually impossible), seeing it happen in some cases and not others with identically named objects would lead to false bug reports, I think. Probably more profitable is addressing the main concerns in other ways -- getting dwarves to grab all the bolt stacks they need when they go to get a one bolt stack or making coins work in a sensible fashion don't really depend on stacking so much. Increased CPU drag from broken stacks is probably the main thing that can't be dodged without some direct handling, and I'm leaning toward not suppressing the information by using some extra objects (could use the same ones as the interface would I guess, though there are all sorts of ways to think about it).
Can't you just create a new "stack" container that would be created by stacking two objects of the same types (bolts, coins, ...) and destroyed when it contains only one object?
This is roughly what I meant by "extra objects", except for the "just" part. The main point is that it's not only an interface problem. If items are potentially contained in fake stack items, all the job handling needs to be revised to support that. It's a bit less of a pain if you restrict to only coins or bolts say, but there are still things to be considered. Take bolts for example. Your guy wants to equip some. First it needs to recognize the container as a valid ammo object. Depending on exactly what you choose to stack, some members might not be as eligible as others -- what if some in the stack were forbidden, or tasked for a job, prior to stack formation, for example? Those cases would need to be disallowed or handled. Then the equipment checker needs to realize he's got the bolts when he places the container in his quiver. Not so bad, possibly handled by current code, but something to check. Now he wants to fire a bolt -- shooting needs to be rewritten to support bolts in the fake objects. Sort of a piece of the quiver code already, maybe nothing needs to be touched at all, but takes some checking and testing. There are side issues like valuations and stocks screen stuff, which might be handled already, and other thises and thats (environmental functions would need to be taught about fake containers -- rain for example should get the bolts wet, but currently there's no mechanism for jumping through a container on the ground to get the inside stuff wet, I think, since containers are generally real -- not hard, but yet another issue), as well as any applicable storage jobs, which run into some of the first problems I mentioned. As you get to other objects, like food, you inherit more difficulties (vermin need to see and realize that they can eat food in stacks for free, etc. etc.). As far as I can tell, the problem is not super difficult, and a lot of the issues just need a nudge one way or the other, or no nudge at all, but there are a Lot of issues that turn this into a tedious and long haul. There's generally more to do and check than people expect.
Well, most of the detail could be exported to a single pointer on the actual coin, and if you rewrite stacks to be pointer counts, that helps. (Exported detail is everything on the coin when it's made, date manufacturer, material, etc)
The problem is contaminants...
Results of a poll from '08
What is your most desired suggestion out of the collected top 10?
show levers / name levers / blink connected 35 (6.4%)
excavate - mine without stone production 32 (5.9%)
stop dwarven entrance dance when ordered inside 73 (13.4%)
forbidden area designation 20 (3.7%)more raw files 50 (9.2%)
job priorities 55 (10.1%)
combine stacks / better stack handling 65 (11.9%)
more underground diversity 93 (17%)
improved sieges 48 (8.8%)
framerate improvements 75 (13.7%)
two things to notice: Most of this is fixed or unrecognizably different, and the big vote from 3 years ago had about 450 people involved.
Responding directly to the OP: your stacking ideas won't work. If there's a max size for stacks, it has to be item-dependent for small items like coins and bolts. Items spilling into adjacent tiles is an interesting idea, but there needs to be some distinction between deliberately stacked items (don't spill) and carelessly dropped items (do spill).
Restacking also has some complexities that you might not have considered -- Toady had some posts about it a while back.Quotecombine stacks / better stack handling
-- this has been a problem since before the first public release, and the two ideas were basically to actually do it and to fake doing it via interface presentation of item lists etc. Actually combining items would virtually require losing information, or you'd almost never do it (most items carry a lot of info), so I've grown to dislike this option. The other option is to make the player think its happening more or less, and that's fairly easy in the interface lists. The problem comes with jobs -- dwarves need to learn how to use multiple item stacks and consider it in all of their thoughts, which makes it sort of a pervasive rewrite. As with job priorities, just something I haven't jumped into, but definitely there to be done.QuoteCoins have their coin batch index (year for dwarves) and the material, but there's also ownership, contaminants, temperature, damage, age, storage information, maker id, flags based on if your fort owns/made it, melt/dump/forbid/hide designations, job information... and probably more. Some of it can be scrapped or otherwise dealt with, but it's not a triviality if you want to keep track of things. Even if it checks all of these things and consolidates equal stacks (which would be rare -- or with age+maker id, virtually impossible), seeing it happen in some cases and not others with identically named objects would lead to false bug reports, I think. Probably more profitable is addressing the main concerns in other ways -- getting dwarves to grab all the bolt stacks they need when they go to get a one bolt stack or making coins work in a sensible fashion don't really depend on stacking so much. Increased CPU drag from broken stacks is probably the main thing that can't be dodged without some direct handling, and I'm leaning toward not suppressing the information by using some extra objects (could use the same ones as the interface would I guess, though there are all sorts of ways to think about it).QuoteThis is roughly what I meant by "extra objects", except for the "just" part. The main point is that it's not only an interface problem. If items are potentially contained in fake stack items, all the job handling needs to be revised to support that. It's a bit less of a pain if you restrict to only coins or bolts say, but there are still things to be considered. Take bolts for example. Your guy wants to equip some. First it needs to recognize the container as a valid ammo object. Depending on exactly what you choose to stack, some members might not be as eligible as others -- what if some in the stack were forbidden, or tasked for a job, prior to stack formation, for example? Those cases would need to be disallowed or handled. Then the equipment checker needs to realize he's got the bolts when he places the container in his quiver. Not so bad, possibly handled by current code, but something to check. Now he wants to fire a bolt -- shooting needs to be rewritten to support bolts in the fake objects. Sort of a piece of the quiver code already, maybe nothing needs to be touched at all, but takes some checking and testing. There are side issues like valuations and stocks screen stuff, which might be handled already, and other thises and thats (environmental functions would need to be taught about fake containers -- rain for example should get the bolts wet, but currently there's no mechanism for jumping through a container on the ground to get the inside stuff wet, I think, since containers are generally real -- not hard, but yet another issue), as well as any applicable storage jobs, which run into some of the first problems I mentioned. As you get to other objects, like food, you inherit more difficulties (vermin need to see and realize that they can eat food in stacks for free, etc. etc.). As far as I can tell, the problem is not super difficult, and a lot of the issues just need a nudge one way or the other, or no nudge at all, but there are a Lot of issues that turn this into a tedious and long haul. There's generally more to do and check than people expect.
Holy wall of text
Part of the problem is all the old forum posts didn't copy over. A lot of this was fleshed out. I'm pretty sure Toady knows what he wants to do with this, at least in general, it's just that it doesn't meet the effort/reward ratio yet. I mean yeah, it's a pain in the ass, but it works, and 'fixing' it isn't going to be an order of magnitude improvement, but it'd take a lot of really in depth work to produce something that's liable to be buggy, and doesn't increase the feature set of the game.
Coins have their coin batch index (year for dwarves) and the material, but there's also ownership, contaminants, temperature, damage, age, storage information, maker id, flags based on if your fort owns/made it, melt/dump/forbid/hide designations, job information.
awesome... my mouse sucks. I did say what's in the quotes, put the poll numbers in, and wiped the original. Still meant it.
I'd just suggest that you edit the post to be more bullet pointy, and put all the text in the second post.
Topic for discussion:QuoteCoins have their coin batch index (year for dwarves) and the material, but there's also ownership, contaminants, temperature, damage, age, storage information, maker id, flags based on if your fort owns/made it, melt/dump/forbid/hide designations, job information.
What would you say can be offloaded, converted to the stack info, or dumped. Every one of those has to be one of the three.
you interpreted correctly.
Contaminents is the interesting one. One arrow has the dried poison blood of a basilisk on it. Stack with similar arrows and they all do? How does that play with poison, or sympathetic magic, or evidence?
Temperature: Everything has it's own temperature. Everything. EVERY THING.
Anywho, what about an object with an innate temperature (magic arrow). What if you put a fire arrow and an ice arrow in the same stack? What if you pull the shuriken out of the lava a stick it in the pile. Should it instantly cool off? Should everything instantly average? Should they all get burning hot? Each item also has a specific heat, so the math for avg isn't just sigma temp / items...
Spoiler (click to show/hide)
for starters, it means your fantasy education is lacking
You're pretending that you can't look at the map when trying to determine if something is "5 squares away."It is L*x, where L is length of path and x is maximum distance to find job (it is in fact BFS with found path as starting points with limited path length). It also will annihilate FPS.
Lets say this is our map. * are the path, T is a Task Object.
.*......
.*......
.*..#...
.*..#T..
.*..#...
.*..#...
.*......
.*......
If we count diagonals as 1 space, then T is 5 away from the path (go around the wall on the top). So we want this to be included, but we don't want to run an A* check on it (and by definition, every other task on the map).
What can we do?
Well, we do this: find every square adjacent to the path, and every square adjacent to that, etc. until we have 5 deep and note every task that falls inside that area. Essentially we're doing a limited distance flood-fill.
1*12345.
1*12345.
1*12#45.
1*12#5..
1*12#5..
1*12#45.
1*12345.
1*12345.
Voila. And if the object is in another room not direclty accessible, then the item isn't categorized:
1*12345.
1*12345.
1*12####
1*12#T..
1*12#...
1*12####
1*12345.
1*12345.
Now you have a list of every task that is inside some boundary distance of the dwarf's desired path. Some tasks have a starting point and an ending point ("fetch quests" and for those you need to check the starting point and the ending point with the requirement that the start location is closer to the dwarf's starting location (and vice versa) which you can do through normal distance calculations (no need to do a path-find distance, as we already know that both points are within 5 of the existing path).
Alternately, it can actually look at the physical map around it, and search every tile for objects, then search those objects for jobs that take that object someplace, and then test to see if they are going the same place, and THEN search to see if they are actually "nearby" and don't have an obstruction between them. This causes problems to start with, since you have to search a lot of tiles for items to even start with, and then, if you have something horribly complex like a dump zone/quantum stockpile with 5,000 items in one of those tiles. Preventing quantum stockpiles or putting limits on the number of items that can physically enter a tile (meaning dumping an item into a pit which already has 100 items in it will simply create a "solid wall" that prevents more items from entering it) will put some sanity check into that, but it's still going to be a problem. This has a complexity of R^3 (Op+Oq+Od), where R is the radius of tiles around the object that you consider "nearby" (Alternately, you can only search for "nearby" as being on the same z-level - this means that objects on different portions of a ramp will never be tasked with one another, but it can cut out a lot of false positives where you assume that anything on another z-level is probably going to be very difficult to path a direct line to. This means you only need R^2 complexity.) while Op are the number of items present in a given tile, Oq are the number of items with hauling jobs queued, and Od are objects with nearby enough destinations.Complexity is rather: n+m*pf, where:
Complexity is rather: n+m*pf, where:
n: number of items nearby start
m: number of items nearby start with nearby end
pf:cost of pathfinding from end to additional possible end
Problem is that worst case is sth like this:
{pile of dumped objects and sock}++++++++++++++corridor 1+++++++++++++++++++{socks stockpile}
+
++++++++++++++++++++++++++++++++++++++++corridor 2+++++++++++++++++++++{quantum dump}
In result every single dump-marked object must be processed before picking up sock to check is it possible to consolidate them in single job (with result: bad idea, pick only sock).
It would result in effects similar with ghosts (100 FPS - 0 FPS - 100 FPS). So without limiting number of objects on tile it is completely impossible.
@10 jobs for one dead dwarf. Better solution is to replace 2 shoes, 2 socks etc with single object "typical civilian clothing"
One solution is searching the entire jobs list, and then looking at the "starting from" location to find the "nearby" ones. This causes problems when the jobs list becomes longer, because it means that every new job will have to make a search of every single job already put on the job queue to search its starting location. This basically has a complexity of O^2+O where O is the number of jobs being put into the jobs queue. (Subgeometric complexity growth with number of tasks.)
Item myitem
for each emptyslot in stockpile.emptyslots
{
myitem = getUnstockpiledItem(stockpile.acceptableItems)
CreateJob(Haul,myitem,emptyslot)
}
You would havebulkhaulingjob[] myjobs
item myitem
for each emptyslot in stockpile.emptyslots
{
myitem = getUnstockpiledItem(stockpile.acceptableItems)
for each job in myjobs
{
if (job.items[0].location.getXYDist(myitem.location) < 2 && job.items < MAXBUNDLEDITEMS)
{
job.addHaul(myitem,emptyslot)
break
}
elseif (job == myjobs.lastjob)
{
myjobs.add(CreateJob(BulkHaul,myitem,emptyslot))
break
}
}
}
The code for the job itself would be.goTo(items[0].location)
for each item in items
{
if item.location.getXYDist(dwarf.location) < 2
{
pickup(item)
}
else
{
dwarf.complainAboutItemBeingMisplacedAndGoGetABeerOrSomething()
}
}
for each destination in destinations
{
goTo(destination)
drop(items.firstitem)
}
- As a reminder, dwarves have a limited carrying capacity. They can only carry so much goblin shoes, depending on their force and containers. This will naturally limit the items we have to consider.
- Grouping jobs: instead of striving for maximum time and resource efficiency (every free dwarf must haul if there's an item to be hauled), strive for organizational efficiency: all hauling jobs for a group of items are grouped, and assigned to a dwarf. That dwarf goes over there, takes what he can carry (preferring items for the same stockpile(s)), goes to stockpile it (visiting several stockpiles in a row and dropping of items, if necessary), and returns to pick up more. When he goes to sleep somebody else can be assigned to continue the group of jobs.
- Ad hoc: a dwarf goes to an item to be hauled, picks it up and checks for a similar nearby item (using a limited floodfill like Draco suggested; the distance of the floodfill should be larger if the trip to the stockpile is longer (people think twice whether they have everything before leaving the store and driving back home, but if you're taking food out of the pantry it's not important if you have to return one more time)), if he finds one he picks it up and checks for more until his backpack is full and goes to the stockpile, if he doesn't he goes to the stockpile.
- Central depot stockpiles: for large quantities of diverse items (eg. battlefield remains) it makes sense to haul everything to a central location (inside your fortress, presumably) and sort it out later.
The way it could work is to flag a stockpile as central/depot: this stockpile would then get priority as a destination for non-stockpiled items. However, it would also be emptied ASAP by moving stuff to normal stockpiles. That way you could set up a central stockpile for everything to start, and later diversify with other central stockpiles when you don't want corpses to be brought into your central hall, or set up the ore processing facilities elsewhere.
I don't think it would be near that complex. You don't need to check the existing jobs at all. The empty slots in the stockpile generate the jobs after all. When the stockpile goes through the item list grabbing items to fill its slots it could create bundled item gather jobs instead of tons of single hauling jobs.
With MAXBUNDLEDITEMS at 2 this would double hauling efficiency for 90% of the jobs that occur. Workshops are always full of items most of which will end up in the same place. Crafting, butchering, brewing, plant processing, elf murdering, ect... Forging usually needs metal and fuel both of which come from the bar stockpile, and could very well be in the same bin.
It considers only directly adjacent tiles to be acceptable for bundling which removes the need for additional path-finding at the pickup end. So long as the items are close enough you just teleport them to the dwarf's inventory. They don't even have to move.
Most of the crap my dwarves swarm to pick up and carry back to the hoard can fit in bins, and the dwarves can carry bins full of stuff. Why, oh why, can't they just bring a bin with them? It could be automated - when 10+ binnable things are a certain distance from their target stockpile and in the same region, a dwarf on their way to haul them can pick up an empty bin and put the stuff in it like they would put bolts in a quiver. Or if they HAVE to put down a bin before putting something in it, maybe a spot could be designated just for bins - a temporary "anything" stockpile that accepts only items on adjacent tiles. Picking things up one at a time and putting it into a nearby bin wouldn't be as bad as having to go all the way to hell and back for every sock, helm, and amulet.
Indeed, jobs on the way shouldn't be considered, because that inevitably requires computing "the way", i.e. pathfinding, adapting the path etc. What I meant was grouping jobs that are close together.- Grouping jobs: instead of striving for maximum time and resource efficiency (every free dwarf must haul if there's an item to be hauled), strive for organizational efficiency: all hauling jobs for a group of items are grouped, and assigned to a dwarf. That dwarf goes over there, takes what he can carry (preferring items for the same stockpile(s)), goes to stockpile it (visiting several stockpiles in a row and dropping of items, if necessary), and returns to pick up more. When he goes to sleep somebody else can be assigned to continue the group of jobs.
I was going to try to address Draco's post with this, but... I'm not sure if looking for things "on the way" is worth the pathfinding cost, actually. Basically, we could just go by absolute job starting and ending point definitions of "nearby". It means that maybe the haulers will go to a distant site, pick up half a wheelbarrow's worth of socks, and roll right over a sock on the way to the sock stockpile, but that at least prevents additional limited-range flood pathfinding along an already pathfound route.
Basically, I'd think it would come down to how likely it is that you would save trips and jobs and extra full pathfinding tasks by doing so compared to the costs of doing so. You'd have to test this with plenty of live forts to really get a sense of the average number of jobs you'd save for doing this.
If a goblin dies, and the stockpile for metal armor are clearly in another part of the map than the stockpile for fabric and leather clothing, then there should probably be completely different tasks for dwarves to jump for. If there are a large enough number of items that the dwarf cannot claim all the hauling jobs for that pile of crap, then the jobs that he/she cannot claim should still be left up on the jobs queue for other haulers to try to take.
a decent path between which corpses to visitThat's a travelling salesman problem. Might be tricky. What I meant was floodfilling until a hit, then picking that up, then floodfilling again from that position.
We also need to have a way to use wheelbarrows, however, which would presumably be able to carry more items at once, so that you could stuff all the crap a goblin drops into the wheelbarrow if it's all going to the same stockpile, anyway.
NW, check out the job priorities thread in my sig. It's got a lot of job info in it.
That's a travelling salesman problem. Might be tricky. What I meant was floodfilling until a hit, then picking that up, then floodfilling again from that position.
After his load is full, the dwarf is simply emptying his load. So he just needs to pathfind to the nearest stockpile, drop off what he can, and then he'll look for the closest stockpile out of all the stockpiles he needs to visit. Repeat until empty. Trying to find the ideal route between the stockpiles is not worth it, IMO. I don't care if the haulers are a bit dumb. They're normally safely inside the fortress at that point anyway.
Just add a check to my pseudocode in the job section. If the number of items in the job is higher than, lets say, 5 the dwarves will grab a wheelbarrow or empty bin. If they can't find one then they can grab what they can, and maybe split the bulk haul job so another dwarf can help.
Or wheelbarrows could be creatures that can't move on their own. They have their own AI that looks for clusters of hauling tasks when they are idle, and claims them. Then they create a job for a dwarf to come move them to their destination and load them. If you don't have wheelbarrows then there would not be any extra checks for wheelbarrow usage. This would also allow for other creatures to be haulers with dwarven assistance and proper training. Train a bunch of hauling kangaroos, fill their pouches with items, and drag them around the fort!
Edit: After testing with stone stockpiles it seems that the stockpiles as a whole don't create the jobs because when I created 3 stone stockpiles instead of claiming items in the order of their creation they claimed them in a jumbled order. I think therefore that the individual stockpile tiles claim the items. More testing could determine the exact order, but I have to go to bed in 30 minutes so I won't be doing it anytime soon.
It is L*x, where L is length of path and x is maximum distance to find job (it is in fact BFS with found path as starting points with limited path length). It also will annihilate FPS.
It's just a cost benefit thing. Anticipated cost >> Anticipated benefit.It is L*x, where L is length of path and x is maximum distance to find job (it is in fact BFS with found path as starting points with limited path length). It also will annihilate FPS.
So rather than countering the idea with a better idea all you did is tell me why it wouldn't be efficient enough.
And how efficient does the algorithm need to be in order to be good enough for you?
So rather than countering the idea with a better idea all you did is tell me why it wouldn't be efficient enough.
And how efficient does the algorithm need to be in order to be good enough for you?
It's just a cost benefit thing. Anticipated cost >> Anticipated benefit.
From my perspective, any change would need to pay for itself. Since haulers are effectively free, doubling up would need to be at least as efficient (processor wise) as taking two trips. Right now, I don't think it is (I mean sure, when they DO take two things, it's more efficient, but the overhead is added to ALL hauling jobs).
It is L*x, where L is length of path and x is maximum distance to find job (it is in fact BFS with found path as starting points with limited path length). It also will annihilate FPS.So rather than countering the idea with a better idea all you did is tell me why it wouldn't be efficient enough.
And how efficient does the algorithm need to be in order to be good enough for you?
I see that 'on your way' may be nice but I have no idea how to make efficient algorithm (n log n or better) to solve it.
If we do it that way, we can wind up "following a trail of breadcrumbs" away from a task we should be lumping in as a nearby task. If there's a single item on the floor that by chance creates the first task on the job board, then there is another single item a single tile away to the north, but a pile of goods that needs to go to the stockpile two tiles to the south, and then when the flood search takes place from the single item that was to the north, it finds another item 2 more tiles to the north of that, then another item 4 more tiles to the north of that, then another 8 tiles to the north of that, then you've gone out of detection range for the whole pile of items to the south.Hey, I like that. That's how children get lost in forests and stories start. I seriously think there is room for dumbness here. We want to save the player's time by improving processor usage, not save the hauler's time by improving his GPS.
I think wheelbarrows are still a mess, though. Should I use one? Where's the closest one? How far do I actually have to go? Do I pass close to one?
It is L*x, where L is length of path and x is maximum distance to find job (it is in fact BFS with found path as starting points with limited path length). It also will annihilate FPS.So rather than countering the idea with a better idea all you did is tell me why it wouldn't be efficient enough.
And how efficient does the algorithm need to be in order to be good enough for you?I see that 'on your way' may be nice but I have no idea how to make efficient algorithm (n log n or better) to solve it.
It is L*x, where L is length of path and x is maximum distance to find job (it is in fact BFS with found path as starting points with limited path length). It also will annihilate FPS.So rather than countering the idea with a better idea all you did is tell me why it wouldn't be efficient enough.
And how efficient does the algorithm need to be in order to be good enough for you?I see that 'on your way' may be nice but I have no idea how to make efficient algorithm (n log n or better) to solve it.
If you have no idea how efficient an algorithm is, then you can't comment on its efficiency.
Hey, I like that. That's how children get lost in forests and stories start. I seriously think there is room for dumbness here. We want to save the player's time by improving processor usage, not save the hauler's time by improving his GPS.
It is just one hauling strategy: if there's a big pile of items, it should be grouped as one set of jobs and assigned to one or more dwarves). The items that are not eligible for that treatment because they are not concentrated enough will inevitably have to picked up that way.
Another big problem with wheelbarrows: stairs. Giving each dwarf a backpack (or basket, bag, etc. depending on available materials and crafters) should be sufficient to do most hauling. Wheelbarrows can then be thought of as a tool to move heavy loads faster, rather than a container. Gravel, ore, sand, clay and other bulk goods should be put in bins for transport anyway: those can then be carried up the stairs as necessary.
Wheelbarrows would be the most useful for a group of identical jobs, eg. 35 pieces of bituminous coal to stockpile: the dwarf would fill a bin, put it on the wheelbarrow, deliver it to the stockpile, get a new bin and go back to fill it - keeping the wheelbarrow with him all the time. Otherwise the decision to get an empty wheelbarrow (probably somewhere near a stockpile) will be a calculation in itself (requiring pathfinding as well, wheels don't walk and the terrain needs to be checked).
that makes a lot of sense. What are the cases for massive amounts of A -> B hauling?
1: To/From Depot (check)
2: To stockpile from workshop (check)
3: To stockpile from corpse (check)
4: To workshop from stockpile (check), (think 'needs 5 steel bars)
Slightly more difficult:
4: To stockpile from mining area
5: To stockpile from battlefield
What other use cases for massive hauling are there?
I don't like the idea of institutionalizing the quantum dumps.
He said it wasn't n log n and he didn't know a way to make it n log n, not that he didn't know how efficient it was.
I only said that I have absolutely no idea how to do it and that it is probably very hard to do it in FPS-friendly way (harder than normal pathfinding that is too slow already).He said it wasn't n log n and he didn't know a way to make it n log n, not that he didn't know how efficient it was.So because he doesn't know how to make it efficient, therefore its not efficient, therefore its bad?
Well, again speaking for me, bad to do unless a way to make it efficient is found. It may not be possible.He said it wasn't n log n and he didn't know a way to make it n log n, not that he didn't know how efficient it was.
So because he doesn't know how to make it efficient, therefore its not efficient, therefore its bad?
I don't like the idea of institutionalizing the quantum dumps.
I'm not talking about institutionalizing quantum dumps, I was just talking about dump zones (which forbid items that go into them) that you unforbid. There are reasons to use dump zones other than creating quantum stockpiles. For example, dumping wood from the surface down a long pit to the magma forges, then unforbidding them at the bottom from time to time to save transit time on woodcutting. (Besides, it's not a quantum stockpile if you're taking it to a real stockpile.)
More generally, any sort of mass-unforbid action will probably cause a big push to stockpile a huge number of items.
what about making wheelbarrows something a bucket is to a water pond or a soap is to a hospital?
only if an item is coming or going from a certain stockpile and to another certain stockpile the dwarf will pick up the wheelbarrow at the stockpile that he needs to pick up the item from to pick up the items and transport it to the second stockpile, then he does the next transport task until he becomes hungry or the end stockpile is full
We could also let wheelbarrows (and minecarts) be associated with stockpiles like bins and barrels are. They would then only be used for jobs relevant to that stockpile (or possibly workshop). Useful for furnaces, that need a lot of coal, carpenters that need wood etc.How do they get back to the stockpile?
Finishing the hauling job would include bringing back the wheelbarrow. Or rather, the wheelbarrows would be at the furnace, and the haulers would got there to get instructions and equipment, do their jobs and when they're done a wheelbarrow full of coal is at the furnace, and the hauler free to take another job.We could also let wheelbarrows (and minecarts) be associated with stockpiles like bins and barrels are. They would then only be used for jobs relevant to that stockpile (or possibly workshop). Useful for furnaces, that need a lot of coal, carpenters that need wood etc.How do they get back to the stockpile?
I'd also like to see an improvement to stockpiles: one pointer for taking items from another stockpile and one for giving to another stockpile. While not a true n:m relationship, it would still be more versatile than the current system, where a stockpile can be given items by many stockpiles, but one stockpile can give items to only one stockpile. With my suggestion giving and taking are functionally similar, but distinct. Any stockpile could give items to one stockpile and take items from one stockpile. That way you could have 1:n and n:1 in various combinations, and m:n would require a "relay" stockpile in the middle (m:n could be formed by m:1 and 1:n). It would ofc require coding various borderline cases (Can stockpile A take from B, while B gives to A? etc)
NW Kohaku, please clarify, weather you want to optimize FPS time or Dwarf walking time.
I remember another suggestion about improved hauling: workshops having miniature input stockpiles, that generate hauling jobs for materials for queued items, so crafters don't need to go themselves to nearby quantum dumps. Do the same for building walls.
I have another suggestion about improving proximity(A) to B type hauling: Mass melting. Melting would be more efficient if multiple items made of the same material were hauled to smelter before actual smelting. It seems more realistic to collect a whole batch of scrap metal before smelting, than burn all that coal to melt 1 bolt or cup or something, and get almost nothing from it.
So an extra trip... (not that that's horrible, it just needs to be explicit. You don't start saving time until you're bringing three things at a time.Finishing the hauling job would include bringing back the wheelbarrow. Or rather, the wheelbarrows would be at the furnace, and the haulers would got there to get instructions and equipment, do their jobs and when they're done a wheelbarrow full of coal is at the furnace, and the hauler free to take another job.We could also let wheelbarrows (and minecarts) be associated with stockpiles like bins and barrels are. They would then only be used for jobs relevant to that stockpile (or possibly workshop). Useful for furnaces, that need a lot of coal, carpenters that need wood etc.How do they get back to the stockpile?
Another thought on wheelbarrows.
Wheelbarrows are buildings. You place them, make a room out of them, and then choose a destination. You then choose whether this is a one time run or a continuing order. You can choose how many items will make a trip worthwhile, and what items the wheelbarrow should grab using the stockpile interface. If the destination is a stockpile then the wheelbarrow picks up what it takes automatically.
Dwarves run up, and fill the wheelbarrow with all loose items in that room. Then the dwarves take the wheelbarrow to the destination, and either dump it all on the floor or transfer it into the stockpile. If the building is set to a one time haul the building is deconstructed, or if it is set to repeat the dwarves rebuild it by taking the wheelbarrow back to it.
Wheelbarrow wrangling could be a skill with low skill dwarves moving slower or spilling items out along the way, and having to stop to put them back in. You could have multipliers for stair tiles making this more likely.
I posted my own idea here: http://www.bay12forums.com/smf/index.php?topic=78517.msg2028383#msg2028383So you started a new thread to post something after reading a thread discussing that topic? This thread is only 4 pages long. There isn't any need to try to split it yet.
Because it contains many, many various ideas how to do it. And mine is not going to fix everything and add multiple pointless objects (top suggestion from ESV) but to fix small part of it.
Dunno, I could see some merit to that... dwarfs treat the wheelbarrow as part of the stockpile?
It's micro, but not horrid micro, esp for mining...
I don't think it could.work for a repeat...
More like... build it in the shaft, say 'fill' as a stockpile, then when you are done, say 'dump here' which sgould be close to where you want it... i.e. your dump, or where you plan to build a stockpile.... makes a good addition to other options.
New stockpile setting: max distance stockpile will pull an item from.
No micro unless you set it up for a one time haul.
You build the wheelbarrow up next to a craft shop, and increase the room size to include the craft shop.
Set the destination for the craft stockpile next to the depot.
Dwarves fill the wheelbarrow up with crafts till it is full, and then dump it at the destination. If it is set to repeat they then take it back to where it was before.
It basically acts as a one tile stockpile, but when it gets full it is emptied at another place.
Honestly, I think carts are a bad fit for DF. No way to lay tracks that is at all efficient.
Hauling Improvements
- Being able to haul multiple small objects
- Having multiple dwarves involved with item hauling for a job
- Being able to move multiple objects with roughly the same destination at once
- Wheelbarrows to haul more objects than can be carried
- Minecarts
- Wooden, stone-carved and metal tracks
- Can be filled like stockpiles and moved between destinations
- Work animals to tow carts and haul objects
Eventually, a fortress would look like an anthill, with all the materials they dig out winding up in piles on the surface. (Solves the quantum dumping issue by making a certain number of stones consolidate into a wall when the last stone is added into a tile. To prevent extra checks, it only happens on the enterance of an item into the tile.)
Hilarious exploit:
If 7 stones become a "wall tile" that wall tile can then be dug out dropping.... 1 stone.
(Note, I am not actually proposing a 7/7 system, which I see as just an artifact of previous map coding compression that should probably be phased out at some future point.)
If they're both manually operated then they're going to need to carry a huge amount of rock to justify the overhead of operating them, let alone the overhead of building them and their tracks. (sidenote, I think some mines used grooves carved in the floor which might work better)
Moving water or magma around might possibly be another use? If you can move large amounts at once it would help stop evaporation problems, and it would be worth the overhead compared to building a pump network in some cases.
If they're both manually operated then they're going to need to carry a huge amount of rock to justify the overhead of operating them, let alone the overhead of building them and their tracks. (sidenote, I think some mines used grooves carved in the floor which might work better)
Moving water or magma around might possibly be another use? If you can move large amounts at once it would help stop evaporation problems, and it would be worth the overhead compared to building a pump network in some cases.
I am looking forward to sections of the fortress moving though, if only for a "Transformers: Fortresses in Disguise" thread.
Wow, nice... grooves could make the whole shebang work.
Mine carts on rails would really only be justified for the continuous mining of huge deposits of ore and coal. While I'm not against crossing the tech border in specific cases, there's a reason mine carts didn't really turn up before the industrial revolution and mass production: too much trouble for those relatively small quantities of goods. So I suspect that mine carts will only see much use for the future huge, concentrated deposits that will be mined for years: only there the distance from the items to the mine cart will always be small.
Yes, I have no doubt that the first thing everyone will do upon the instant a version of DF with moving fortress parts is released is to build a horde of giant spider walkers that vomit magma at attacking goblins. I look forward to the arms race. (I also wonder if "improved sieging" will include goblin spider walkers with magma, just as part of that arms race - SURPRISE, SUCKERS! Remember that child the goblins kidnapped 4 years ago? Now goblins have the secrets of magma-spewing spider-walkers, too!)
... with all that said, I think we've sailed waaaay off topic in the past couple pages. Weren't we talking about ways to optimize pathfinding before?
Or is there nothing left to talk about regarding how to make gathering as many items as possible into the most efficient task processor-wise possible?
I'd also like to see an improvement to stockpiles: one pointer for taking items from another stockpile and one for giving to another stockpile. While not a true n:m relationship, it would still be more versatile than the current system, where a stockpile can be given items by many stockpiles, but one stockpile can give items to only one stockpile. With my suggestion giving and taking are functionally similar, but distinct. Any stockpile could give items to one stockpile and take items from one stockpile. That way you could have 1:n and n:1 in various combinations, and m:n would require a "relay" stockpile in the middle (m:n could be formed by m:1 and 1:n). It would ofc require coding various borderline cases (Can stockpile A take from B, while B gives to A? etc)
I'm not sure I follow you on this one...
Each stockpile can take from any number of other stockpiles, but can only have one stockpile taking from it in turn. This limit applies even if the two stockpiles you want it to feed into don't share a single material that can be stored in both of them. Additionally, you can't make two stockpiles feed into each other, although larger loops (e.g. 3 stockpiles that feed into each other in a circle) are allowed.
Just like to point out that both of those do the same thing. I'm all in favor of adding another UI option, but mechanically those are identical.They are identical in that weather A takes from B or B gives to A, items get moved from A to B. They are not identical in how they are implemented, as "take from another stockpile" fills *this stockpile, and "give to another stockpile" empties *this stockpile. This suggestion is also different from what we have now in that multiple stockpiles could take from one stockpile as well as one stockpile could be given by many stockpiles. In the current system (unless wiki is wrong) a stockpile can take from multiple stockpiles, but any stockpiles can be taken from by only one stockpile.
Saying "A gives to B" or "B takes from A" will have the same effect... You're just allowing two links instead of one.
Personally, I think that a major hauling improvement could be gained by letting dwarves poach jobs from other dwarves. Once you're picking up five objects in one go, you start running into situations like the way that dwarves mine. "I'll mine out all but one tile of this room, and leave it for the other dwarf who's hustling over from across the map!"
Let dwarves keep track of estimated time remaining that it will take to complete a goal. If another dwarf is close enough that they can complete it a lot faster, let them poach it. Hauling, mining, anything.
Personally, I think that a major hauling improvement could be gained by letting dwarves poach jobs from other dwarves. Once you're picking up five objects in one go, you start running into situations like the way that dwarves mine. "I'll mine out all but one tile of this room, and leave it for the other dwarf who's hustling over from across the map!"
Let dwarves keep track of estimated time remaining that it will take to complete a goal. If another dwarf is close enough that they can complete it a lot faster, let them poach it. Hauling, mining, anything.
You'd have to handle the serial theft case too.
There is but it's hard to think of an efficient algorithm to solve a problem we don't understand. If we accept that the only use of minecarts (And related lifts, etc.) is to move things from one or two specific locations or stockpiles, to a few other specific locations, and we think manual control is tolerable, then that sets some fairly easy limits on what the algorithm is actually expected to cope with. In this case we don't really have to change the algorithm at all, we just have a minecart that create store jobs like a stockpile.
Personally, I think that a major hauling improvement could be gained by letting dwarves poach jobs from other dwarves. Once you're picking up five objects in one go, you start running into situations like the way that dwarves mine. "I'll mine out all but one tile of this room, and leave it for the other dwarf who's hustling over from across the map!"That would possibly involve a lot of calculations to see whether and if a suitable dwarf is close, and then if he is close enough.
Let dwarves keep track of estimated time remaining that it will take to complete a goal. If another dwarf is close enough that they can complete it a lot faster, let them poach it. Hauling, mining, anything.
Personally, I think that a major hauling improvement could be gained by letting dwarves poach jobs from other dwarves. Once you're picking up five objects in one go, you start running into situations like the way that dwarves mine. "I'll mine out all but one tile of this room, and leave it for the other dwarf who's hustling over from across the map!"
Let dwarves keep track of estimated time remaining that it will take to complete a goal. If another dwarf is close enough that they can complete it a lot faster, let them poach it. Hauling, mining, anything.
You'd have to handle the serial theft case too.
Seems easy enough. Only take the job if you are significantly closer (say, will complete it in 50% the time), and don't take a job that's already far enough in progress (already holding item to be hauled, already mining the square, etc).
What I mean is, take a 3x3 room and (d) all the walls. If you have 2 miners, one might take the other miners job all the way around the room.
Edit from a long time later:
Actually, you could track item wear or damage more abstractly, if you don't mind a loss of granularity over some types of items. (If you are stacking them, anyway, then you shouldn't be all that concerned with granular details.)
Simply track an abstract "Average" wear or damage number, and then a "Standard Deviation" number, and bell-curve the results abstractly. Hence, throwing a badly damaged item into a stack of reasonably well-maintained items will result in not just a drop in the average wear, but also a spike in the standard deviation, so the bell-curve will be spread out, and abstractly include plenty of low-wear items as well as a few outlier damaged items.
Dwarves could simply draw at random or puposefully pick the most damaged or most repaired ones as specified (so that if they are looking for socks in a sock pile, they would probably want to pick the least threadbare socks to own for themselves).
Please, RAWify stockpiles (p menu and settings of stockpiles), I hate disabling seeds, fat and extracts on every single food stockpile in every single game.
Reposted from http://www.bay12forums.com/smf/index.php?topic=104479.0 (as this thread is linked to ESV)
If a minecart is an option, a dwarf should probably always choose the minecart. The trickier thing is keeping track of which goods can and cannot go on the cart. If a minecart acts like a stockpile, it needs to be connected to a stockpile to dump things in, and it needs to keep track of wich stockpiles the track is connected to.There is but it's hard to think of an efficient algorithm to solve a problem we don't understand. If we accept that the only use of minecarts (And related lifts, etc.) is to move things from one or two specific locations or stockpiles, to a few other specific locations, and we think manual control is tolerable, then that sets some fairly easy limits on what the algorithm is actually expected to cope with. In this case we don't really have to change the algorithm at all, we just have a minecart that create store jobs like a stockpile.
Well, if we don't understand it, it's time to think about it more until we do. :P
... OK, that's all that a first-blush look at how to make a minecart path will tell me... so could someone blow this idea up for me so I can try to think of a better way to rebuild it?
Gah.. so many people posting in this thread... 7 replies... I need to stop reading books in the middle of posts.And writing them. ;)
QuoteGah.. so many people posting in this thread... 7 replies... I need to stop reading books in the middle of posts.And writing them. ;)
If a minecart is an option, a dwarf should probably always choose the minecart. The trickier thing is keeping track of which goods can and cannot go on the cart. If a minecart acts like a stockpile, it needs to be connected to a stockpile to dump things in, and it needs to keep track of wich stockpiles the track is connected to.
I don't think you would necessarily have a single track that winds the full hight of the fortress. You might for mining purposes, but I imagine having issolated tracks carts for different industries, running side by side.
It is? It is! Wait, when did my thread get linked to the ESV?
I asked Draco to link the ESV item to this discussion
Yeah, you still need the mine cart zones of influence. That can be a breadth first search for tiles within a certain walking distance. It could be displayed in a mannor simmilar to the depot wagon access.If a minecart is an option, a dwarf should probably always choose the minecart. The trickier thing is keeping track of which goods can and cannot go on the cart. If a minecart acts like a stockpile, it needs to be connected to a stockpile to dump things in, and it needs to keep track of wich stockpiles the track is connected to.
I don't think you would necessarily have a single track that winds the full hight of the fortress. You might for mining purposes, but I imagine having issolated tracks carts for different industries, running side by side.
Well, no, minecarts can't always be the option, because what if you are, say, looking to bring some food from a farm to a food stockpile, and the food stockpile is 5 tiles away, wheras the nearest mine cart is 100 tiles away, and the tracks don't even lead to anywhere within 60 tiles of the food stockpile, because you've only laid down tracks in the heavy mining segments. You have to check to see if that's the case before making your decision, of course, but there are reasons not to go there.
As for the tracks and carts for different industries, I was thinking about this when I was talking about the "rollercoaster" aspects of the minecarts in some of the other threads, and an old idea I had about conveyor belts-I'm a little weary of full on converyers in a pre-industrial setting. At some point one starts to ask "If dwarves know this much mechanics, why can't they harness steam already?". I'm not sure exactly where to draw the line, but conveyers are near it.Spoiler: quoting myself (click to show/hide)
Basically, if you can create a way to "mechanize" minecart motion by throwing mechanisms under the tracks, and setting up roller-coaster style "acceleration wheels" that are simply wheels that turn due to the torque applied through mechanisms in order to push along a cart that is moving slowly enough through one of the tiles that is mechanized, it will be able to push those carts along a path, or push it up hills so that when it goes down the hills, it can just carry on its own momentum forward.
If you build a full circuit of mechanized carts, however, you could recreate that conveyor belt approach.
That is, if a workshop is attended, all "finished products" from a workshop could be offloaded from a workshop into a cart as it passes by, and all "raw materials" that it will need for the queued jobs can be input into the workshop without needing to have a hauling job performed - workers just automatically offload or take in whatever they need as long as they are at the workshop as the cart goes by.
Then, you can have supply chains like sending a cart track on a loop between a farm, the farmer's workshop, the dyer, the loom, the stockpile, and maybe the trade depot. Minimize your need to actually haul things through automated movement of goods.
In response to NW_Kohaku and anyone else that might be interested, things akin to conveyor belts certainly existed way before 1400... chain pumps (http://en.wikipedia.org/wiki/Chain_pump) come to mind!Well, something like a steam engine existed too, but it's mostly a matter of atmosphere. In addition, mass production would upset the economics of the world, and consequently everything else.
I believe that what probably kept people from developing effective steam engines before the 1700's or whatever was a slow rate of communication, sharing of ideas and designs, exacerbated by lack of literacy, social mobility, and a whole host of other factors...
So a boy was standing there all day watching a guage hit maximum before pulling the lever when he got an idea: He simply devised a system to tie the gauge by a string to the lever, so that when the gauge hit the pressure levels it took before he could pull the lever, the gauge would pull the lever for him. Then he ran off and played.
So a boy was standing there all day watching a guage hit maximum before pulling the lever when he got an idea: He simply devised a system to tie the gauge by a string to the lever, so that when the gauge hit the pressure levels it took before he could pull the lever, the gauge would pull the lever for him. Then he ran off and played.
If you want something done efficiently, ask a lazy person to do it.
When you get the machine to push the button for you, you can run the whole factory with one guy who only has to walk by every now and again to make sure the whole thing isn't on fire, and the machine is still doing its job.
When you get the machine to push the button for you, you can run the whole factory with one guy who only has to walk by every now and again to make sure the whole thing isn't on fire, and the machine is still doing its job.
And even that you can outsource to a machine.
There's a 365T (http://365tomorrows.com/) on this topic somewhere, but I can't remember enough detail to know what to search for.
So I, for one, welcome our new AI overlords.
So I, for one, welcome our new AI overlords.
I don't, per say, but I will enjoy what comes with them. Probably. It depends on if we get a benevolent AI or not.
In that timeframe, as well, we will be slamming into the Carrying Capacity of the Earth (http://en.wikipedia.org/wiki/Carrying_capacity), where humanity will have to destroy itself over the finite resources no longer being able to be stretched further, no matter how much more productivity we may have.
In that timeframe, as well, we will be slamming into the Carrying Capacity of the Earth (http://en.wikipedia.org/wiki/Carrying_capacity), where humanity will have to destroy itself over the finite resources no longer being able to be stretched further, no matter how much more productivity we may have.
In that timeframe, as well, we will be slamming into the Carrying Capacity of the Earth (http://en.wikipedia.org/wiki/Carrying_capacity), where humanity will have to destroy itself over the finite resources no longer being able to be stretched further, no matter how much more productivity we may have.
http://www.space.com/15395-asteroid-mining-planetary-resources.html
This is a company that has just recently been formed that has plans on harvesting asteroids, so there may be a finite amount of material on Earth, but you can't forget the space based resources.
I hope re-stacking bolts comes soon.
Well it's Toady's decision anyway:) I think a lot of info is unimportant to store but i am not a developer of dwarf fortress:)
Topic for discussion:QuoteCoins have their coin batch index (year for dwarves) and the material, but there's also ownership, contaminants, temperature, damage, age, storage information, maker id, flags based on if your fort owns/made it, melt/dump/forbid/hide designations, job information.
What would you say can be offloaded, converted to the stack info, or dumped. Every one of those has to be one of the three.
Well, assuming I am correct in interpreting that "offloaded" means "it's got a pointer to manufacturing data" and "stack info" is "the stack applies this data to everything in the same stack", and "dumped" means "Screw it, we don't need to track this at all", then...
- Coin batch index and material data - offload (Although possibly having some sort of generic "item type" data that would be stack data. That way, you don't have to search where the pointer points to see something very basic like "this is a bolt stack" when you're looking for something like iron bars.)
- ownership - stack
- contaminants - stack
- temperature - stack (Actually, should this be by stack, or by tile, anyway?)
- damage - ***
- age - offload
- storage information - stack
- maker ID - offload
- fortress or import flags - offload
- melt/dump/forbid/hide designations - stack
- job information - stack
*** Damage is the only problematic one. Items in the same stack with one another should probably be damaged the same, but the problem is that they will all start with different damage values. Probably, the best way to really handle this is to make every individual item in the stack have its pointer to the offloaded data, the number of items exactly like that, and also a damage value.
Edit from a long time later:
Actually, you could track item wear or damage more abstractly, if you don't mind a loss of granularity over some types of items. (If you are stacking them, anyway, then you shouldn't be all that concerned with granular details.)
Simply track an abstract "Average" wear or damage number, and then a "Standard Deviation" number, and bell-curve the results abstractly. Hence, throwing a badly damaged item into a stack of reasonably well-maintained items will result in not just a drop in the average wear, but also a spike in the standard deviation, so the bell-curve will be spread out, and abstractly include plenty of low-wear items as well as a few outlier damaged items.
Dwarves could simply draw at random or puposefully pick the most damaged or most repaired ones as specified (so that if they are looking for socks in a sock pile, they would probably want to pick the least threadbare socks to own for themselves).
World war over water is impractical - there is no reason why China would attack USA over water, but increasing number of local conflicts is very likely.
In that timeframe, as well, we will be slamming into the Carrying Capacity of the Earth (http://en.wikipedia.org/wiki/Carrying_capacity), where humanity will have to destroy itself over the finite resources no longer being able to be stretched further, no matter how much more productivity we may have.
http://www.space.com/15395-asteroid-mining-planetary-resources.html
This is a company that has just recently been formed that has plans on harvesting asteroids, so there may be a finite amount of material on Earth, but you can't forget the space based resources.
You don't mine for food.
The carrying capacity of the Earth is the amount of people we can feed if we convert every arable patch of land solely to the production of more food.
It is the functional wall into which our world will have to start killing each other over the scarcity of food as portions of humanity start to starve to death and tax our natural resources to the point where we will cause severe environmental degradation that will make our planet less capable of feeding even the few remaining humans.
This will only be averted by either severely limiting human population growth, invention of "superfoods" that take far less arable land to grow, or interplanetary colonization on a scale that allows us to just export excess humans into space.
Bottom line, the next world war is probably going to be fought over water.