Bay 12 Games Forum
Dwarf Fortress => DF Modding => Topic started by: kane_t on June 16, 2015, 12:38:42 pm
-
I've found myself in a situation where I have a barrel full of single strawberries or plump helmets, which my brewer then turns into multiple barrels each containing 5 units of alcohol. It's a bit of a waste. Seems like if a hauler brings a barrel with 5 units of strawberry wine to a stockpile with another barrel containing 5 units of strawberry wine, they should just pour the contents into the existing barrel and leave the one they were carrying empty.
I was thinking of trying to make a script that just checks if there are multiple barrels of the same kind of alcohol in a stockpile and, if so, merges their contents up to some sensible maximum, by removing the contents of one barrel and just marking up the stack size in another barrel. Or, solving the problem on the other end by just looking at a single barrel (of fruit, in this case) and merging the stacks within it (so all my strawberries [2] become a single strawberries [26]).
But this seems like such an obvious problem, I figured somebody must have already made a script or plugin to do it. Does anybody know of such a plugin? I tried looking around but haven't been able to find anything.
-
None mentioned in the readme or the most recent thread, so I guess not. It might be some kind of horribly thorny problem though.
-
It's a kinda hard problem to solve, how do you decide which items to merge?
-
It's a kinda hard problem to solve, how do you decide which items to merge?
Well, the simplest solution would be to just leave control up to the user. Have them select an item in a container, and merge any items of the same type in the stockpile into it up to a maximum stack size (and container capacity). There are already scripts that work on the selected item, you could just c/p that functionality.
The more complex, automated approach would be to, whenever a dwarf stores a container in a stockpile, for each element in the container, search the stockpile for a stack of that item of less than maximum size in a container which isn't full and, if found, merge into the other container up until the maximum stack size or container capacity. Continue until either all the items in the stack have been moved out of the barrel, or there are no valid destinations for merging, then move on to the next element of the container.
I took a quick look at the data structure for stockpiles in dfhack, and it looks like stack size is just an int you could increment, and you can iterate the contents of a stockpile pretty easily. The only complication I can see is whether it's possible (or desirable) to straight-up remove an item from a container. How well the game handles something like that.
-
Doing it automatically could kill fps with large stockpiles, this would need testing. Manually would be possible, even fairly easy, but would be a lot of micro management to make it really effective.
Annother option is to make a "stacker" workshop, as long as the workshop is powered it will take an item from it's inputs (using the Rubble powered workshop API, which is the easiest system available for making automated workshops) and try to stack it up to a user settable limit using other items at it's inputs. This would be easy to do, but I don't have much time right now, so we will see.
-
I cooked up most of a quick manual script myself this morning, but haven't had time to test it. I'll let you know how it goes when I get a chance to finish it. Personally, for me, the main thing it's a problem with is alcohol, so the manual approach works just fine for my purposes. If other people want a copy of the finished script I can fancy it up a bit with some user-settable options for maximum stack size and such and post it here.
I wouldn't think the automated approach would cause performance problems, as long as there's an Event for "item stored in stockpile." Actually iterating stockpile items and increasing or decreasing their stack_size should be trivial, and depending on how fast dfhack's items.remove() function is, removing empty stacks shouldn't be a problem. It's just finding out when a dwarf puts a barrel in a stockpile.
I wasn't able to find documentation for plugins (and didn't feel like going through the source) but one of the events exported to Lua is "onJobCompleted," and a quick skim through the job class suggests that it's possible to just check a flag to see if it's a hauling job. I'd just have to look into it in more detail to see if it's possible to extract the item that was hauled from that event.
-
I was thinking of having to iterate all items in the stockpile looking for a merge candidate, if no mergable items exist and you are doing a lot of hauling it could be a real fps drag (if the stockpiles are really large). Obviously this needs to be tested to see if it is actually the case, it may not be a problem.
I like my automated workshop idea, but it would need some modifying to prevent problems with rotting...
-
I don't know if there is an event for "item stored in stockpile", but I would just have it run every X number of ticks. Otherwise if you have 10 dwarves bringing something at roughly the same time to a stockpile of 100 items, none of them stackable, you are going to run through the same 100 items 10 times. I don't think it would make much of a fps as that is still a relatively small number of things to check, but it is inefficient. Where as checking every X number of ticks shouldn't be too bad.
You also have an issue of dwarves going to get an item from a stockpile and then it suddenly being gone. Not sure how much of an impact that would have.
-
Well, my stockpiles generally have closer to 10000 items...
-
I don't know if there is an event for "item stored in stockpile" [...]
You also have an issue of dwarves going to get an item from a stockpile and then it suddenly being gone. Not sure how much of an impact that would have.
Isn't that the onJobCompleted event? Or does "store item in stockpile" not count as a job for the purposes of that event? (Or, I guess, is it only available though the Lua interface or not possible to extract the stored item...) If the onJobCompleted event works, that also solves the problem of dwarves cancelling a job because the item disappeared: presumably, the event is fired on the same tick that the item is stored, before another dwarf has a chance to claim the item. Items can't be claimed while they're being hauled, after all, and in my experience it takes a few ticks before they become aware of a newly-available item.
Well, my stockpiles generally have closer to 10000 items...
Okay, yeah, that's a couple orders of magnitude beyond any fort I've gotten off the ground ;)
Still, I wouldn't expect it to be a big deal, since it's just iterating an array and doing straight integer comparisons. Even if you need to do a couple pointer indirections/table lookups to get a pointer with the right type, it's nothing. That's assuming you do it in C++ as a plugin, rather than in Lua, which might be slower.
-
You know you have a lot of items if when scrolling the stocks screen it pauses for up to a second when reaching prepared meals, logs, bars, or boulders...
And yes, it really does that. Makes using the stocks screen a real pain sometimes. Not sure how an automatic stacking script would react to such a fort, but it couldn't be good, particularly since all items are stored in one of 4 stockpiles... (the fort is only 10 years old, I hate to see what it would be like later in life)
More sane fortresses will probably be fine though.
-
You know you have a lot of items if when scrolling the stocks screen it pauses for up to a second when reaching prepared meals, logs, bars, or boulders...
And yes, it really does that. Makes using the stocks screen a real pain sometimes. Not sure how an automatic stacking script would react to such a fort, but it couldn't be good, particularly since all items are stored in one of 4 stockpiles... (the fort is only 10 years old, I hate to see what it would be like later in life)
More sane fortresses will probably be fine though.
Yes, but the stocks screen is actually displaying something, not just checking numbers, which will be much faster.
Just a quick check I did, a stockpile has a list of all barrels/bins/wheelbarrows in it. The dfhack.items.getContainedItems() command can then easily check for contained items, and you can check those against some list you have prepared. If you want to put even more work into it you can have a flag that says you have already checked a certain barrel/bin and not to check it again unless it has changed. Overall I can't see it having much of a performance impact, except for maybe if you used it on a fully grown fortress, but that would be a one time thing, and then would be much faster after it has merged all stacks. (But I could also be very wrong!)
-
Just a heads up, I just checked the onJobCompleted event exported to Lua and, for some reason, it doesn't report the completion of hauling jobs. Not sure if that's intentional or a bug, though, or if it's just a limitation of the Lua interface's less sophisticated event handling.
Shame about that, because the job struct includes a vector of the items that were included in the job. From the item you can get the container and stockpile, which would've made this a lot easier.
-
PTW
I'm very interested in this, even if only for booze. I have been toying with the idea of making custom reactions for it, but it seems like too many places for it to go wrong.
-
PTW
I'm very interested in this, even if only for booze. I have been toying with the idea of making custom reactions for it, but it seems like too many places for it to go wrong.
Then you're in luck, sorta, because I made some scripts that merge both plants and drinks manually. The links to them and a description of their use is here, in the DFHack thread (http://www.bay12forums.com/smf/index.php?topic=139553.msg6324438#msg6324438). They aren't the most convenient ever, I'm afraid, but I don't have a ton of time right now to improve them.
I did, just now, upload the current version I've been using, which adds support for flour to combine_plants, and lets you specify a container or stockpile ID with -stockpile (for _plants and _drinks) and -container (for _plants). That way, you can use the repeat script to run my combine scripts at a regular interval, if you want. Anything more complex than that will have to wait until I have some free time.
If you notice any bugs, give me a shout and I'll fix them as I'm able.
-
PTW
I'm very interested in this, even if only for booze. I have been toying with the idea of making custom reactions for it, but it seems like too many places for it to go wrong.
Then you're in luck, sorta, because I made some scripts that merge both plants and drinks manually. The links to them and a description of their use is here, in the DFHack thread (http://www.bay12forums.com/smf/index.php?topic=139553.msg6324438#msg6324438). They aren't the most convenient ever, I'm afraid, but I don't have a ton of time right now to improve them.
I did, just now, upload the current version I've been using, which adds support for flour to combine_plants, and lets you specify a container or stockpile ID with -stockpile (for _plants and _drinks) and -container (for _plants). That way, you can use the repeat script to run my combine scripts at a regular interval, if you want. Anything more complex than that will have to wait until I have some free time.
If you notice any bugs, give me a shout and I'll fix them as I'm able.
Awesome! Thanks very much. Both scripts seem to work like a champ so far.
I've been trying to micromanage my drink & plant stocks by forbidding and unforbidding around jobs, and trading all small stacks away but it's been really getting old. Plants I use a QSP for, but the tons of barrels of like 5 drink in them were killing me.
-
No problem. I'll post here if and when I update the script in the future. The main thing I'll be doing is improving the performance and maybe adding support for other types of item (like cheese and ammunition, maybe).
-
Scripts still working well. I use them constantly.
One thought, might be better to reduce plant stacks to 6.
What happens is when you bag 12 quarry bushes you get a stack of 60 leaves, this turns into a lavish roast so large it can't fit in a barrel.
I'm on the fence if this is a good thing or not since it makes it easy to select and trade, but then it's a huge value item, like hippies and humans don't usually even have enough stuff I want to trade them 1 even.
I was about to go do this, but then I wasn't sure what happens when you get a larger stack to begin with (like say from L20 planter + potash) if it is going to break something or not.
Booze of course only will brew half a 12 stack, although if you queue the job 2x they will use the second 6 instead of hauling. Thread, sugar, flour and dye of course the large stacks are nice, saves bags.
-
What happens is when you bag 12 quarry bushes you get a stack of 60 leaves, this turns into a lavish roast so large it can't fit in a barrel.
Oh yeah, I hadn't thought about that specific problem. There are probably other plant types that process into stacks based on their input stack size which might cause problems.
The reason I picked 12 is as you suspected: brewing is done in multiples of six, so picking a multiple of six for the combined stack size allows two maximum-size brewing jobs to happen in a single hauling action (for the plant, at least). With fertiliser and a high farming skill it's possible to get stacks of 9 or 10, so 12 isn't a significant difference. (And, no, the script doesn't break if you have unusually large stack sizes in the container. Though, it may choose to take items from those large stacks to merge into smaller ones.)
Maybe this should be reported as a bug, actually—brewers obviously split stacks to ensure that drink doesn't exceed barrel capacity (although, they don't take into account the larger capacity of a pot), so maybe cooks should do the same thing.
When you say the lavish roast can't fit in a barrel, are you sure it isn't getting stored? I know I've cooked roasts larger than a barrel's maximum capacity (usually because my cook selected a full barrel of 30 drinks for one of the inputs) and they've gone into a single barrel just fine. I'm pretty sure the maximum capacity is just a limit for adding more items. So you'll end up with a barrel just containing your one giant roast.
I might have to change the script to better handle barrel capacity regardless, because I realised yesterday that it's not doing that properly. Because of the way it processes items, it has a high chance of filling barrels past their capacity. As near as I can tell, this doesn't cause any problems, but it does violate the simulation's mechanics in a way I don't like. I've got a couple ideas of how to handle that, but not much time to implement them. If anyone can confirm that over-capacity containers are causing problems, I'll try to squeeze in the time to fix it as soon as I can, otherwise I might have to put it off for a week or two.
In the meantime, you can use the -max parameter to manually set the desired stack size. That's the main reason I separated the drinks and plants versions of the script, so you could individually change the stack size for them. Later, I might add the option to specify maximums for specific plant types, so you can ensure that quarry bushes only get combined into stacks of 6, but leave strawberry plants in stacks of 12.
-
Yeah, I'm looking at a roast that started as 72 I think, they've eaten it down to 66.
Another of 85. Both in meals storage pile, that has other barrels of roasts, and I've got plenty of empty barrels.
I'm thinking it is when a single item is too large for the barrel that it doesn't go in. The other barrels in there seem to fill up to around 40-50 urists, which is about what these weigh. I'll try to keep an eye on the shrinking one to see what the threshold is for it getting barreled.
-
Alright, that's good to know. Immediately, there's a quick fix you can do to your copy of the script. After line 67, which reads:
local itemsNeeded = max - currentPlant.stack_sizeAdd the following line:
if currentPlant:getType() == 53 and currentPlant.mat_type == 419 and currentPlant.mat_index == 176 then itemsNeeded = 6 - currentPlant.stack_size endWhich will use 6 as the desired stack size for quarry bushes. Exempting quarry bushes from merging altogether would be easy in a minimally-adequate programming language with a continue statement, but in Lua it would require adding two lines of code in different parts of the file, so if you want that I'll just upload a modified version of the script to github.
Like I said, I don't have time to do much work on it right now, but I figure I'm going to have to rewrite it to allow customising the items included in merging and setting separate stack sizes for them. It'll be substantially easier to write code that complex in C++, so I'll rewrite it as a plugin, and I actually thought of a more-or-less adequate way of automating it, so I might make an automerge plugin as well. I figure some people would want it done automatically, and some would only want to do merging in very specific cases. It's just a question of whether I find the time to do it this month or the next.
This situation does seem like a bug, though, even without my script mucking things up. Just doing the math in my head, it's possible to get a stack of 9 bushes from a fertilised plot, which process into a stack of 45 leaves. If your farmer selected four such stacks as the inputs for a lavish roast, you'd end up with a stack of 180 quarry bush leaf roasts. Slightly more than fits in a barrel. Cooks should either split input stacks like brewers do, to ensure they never have inputs totalling more than 60, or they should split their output into smaller stacks if necessary. (Cooks should also intelligently choose different ingredients for each input, rather than selecting four stacks of quarry bush leafs, but that's another gripe altogether.)
-
Cool. I'll give it a shot.
60 roasts seems to be the magic number, at which point they put it in a barrel.
I haven't bothered with pots in this fortress. Mountain embark, no vermin, so it really hasn't bothered me with the roast sitting on the floor.
Like you said, even without mods or scripts it's still quite possible to go oversize.
Didn't seem to work. I've done some other modding to my game though, although I think the itemcodes are hardcoded anyway.
Haven't noticed anything else amiss with barrels, but then I'm a recent QSP convert.
-
PTW
I'm very interested in this, even if only for booze. I have been toying with the idea of making custom reactions for it, but it seems like too many places for it to go wrong.
Then you're in luck, sorta, because I made some scripts that merge both plants and drinks manually. The links to them and a description of their use is here, in the DFHack thread (http://www.bay12forums.com/smf/index.php?topic=139553.msg6324438#msg6324438). They aren't the most convenient ever, I'm afraid, but I don't have a ton of time right now to improve them.
I did, just now, upload the current version I've been using, which adds support for flour to combine_plants, and lets you specify a container or stockpile ID with -stockpile (for _plants and _drinks) and -container (for _plants). That way, you can use the repeat script to run my combine scripts at a regular interval, if you want. Anything more complex than that will have to wait until I have some free time.
If you notice any bugs, give me a shout and I'll fix them as I'm able.
Can anyone explain to me how to actually use the combine drinks script?
I just get error messages when I type in the command.
-
You need to have a stockpile selected when you call the command. Select it with the buildings selection command (q) then call combine_drinks, and it'll combine the drinks in the selected stockpile. Does it still not work when you have a stockpile selected? If so, let me know what the error is.
(You can also, optionally, specify a stockpile by ID with an argument, for use with the repeat command. The parameter for that is "stockpile", as in "combine_drinks -stockpile <stockpile building ID>". This way, you don't need to have a stockpile selected.)
-
It's still giving an error. This:
D:\DFMasterwork\Dwarf Fortress\hack\lua\dfhack.lua:354:...asterwork\Dwarf Fortress\hack\scripts/combine_drinks.lua:5: unexpected symbol near '<'
stack traceback: [C}: in function 'error'
D:\DFMasterwork\Dwarf Fortress\hack\lua\dfhack.lua:354: in function <D:\DFMasterwork\Dwarf Fortress\hack\lua\dfhack.lua:344>
-
Okay, that's weird. There isn't even a < symbol in the script. The only thing I can imagine is that the script somehow got modified and a bug was introduced? As far as I know, all Perdexis did for the LNP was add the documentation comment at the top (and it's more than five lines, so the error would be inside the comment, which doesn't make sense).
If you attach the file, I'll take a look and see if I can figure out where the bug is. If it hasn't been modified, the only other possibility I can imagine is that it doesn't play well with mods (since it hard-codes item types), but even that seems unlikely, since those item types should be hardcoded in the game itself.
-
Maybe I messed something up when trying to download it? What's the correct way to get the script downloaded and installed?
-
Apologies for the delay. You should just need to save the file as combine-drinks.lua in your hack/scripts directory. (You can copy/paste the entire text into a new file if you don't want to download it using Github.) It also comes bundled with the newest version of the LNP, but it looks like you're using some mod that has its own special launcher and stuff.
-
Hmm. I think I might have screwed up the actual download. It's not that obvious on github how you actually download the script. I've done it by right clicking on Raw this time. I don't recall what I did before.
Hopefully this will work.
Is there a way to do this with meat and fish etc by any chance? It seems like all my barrels are being taken up with food leaving none for drinks.
-
If the problem is barrel capacity, I'm afraid not, you'll just have to build more barrels. For everything but liquids, barrel capacity isn't based on number of stacks, it's the total amount of items. One stack of ten takes up as much space as ten stacks of one, so it doesn't matter how your fish and meat are stacked, they're already using your barrels as efficiently as is possible. You'll just have to keep pumping out barrels at a regular pace to make sure you've always got empty spares for brewing.
The reason why I made the combine-drinks script is because, for drinks, barrels can only contain a single stack, and dwarves won't combine partially-empty barrels with the same kind of liquid in them, so you can end up with two barrels containing strawberry wine [2], wasting 28 units of space apiece, when really dwarves should just be merging them into an empty barrel and one containing strawberry wine [4]. And combine-plants has a similar motivation: each plant brews into 5 units of drink, but brewers won't intelligently go through the stockpile and grab 6 plants of the same type to fill a full barrel, they'll just grab that single strawberry next to the strawberries [4] and make a pointless barrel of strawberry wine [5], so merging plants improves brewing efficiency by ensuring that brewers (usually) make the maximum amount they can in a single action. (And there are similar problems with cooking that make it preferable to combine cookables into similar stack sizes.)
All of this would be solved if Toady made workers form ideally-sized stacks in their hands when grabbing items for a job and added a minor "maintain stockpile" job to sort and store loose items and merge barrel contents, which is something that I'd like to see happen, but probably won't for quite a while.
When I come back to Dwarf Fortress, I'll probably make a better version of these scripts, and I might add support for meat and fish in just to help with cooking, but I don't know when that'll be. Sorry!