OK, I mistakenly closed the browser to clean up memory leaks in Opera, forgetting that I had a post that was half-written in here, which was covering how fluid dynamics could start to take place.
Also note, I am mostly talking about water when I am talking about fluids, here. In fact, I have to constantly correct myself from saying "water" when I mean "fluid". I will make another post later that will incorporate special quirks of magma, like pressure that builds up in magma chambers to cause eruptions.
So to try to make a quicky version of the previous deleted post (spoiling this into sections to keep length managable:
Current fluid dynamics are one of the most inefficient aspects of the game, only surpassed by pathfinding, and even then, largely because pathfinding is unavoidable, wheras you can always just avoid using fluids.
The problem with fluids in this game are that they are a part of every tile (every tile has 4 bits for fluids, it's just that most tiles have a fluid level of "0"), and apparently, every single frame, the game checks every tile to see if there is going to be a fluid mechanic process run. If there IS fluid that is not locked up (like water in the ocean that isn't on the surface z-levels, where the game can assume nothing is moving), then the process is quasi-realistic in that water units move about randomly, and will eventually fill up a container to a fairly level amount, but is massively inefficient about doing so.
The process seems to run on every tile with water in it individually looking around it for a tile with lower water level, and then passing one unit of water over to one random nearby tile with a lower water level, even if this means a 5/7 tile passes water to a 4/7 tile, causing the tiles to become 4/7 and 5/7, respectively, and, in the iteration to the very next tile that just recieved water, that tile will pass the water straight back to the first one... accomplishing nothing but consuming CPU time.
The thing is, this would be realistic if we are dealing with individual molecules of water, and wanted hyper-realistic mechanics with CPU time being absolutely no object, but instead, every fluid unit constitutes around a foot and a half of water depth sloshing around at high speeds, frequently almost eternally in a cistern at the bottom of your fortress, unless you are particularly careful about making sure all water is exactly 7/7.
What's worse is that even though all these calculations are being done on an iterative per-tile basis, the game still needs to measure water in "bodies" just to find out how pressure works, anyway, meaning the game has to track water as single large units, anyway.
Running fluids as single large multi-tile bodies, where more complex calculations can be run, but are only run once per frame per fluid body, rather than on a per individual tile basis, would be able to greatly reduce the amount of total calculations that the CPU will have to run.
First, bodies of water must be composed of objects that are handled in their own special case of dynamic memory, rather than as properties of every tile. For the purposes of pathfinding, there will probably need to be flags for "dangerous amounts of water, non-aquatics cannot path here" or "MAGMA! HOT!" that can be put in individual tile mapdata memory, but otherwise, like contaminants or furniture, fluid bodies can potentially be seperate chunks of memory, with a seperate "fluid mechanics" phase of every frame.
After that, we need fluid body to recognize when they are in a state of rest - that they are filling the container that the fluid body is in as efficiently as possible, and that no further motion is necessary, unless it is somehow disturbed. This means that fluid, which would probably now be measured in liter form or the like, would try to level off, and then form a "placid lake" when not disturbed, with the top levels of a fluid body evenly distributed, with a remainder of water bodies, for example, probably gravitating to the edges in a little mockup of surface tension just to make things as even as possible. Once it hits this state, fluid mechanics only checks for disturbance of the water, even if there is non-filling of the tiles.
Density can then become a matter of bouyancy. Water has a density of 1000 in the game's choice of units, but this has problems since creatures are mainly made of 500 density organs, bones, and the like, which would make most creatures bouyant, and we need to have creatures capable of sinking to be capable of drowning. Most woods, then, would also be bouyant, which may make for some fun mechanics, like floating logs down rivers to get to the sawmill. (Of course, I do consider bouyancy something of a low priority if you do not wish to do all this in one go.)
With that, however, we can also start modeling Displacement. Dumping a 70-liter stone into the water will displace 70 liters of water when it sinks. This causes a 70-liter rise in one tile of the fluid body (as the fluid mechanics would have to recognize how much space is available in every tile), and this would then have to start getting distributed.
This brings up distribution rates, which is where we typically have to start breaking open the calculus. Distributing the fluids, when we no longer have simple tile-to-tile transfers and delays between checks being the means of altering flow rates, depends on a great many factors, although fortunately, constant gravity, incompressible fluids and several other quirks of DF mechanics allow us to reduce several difficult variables to constants. This means that we really only need a few variables: viscosity of each fluid (which will need to be a constant variable associated with every type of fluid), density (again, already a constant variable associated with every type of fluid), and fluid pressure (which is a simple, direct function of fluid depth, unless we start getting vulcanism involved with magma, where we might get some fun-fun with magma getting pressure that builds up from below).
Pressure I could go into a long calculus formula about, but the whole thing simply reduces, because of the number of constants, to the pressure at any given point essentially being a function of a fluid's density times the distance to the surface plus the pressure of all fluids above the fluid body (which means you add air pressure, which is probably a constant), so a point 5' below water has half the pressure of a point 10' below water (if you ignore air pressure).
This means that if you have a one-tile (with each tile being a 3m cube) pit filled completely with water, and then mine out a tile below that water so that the water can escape, the pressure at the moment of being allowed to flow would be the pressure of 3 meters of water (plus atmospheric pressure) (I'll get into the exact mechanics of the rate of flow a given pressure implies later) (Figure A), while if the pit were two tiles deep and completely filled with water, and the bottom carved out, then it would have the pressure of 6 meters of water, which would cause twice the rate of flow of the first pit at the instant of being undermined (Fig B).
If we were to carve out that pit from the side, however, we would have a different problem, in where to measure the "depth" of the water. If you carve a full wall away so that there is an entirely empty tile next to a tile that is completely full of water, then we are actually talking about the average of densities from the top of the tile to the bottom of tile to find a sort of average pressure between the two. In a one-tile deep pit of water, this means that water will flow as if it has 1.5 meters of depth for simplicity (Fig C). If it was two-tiles deep pit, and the hole was carved at the bottom, it would flow like the water at the bottom of the pit had 4.5 meters of depth (Fig D). If the empty space were carved at the upper tile in the water pit, then it would still only be an effective 1.5 depth of water, and the lower part of the pit would obviously not matter for this fluid calculation. (Fig E)
Using this, what's really nice about this is that it can also be used (if with some slightly more complex mechanics), we can find how fast a fluid will flow when we are talking about a 3-meter-deep tile of water flowing into a 2-meter deep tile of water, which would have a .5 meter depth pressure pressing the flow, because of taking half the relative difference in the water heights, which is a difference of 1 meter. (Fig F)
This also works in the exact same way if you have 1-meter deep water flowing towards a dry tile, a .5 meter pressure flow. (Fig F)
Now then, why am I saying "4.5 meters of pressure"? That's because, thanks to all the variables being constants (at least, constant to each fluid type) except pressure, which is, itself, a direct function of fluid depth, that means that, except for specific density and viscosity of each fluid, all we need is a constant to find the rate of flow for all these fluids.
(http://sadpanda.us/images/174888-Q29RNMA.jpg)
These constants, however, are actually a bit of a matter of taste, as a rate of flow necessarily requires the dimension of time, which, as I've already said, we don't want to make on the same scale as everything else.
This means we basically just need to find a constant to force flows to move at a rate that, for lack of better technical terms, "looks good".
Viscosity can simply be a constant of 1 or 1000 or some such for water, and be a different variable for other fluids to reflect their resistance to flow. (This is assuming simple newtonian fluids.) You can simply divide the flow rate by the viscosity of the fluid to produce the "fluid friction" specific to that fluid because of viscosity. This can, in fact, be kludged by simply setting a different effective "SPEED" rate for every type of fluid's checks - if magma is arbitrarily set to be 10 times more viscous than water, you could simply run magma flow checks 1/10th as frequently as the water flow is checked. (Which may be what is already happening.)
Density is the opposite - the denser the material, the more force gravity can apply to a material, and for the purposes of purely gravity-driven fluid flows, doubling the density will double the rate of flow. (So flow rate is multiplied by density when calculating purely gravity-driven flows.)
This is just a basic, 2-d version of fluid flow, however, I'll update on how "puddle mechanics" should work, since this whole thing is getting pretty long already.
Forgive me if this gets a little rambling, but this is, in part, a matter of my reasoning through the problem (and also my showing my work), so it will be long and discursive. I will try to make an "executive summary" post at the end of all this to make a more concise case for a fluid system.
Allright, for my next trick, I have to reconcile the fact that we are dealing not with a single cross-sectional view of how much force will be pushing water across a plane, but to deal with the fact that water in this game is actually going to have to look much more like this:
http://en.wikipedia.org/wiki/Shallow-water_equations
That is, if you are piping water off a cliff into a large pit at a constant rate of flow, then as water is added at the point where the water lands, then the sudden "pile" of water that is generated in one tile has to then be distributed outward into all the surrounding tiles.
One way of handling this is to simply divide the pressure into 8 seperate horizontal flows, but this starts to reveal one of the problems with doing this this way - we would then start to be dealing with fluids in the same interative, per-tile manner that I am trying to avoid making us use all over again.
Instead, I want to go back a step, and talk about how fluid actually MOVES once we have calculated the force behind it.
For the purpose of talking about flows, I decided that I would simply have to create a mock-up constant to illustrate how the system will work. Again, the problem is that it is difficult to find data on just how fast a puddle should spread, which the Internet is simply not helping with. (It keeps wanting to talk to me about refilling my water bottles. Stupid Internet.)
I looked at the current model we have to see what it would be replacing. Amusingly, the first 30 or so steps after I dug into the murky pool, it did nothing but stood there as an immobile wall of water, and the miner had a chance to walk several steps away. On the step that it finally did move the water (lets call this 31), it immediately pushed 4 of the 7 units of water on the adjacent tile into the opened hole. The next step (32) moved the tile behind it, so that it went from 4/7 north of 3/7 north of 7/7 followed by many 7/7s (thanks to the shape of the pool), into being 4/7, 5/7, 5/7, and then 7/7s. The next step (33) did nothing. The step after that (34) pushed one of the 7/7 to be 4/7, 5/7, 6/7, then a 6/7 surrounded on four sides by 7/7. This is what's in the first image below. Note the water not flowing into the open field. Step 35 did nothing. Step 36 finally pushed the water forward, leaving only 1/7 water in the previously 4/7 tile, and 1/7 in one diagonal, 0/7 in another diagonal, and 2/7 in the orthogonal. 37 did nothing. 38 caused a minor shift in the 6/7s. 39 did nothing. 40 gave my miner another step away, and shifted some 6/7s in the back of the pond. 41 simply passed the 2/7 water at the edge to the 1/7 without spreading the water outwards any (third image)... Then 42 just has the 2/7 flow back into the lake, instead of spreading outwards from it. frankly, this only impresses upon me the need to remake the entire system, so I'm not going to use this for any inspiration. In fact, by the time that the water spread to that other tile diagonal from the breach, another 22 steps had passed.
(http://sadpanda.us/images/174990-HGT8LRH.png)(http://sadpanda.us/images/174992-CJDXLOJ.png)(http://sadpanda.us/images/174997-O21GGD8.png)
So then, I'm going to just have to make something that "looks right". By which, I mean I have to basically just figure out how fast water should flow based upon estimating how tall a wall of water a dwarf should be able to outrun. The answer to that is "probably not very tall" or let's just say .5 meters of depth just to give it an arbitrary number.
Now, we can start reverse-engineering what sort of constant we are using to get this number back from the predetermined answer that was pulled out of thin air.
Rate of Flow = Velocity * Area through which the fluid is travelling. We are making velocity up, and declaring it .3 meters per frame, since a tile is 3 meters, and 1 turn tends to take around 10 frames. This means time is a measure of frames, now, not seconds, which would screw up most of the other calculations if I wanted to go out actually using calculus and newtons of force. The area for this one particular case is the three-meter-wide tile, but even if it is a 3-meter square opening, only the bottom .5 meters are actually filled, so we are dealing with a Rate of Flow of .3 m/frame * 1.5 m2. This winds up giving me .45 m3/frame, or 450 liters to be pushed from one tile to the other in this one frame. For reference, .5 meters depth of fluid in a 3-meter-cube tile is 4,500 liters of fluid, total, so 1/10th of this fluid would be pushed in the first frame/check into the empty tile. (Note that this would cause the original water tile to have a depth of .45 m in the next check, and the second tile to have a depth of .05 m of water when the next flow check takes place, for a relative difference of .4 m of water, cutting subsequent flow.)
Now, we need to make the link between this velocity, and the pressure of the relative water depth difference. If the pressure generated from .5 m of water results in .3 m/frame velocity (or, to make it as a force, .3 m/frame2), then we are making this constant .6 units/frame.
Now, we can actually start doing many more equations - a full 3m pit of water when a wall is mined out adjacent to it will then generate pressure that creates an immediate velocity of 1.8 m/frame, and have an opening area of a 3 meter square, or 9 m2. This results in a rate of flow at the first check of 16.2 m3/frame, or 16,200 liters of the total 27,000 liters. Amusingly, this is actually more than half (which probably indicates my arbitrary constant is too large, but whatever), which would result in water sloshing back and forth until it would level off. Because mechanics like this would simply keep sloshing back and forth smaller and smaller amounts of water approaching infinity, there will have to be a point where the volumes that are flowing are considered so small that the CPU just stops tracking it, and teleports water to the point where it is at rest, but I'll go into that in more detail when I have the rest of this better fleshed out.
Now then, I can go back to where I started this post, and start talking about how this model needs expansion based upon two major problems - one is that bodies of water can have multiple "outflows" from it at once, like a single "swell" of water that must be divided in multiple directions, but the other is that I am talking about a velocity of water (and technically an acceleration of water) based upon pressure. If we had a situation somewhat like having a large water tower with a hole punched in the side of the tower near the bottom, the water would not simply drop, it would be projected into a stream of water. This latter problem is the problem that I want to tackle next - bodies that have a flow/velocity.
Therefore, bodies of water have to be divided into two types - lakes and rivers (or pipes), or bodies that either have no flow or do have a flow.
Lakes are those standing bodies of water that have no motion other than sloshing around the water in response to more water being added, or perhaps water being drained. When water is drained off of a lake, thanks to opening a floodgate, or maybe a dwarf digging a channel, water then starts flowing out of the lake at a given velocity. If the tile that is being flowed into is just a dead end, it simply becomes part of the lake whose water it took, expanding the body of water. If, however, once water flows into an empty space, and the computer checks, and finds more space for that newly flowing water to flow into (like a long channel), then a new body of water is created, a river, and the river has a flow. Even though it is contguous to the body of water that is a lake that created it, the river now has a special flow and velocity mechanic associated with the body of water, with the interface between the two bodies of water being the opened floodgate. The flow of water into the new river translates into its velocity travelling onward through the channel. As water is pushed into a river, it tries to push the water it is recieving forward at a similar rate, or picking up the pace if it happens to go down a drop.
This is viewed from the side. The "lake" that this channel is draining is arbitrarily wide, so it does not appear to drain, for the purposes of just demonstrating a continuous flow through an empty channel.
(http://sadpanda.us/images/175070-IZ0Q72A.png)
This might seem semantic at first, but it is important in the building of rivers and brooks and dwarven perpetual motion devices that the notion that water has a velocity, and is maintaining its momentum until it has a reason to stop.
Consider again the water tower with a hole in the side - if that water starts spouting out from the hole under pressure, it will have forward momentum. Falling water itself would actually need special rules for applying gravity to the momentum (downward) so that it accellerates to a terminal velocity before it finally can come to rest wherever it lands. In this case, the "river" would be a continuous flow from the hole down to wherever it lands, although perhaps this might be translated as individual globs of water with individual trajectories being pathed individually, depending on how much "spray" we want to have. (In this case, it might be necessary to create a third type of body, a "spill" body, whose purpose is just to handle projectiles where there is no inflow or outflow of water, simply a traveling body of water that is trying to path to the lowest point it can find to be at rest, at which point it can turn into a "lake" body, unless it joins another body.)
In a brook or even a rapids, we can consider the "river" type body of water to simply be having an inflow and and outflow, which we can easily measure, and as long as the two are equal, we can save on processor power by just assuming the river continues flowing without changing in its shape in any fundementally important way. The same can be said of "Dwarven Perpetual Motion Devices" - when pumps remove water from a current (and place it back into the same current), we can just balance the inflow and outflow to find a stable state where a constant rate of flow occurs, without having to waste CPU cycles finding the exact same answer over and over.
In the case that the inflow and outflow are not equal, such as that diagram I made above, where water is inflowing, then the water has to expand outwards to find a place to store the extra water that is entering this body. If, again, this turns into a dead-end channel, then the whole body of the "river" will eventually turn into a "lake" as it fills its container, and the two contiguous "lake" bodies can be joined (until the floodgate is once again closed).
The important part is that the expanding water has a starting velocity. Real rivers will then have dips and rapids and narrows that can alter velocity accordingly (the narrower the river, the faster it will run, proportional to its starting point). This means that we cannot track a river as having a velocity, but rather that different slices of rivers will have different velocities. (This may actually be better handled by simply making seperate river bodies to track, however.) This would get especially important when dealing with pipes, or aquaducts or other ceilinged water flows, as dropping water down in elevation before forcing it to come back up a bend will actually dramatically increase the water pressure and potentially its velocity as it drops.
This means we can handle the problem I was talking about in the start of this post in a different manner - when water is added into a pool from above, this creates a "swell", a raised portion of water in a lake, which then has to be distributed. How this ultimately gets distributed in a lake is obvious: it should be perfectly level, with all tiles at the highest z-level having the same water depth. The problem is finding the right rate to balance this swell out against the nearby tiles (which is a matter of that constant, above), and having a final smoothing function that simply makes a lake that has sloshing finally just die down instead of having "ripples" that slosh water around at the top of a lake. More important, however, is that this allows us to still treat this as a single body with a single set of calculations, rather than as a bunch of individual tiles. This means, rather than having one tile with more water in it than others, we are tracking a "lake" which happens to have a swell centered in one tile, whose water can be tracked as partially distributed. (Inversely, there should be a "depression" function for water draining out of a lake at a certain point, so that water outflowing a body creates a small whirlpool as water is removed first from the tiles near the outflow, with water flowing towards the depression to even out the lake again.)
I've been keeping this up on my screen for a couple days, and I don't want to lose it again, so I'm going to post this here.
So in my brief attempt at another thread (swiftly picked apart by Kohaku) I tried to come up with a simpler method. I've changed my mind a bit about this. So I'd like to contribute to this thread on designing a way to present volume and mass to the player in a non-confusing method.
Quote of Kohaku's rebuttal:
Ideas like this actually came up frequently as common alternatives to the Volume and Mass suggestion I was pushing for earlier.
What it comes down to, however, is that for either this or Volume and Mass's idea of precise units but fungible sizes of specific pieces of material, you need stacking first.
The reason we only get one log or one boulder now is that large item lists bloat the current item vector (although I do wish DF would move off of a vector and onto a container type that is more functional with large numbers of contained pointers) is that we simply cannot stack 10 wood planks as a single item times ten, we have to store them as ten individual items.
It's less a problem now that dwarves can collect from multiple piles, but the primary reason that this sort of suggestion wasn't acted upon before was that dwarves needed to collect items from multiple piles if they needed 5 units of wood planks or something. Without stacking, this is still a problem.
Mining: for mining, I think that a stone taking up 1/3rd of a tile might actually be easier (so 3 rubble or 2 rubble and one boulder with a possible giant boulder taking up 2 regular boulder's worth) than 7, and 7 is just the number we are most familiar with.
For these, rubble (or gravel or whatever you want to call it) would best be handled with a generic non-ore rubble type and be capable of stacking with the rubble.
People have pointed out that ores should often be a type of gravel or rubble rather than a boulder, as well, which would be a typed distinctly (hematite rubble/gravel) before being worked.
Sand is something we may want to consider more carefully. While I'm all for having a fixed definition of how much sand is allowed in a tile, and making pickaxe mining a Non-Newtonian Fluid impossible, a blunt 7 units of sand per tile is exactly the sort of problem with item sizes that we're supposed to be moving away from, here.
The same goes for wood - part of the objective is to make trees of different sizes, which means more or less wood per tree, rather than a one-size-fits-all 7 wood. This is especially important if we have tree farming later on with improved farming, and we have different "crop yields" in the tree farming.
Anyway, this whole thing came up pretty often in some flavor or another in the actual Volume and Mass thread, and I'll say here what I said there:
So long as there is stacking, there is NO FPS difference between having the game track "10 liters" or "7 kilograms" of wood and the game tracking "wood [5]". Literally the only difference between a stack of one and a stack of the other is that in Volume and Mass, the units have a distinct name.
For example, currently, metal ingots take up 2 liters. So, a stack of "Iron bars [4]" is exactly the same thing as "Iron bars: 8 liters". (Excepting, of course, that in this case, the unit of measure is half that of the other, but as long as stacks are nothing more than integers of a given unit size, that's largely irrelevant.)
So, basically, your "simpler" solution is actually no simpler at all. The Volume and Mass idea is essentially the same thing, but with a name for the units that makes comparisons between different stacks easier.
EDIT:
In fact, the common unit of measurement in the game (at least, the raws) is the cubic centimeter. That is the unit used in [SIZE:whatever] in the creature raws. It is probably also what is used in the hard-coded raws. SIZE, which everything has defined, is just a cubic centimeter.
If we go by my calculations of a tile, we have [SIZE:27000000] in a tile. Toady recently talked about a 2m*2m*3m tile for minecart physics, so that would translate into [SIZE:12000000] tiles. If you just declare that a tile full of sand has 12000000 cubic centimeters of sand in it, you're pretty much done. Again, this is the unit of measure the game already uses for everything, anyway, it just doesn't tell you outright yet.
You just need to use a unit to display that volume, and liters (SIZE:1000, or 12000 liters per tile) would be an easier way of doing so. Then, you can compare a liter of meat to a liter of wood to a liter of metal.
In this system, how does the player actually act? Does the game actually say that 1 chair requires 5.2 kilograms of wood? Does a hauler go find 5.2 kg of wood and haul it to the workshop?
For the player, what is the functional units? Do we try to be realistic and say that 1 arrow requires .2 kg of wood, 1 earring .02 kg of gold?
I would argue that that would be very confusing to keep track of, even with a good in-game interface for it. I think a nice, easy integer solution would be more accessible to more players, even if we sacrifice some realism.