To sum things up: A higher mineral scarcity value will mean that more layers are devoid of minerals, but will usually mean that when a layer has a mineral it is completely saturated with that mineral because there is no competition. A lower mineral scarcity value will mean that fewer layers are devoid of minerals however the layers that do have minerals frequently have multiple types of minerals competing for the same area. Lower scarcity means more variety and more total minerals but typically smaller quantities of each type of mineral whereas higher scarcity means less variety, less total minerals but a greater quantity of the minerals that are present.
Hope this clears some things up for you guys.
If this is the only thing, it was already figured out. Some tests showed that 10000 had variety, and the general topic about it lead to believe this was the case. And UristMcDaVinci put an hypothesis that the mineral scarcity feature might be bugged due to an 16bit limitation.
Also, there are some non-consistent things in your posts, being:
A. Some layers of stone are wrongly placed, as we see some sedimentary rock on extrusive layers and don't usually see extrusive layers typical rocks on their respective layers. The system isn't as realistic as one may assume. Is good, but not THAT good.
B. Also, iron ore availability, which is a common ore in nature, also is missing ins several layers.
C. Some strange behaviors like and almost omnipresent metal ore in a layer missing and replaced by a non typical ore of that layer just by changing the scarcity setting by a small amount, namely 500.
D. The system and you did not take account of metamorphism of the rocks, the law of superposition and rock strata.
To try clear things a bit,
this image should do it.
The tests I ran with temperature did show that temperature didn't mattered in the formation of rocks, pressure I still don't know.
The meaning of all this is that even an "simple thing" like "The first thing done in world gen is pretty simple -- it determines the layer-stones found in the various biomes." isn't as simple, and "The second thing done in world gen is it takes each layer of stone -- not each z-level, but each distinct layer stone -- and determines which minerals and gems are present in those stones."
Gems in particular have and other behavior not strictly linked to layer, but ambient. As you may know, a layer can contain carbon but the atoms may or not be arrange to form diamonds depending on the pressure. Since pressure isn't taken (I think) in the world gen, gems are more... random, maybe. Since I didn't constructed the system, we can only conjecture about it's standard operation routine.
And, as stated in
http://www.bay12forums.com/smf/index.php?topic=79018.30, in a test embark made, you had 21 types of rocks/gems/metal with an 2000 scarcity, while in the same site, with more scarcity, namely 3500, you got only 5 variety.
An actual example is my fort, who has whole layers of Dacite, an igneous, volcanic rock and iron ores veins, limonite and magnetite, that are found in sedimentary layers. Didn't embark in two biomes, and even if I did, I could not find one standing side by side to another.
You did a god joob trying to explain things, but I think UristDaVince hypothesis make a lot more sense.