Just according to Wikipedia, the first "anvils" were rocks. Apparently they could be made of bronze too. You could probably melt bronze and pour it in a stone/clay mould, and chisel off the mold. Could probably do that with iron too with 14th century technology.
It makes sense for DF technology to follow the same pattern of development through materials available as technology and understanding progress. Neolithic dwarves would bang hot copper on rocks with stone tools, make a copper hammer to further work tools, develop a mould from clay or stone to pour molten copper to forge an anvil, and then do the same for bronze, then iron and finally steel. This has just never translated to gameplay, probably because we've always had the opportunity to trade for them off the caravan if we didn't bring one on embark. But being able to use a stone boulder of hard enough material (like granite, gabbro, quartzite...), or make a clay mould for pouring a metal cast anvil, would make sense.
Currently the game doesn't start, at year zero, with Neolithic civilizations. They start out fully formed at year 1 with a settlement, and individual items like anvils probably aren't tracked at that point. They just have them.