Mercenaries that are not in squads do nothing. They'll behave as any other resident, going to the tavern, sleeping in their rented rooms, eating your food, occasionally praying in the temple... (interestingly, I've yet to see any non-scholar resident go to the library.) To make them do anything, you have to assign them to a squad. They'll then follow the orders of that squad, which may be training, patrolling, squad orders, etc. they'll just behave as normal members of that squad. This means that if you have set a specific type of weapon in their uniform, they'll train that weapon's skill rather than their best weapon skill at that time.
Equipping them is slightly more complicated but not really problematic. To give them armor, you need to produce armor that conforms to the uniform for that squad, but when ordering the production, you must make them human-sized. You can do this in the manager/workshop menu by presing (d)etails while selecting an order, and then you can select what creature will be wearing that piece, so it will be appropriately sized.
As for arrows, I've found that since mercenaries can't be squad leaders, and bowmen are not any more common than other military profession, AND dwarves can't use bows (although I have seen bowdwarf visitors), I often just reassign them to a crossbow squad. I don't specifically know about producing arrows, although I doubt you can do it in a dwarf fortress without modding.
Note that ammo is slightly buggy and you must avoid storing ammo in bins. If they are stored in bins, they'll become unusable.