If it was only stored on farms it would be trivial to "rotate" crops. Just deconstruct and reconstruct the farm in the same spot - presto, new land for farming.
Instead it could start out at 0 fertility (the lowest it can drop to) and have that as the basic farming, producing a small crop and allowing you to do it anywhere. Then if you want to fertilize, you add potash (or whatever fertilizer that might be added later) to the spot and it gives it a bit of fertility boost, up to a maximum. Stored entirely in the farm plot, as it is now - and giving a bonus to the crops grown in that plot, with each maturing plant eating up that square of the farm plot's fertility. So if you fully fertilize a 5x5 field, once you plant and grow 25 plants (one in each tile) the fertility would be back to 0.
Water could increase fertility, but only up to a certain point. This would make having a water system important - you could save a lot on fertilizer just by flooding your farms before each fertilizing/planting cycle, and it would boost it up to, say, 25% of the max. So you could farm decently with no fertilizer and a seasonal flood of water and fresh mud, or you could fully fertilize and just use it to cut 25% of your potash usage.
You could even add in a simulated crop rotation system, where every season a farm lies fallow it gains 5% fertility. Alone, it would take 5 years to get 100% fertility out of a field without any fertilization. Starting with water it would be 3 and 3/4 years - but you wouldn't have to wait for 100%, every fallow season would just slightly boost the crop for when you do plant.
This is a fairly simplistic way around the issue, but I dare say it would be effective and hopefully not too difficult to implement. It would be similar to the system that's in place now, except base farming would produce very little, while a ~50% fertile field would produce about what we get now, and 100% would be the same boost that we get if we fertilize with the current system. Fertility would stay with the farm plot across seasons, gaining a bit when you wash them with water or when the field is empty for a season. You could get the same results as you get now without any fertilizer if you made enough farmland to be able to wash the plots with water and then wait 5 seasons between each planting, or better results if you either waited longer or applied fertilizer. So newbies could still farm ok, and farms overall would require either more land or lots of potash to run effectively.
I also think the grow durations should be upped, but since that can be done in the raws, the rest of the system is most important.
Damn I'm long winded... sorry for the wall of text
Short version: Farms would use similar fertilizing tag as now, starting at 0, but 0 would be a significant cut in production. Applying river water would raise it to a point, leaving the field fallow would slowly increase it per season, and applying fertilizer would increase it up to a maximum. Each plant grown would eat up one tile's worth of the total fertilization, until the plot returns to 0. 50% would be equal to current production, 100% would be equal to current production with fertilizer. All data would be stored in the farm plot's building, to prevent extra data usage on all tiles.