The number of people I've seen wearing fedoras at my college is deeply concerning.
The worst part, is most of them aren't even fedoras, or are terrible low quality ones.
I actually love hats, and own quite a few of them of various types. The problem is that many people think a hat carries itself, though - it doesn't.
First: The hat should serve a use beyond fashion. Either to provide shade or to keep your head warm or to designate your association with a particular group or part of your heritage or
something. Have a reason to wear the hat other than "hats are cool", and you'll look much less stupid wearing it.
Second: The hat is an accessory, and needs to go with the rest of the outfit. Fedoras especially.
Third: You need to wear a hat that looks good on you. Get a colour and shape that works with your face. Fedoras probably aren't it.
Fourth: Decent shaped hats are expensive. If you aren't wiling to drop money on a good quality hat or spend the time looking for a deal on one, buy a hat type that looks good on you without breaking the bank.
Fifth:
Get a hat that goddamn fits you. It's not. That. Hard.A good military cap is generally pretty cheap (I just got one for 15 dollars or so), looks much nicer than a baseball cap, and can easily be worn with casual clothes or nicer clothes, a versatility that the fedora doesn't have so much.