The pathways which are emphasized and strengthened in our minds are determined by what we are doing, so we can influence them! When we were very young and our brains were developing, they were acclimated to certain activities and actions and words. This effect is most pronounced at a very young age, when the brain is growing, but continues into adulthood. The brain doesn't have a preset, genetic plan for which areas of the brain will be stimulated/emphasized, it's determined by our actions and choices and what we choose to experience. If we choose to experience something different, our brains will become different. Humans are versatile, not static.
The fact that everything isn't genetic doesn't mean that we have free will. We can't choose what we experience as a baby. Thus we can't choose what we experience as a child, since what we "choose to experience" as a child is decided by our genes and what we have experience as a baby. And what experience as adult is due to our genes and our childhood and so on forth.
Of course we can't control our environment particularly well, so we really cannot choose what we experience anyways. At least not to the degree you seem to imply.
Don't really care to join the freewill argument - mostly because I'm unsure about my stance on it - just wanted to point that out.