If you're the only one optimising, don't.
Alternatively, don't abuse it. The real problem with being the sole min-maxer in a group of people who don't is when your character continually overshadows the rest of the party. Natch it's partly on the DM to create encounters which cater to different skillsets, even if there's only so much that they can do for a party like his, but when it comes right down to it "I have the ability to resolve every problem we face in a trivial manner," doesn't mean "I should," and if they do end up being the same, it's a pointer at your lack of self-control.
Same basic principle as applies to an omnipotent being, really; just because you
can fix every problem people have doesn't make it healthy for you or for them.
Though as such things go a Specialist Conjurer is about as good as you can get for optimized characters which don't overshadow everyone else, given that a whole lot of what you're doing is setting things up for your buddies. To use another metaphor, a basketball player who consistently outperforms the rest of his team won't be resented as long as he excels by being a team player, making good rebounds, drawing fouls, setting up assists, playing strong defense, &c.