Eh, not really. Septua-octogenarians in my family tend to joke about it, but it's more teasing and ice-breaking. They don't mean any harm.
I guess in general it's considered vaguely "unprofessional" maybe, on males. Except in tech jobs where it's still not the norm, but is popular and okay.
Just because daily brushing feels really nice, doesn't mean it's not an investment on par with occasional haircuts.