Dwarfs dislike drinking water and greatly dislike "muddy" water. That is in the game, and is an consequence of the alcohol dependency.
Have you ever drank water from a natural pond, with leaves rotting in it? Please get a good mental image of that water with little bits of rotted vegetation.
Compare the image of pond water to an image of real tea... Not the easily-made bagged tea, but the type where there are tea leaves in the bottom of the pot and after filling the cup, little bits of tea leaves float around.
Most dwarfs are going to look at tea as pond water... It doesn't matter if it is hot. It is dirty water that has no alcohol content.
There might be a few dwarfs that like the flavor, but they are not going to convince many others to join them in a "tea ceremony". "He felt satisfied after participating in a ceremony with a friend. He felt disgusted after drinking nasty water.
I suspect that players who enjoy tea would like to think that their dwarfs would enjoy tea, but I suspect that is more an issue of players applying their own feelings to the dwarfs.