Mosses and Lichens are pioneer plants. It is their role to colonize bare rocks, logs, etc and start the process of breaking these inhospitable environments down into something where other plants can grow. They do this by secreting dilute acids that break down the surface they are growing on and by catching dust and soil that happens to blow/wash past. Takes many years to make much difference but those are the timescales that nature works at.
The point of this is that lichens do secrete substances that can slowly dissolve stuff like bark. The good news is that a living tree will produce bark much, much quicker than the lichen can dissolve it - Mum and Dad's cherry tree still has bark even after many years of lichen covered trunk, right
The Japanese bonsai growers hate anything growing on the trunks of their bonsai and will always get rid of any moss, etc. Hiro made the comment on his last visit that a tree with moss on the trunk reminded him of an old man that does not wash. He appeared less concerned about the health effects on the tree than the visual 'tidiness'
Up to you whether you keep it or not.