I noticed that the diorite behind the levers in the jail rooms of the woodland mansions are unpolished. The andesite is polished (as you can see in the screenshot), so that leads me to believe that the diorite is intended to be polished.