There's diorite, the stele of Hammurabi predates the Romans by a thousand years so it's very tough, but very hard to work. Marble is relatively soft and easy to carve. That softness might contribute to the paint coming off. They used to paint buildings too, like the Parthenon.