You can see the hole where the pins holding the marble in place were. When the Christians came, they stripped the Colosseum of it's marble, effectively turning it into a quarry.