The ruins date back to the early-to-mid thirteenth century. It was originally the bishop's seat of the diocese of Moray. Over its history, it suffered many disasters, but the heaviest damage was a result of the Reformation of 1560.