At one end of the Royal Mile is Edinburgh Castle and at the other end is Hollyrood House (the palace).