Located on a site formerly known as Wattersyde (Waterside) that had a house and kitchen garden, the 5th Earl of Dumfries converted it into a Walled Garden to provide fresh fruit and vegetables for the House.

After a major restoration programme funded by Mr David Brownlow, the newly appointed gardens designed by Michael Innes will be opened officially by Her Majesty Queen Elizabeth on 2 July 2014 as the Queen Elizabeth Gardens.