Kellie's Castle is located in Batu Gajah, Kinta District of Perak. It was built by William Kellie Smith, a Scottish planter as a gift for his wife. He came to Malaysia as a civil engineer but started to plant rubber trees and to deal in tin mining. With the fortune he made, he married his then Scottish sweetheart, Agnes and brought her to Malaysia in 1903.

William Kellie Smith built his first mansion, aka Kellas House in 1909. After the birth of his son Anthony, he started planning for a huge castle with Scottish, Moorish and Indian architecture. The plan was for a 6-storey tower with indoor tennis court and rooftop courtyard.

70 craftsmen from Madras, India were brought in. Bricks and marbles were imported from India too. The plan for the 6-storey tower also included the first elevator in Malaya.

During a short trip to Lisbon, Portugal in 1926, William Kellie Smith died of pneumonia. His wife was devastated and decided to return to Scotland. Hence, the construction of Kellie's Castle was never completed.