Christ Episcopal Church was established in 1839 and the current Gothic Revival building was built in 1858. The original Episcopal Rectory was west of the current church, at the intersection of Van Dorn Avenue and Craft Street. This rectory was destroyed around 1905 when Oscar Johnson, who owned the nearby Walter Place, began designing a grand park in the area. The park was never fully completed, and today this area is a residential subdivision called Johnson Park.
In 1885 the current Rectory building, located on the lot north of the Church, was built. According to local legend, this lot was originally used by the circus every time it came to Holly Springs and the Episcopal vestry eventually bought the lot to prevent the circus animals from interfering with church services.
Christ Rectory was built by the famous architect Samuel M. Patton (d. 1897), who was born in Jackson, Mississippi in 1857. Patton studied architecture in New Orleans as a young man and joined the architecture firm of Sulley, Toledano and Patton. Christ Rectory was one of his earliest projects and was completed in 1885. In 1888 Patton moved to Chattanooga, Tennessee, where he was responsible for many famous buildings, including the Richardson Building (1888), Lookout Mountain Hotel (1888), and Patten House (1893). Patton built the Vicksburg Hotel (1889) in Vicksburg, Mississippi and also completed the designs for the grand Tennessee State Prison in Nashville, Tennessee before his tragic death in a fire in 1897.
The Episcopal Rectory is the only Second Empire building in Holly Springs. It is a two-story hip roof frame house, with a two-story polygonal bay with a tent roof and a three bay porch. There are three arched windows with decorated hood molds. The most important structure is the north square tower with a mansard roof, one of the defining features of a Second Empire building.
The Episcopal Rectory is the home to the current Episcopal rector, the Very Reverend Bruce McMillan.