In 1874, a new Culpeper town depot was built, which later burned in 1903. A year later, the present building was completed to replace it. Years of use and subsequent decline led Norfolk and Southern Railroad to request permission in 1985 to demolish a portion of the Depot.
A committee formed to save and renovate the building, a project that was completed in 2003. Today, the beautifully restored Depot houses the Visitor's Center, the Chamber of Commerce, the Department of Tourism, and space for meetings and conferences.