Clayton, the county seat of Rabun County, was incorporated December 13, 1823. The city was named for Judge Augustin S. Clayton, a prominent jurist and congressman. Clayton was originally named Claytonsville, the name was later shortened to Clayton.

Clayton is home to many regional authors. The most famous publication is the nationally acclaimed Foxfire series, a collection of oral history gathered by local high school students. The Foxfire Museum, in nearby Mountain City, houses exhibits relating to Appalachian life and documents the birth of the Foxfire Magazine and the Foxfire Book series.