Formed in 1871 from parts of Jackson County and Macon County, Swain County was named for David L. Swain, governor of North Carolina from 1832 to 1835. Much of the county lies within the Great Smoky Mountains National Park, including its highest peak, Clingmans Dome. Four scenic rivers, the Nantahala, Tuckasegee, Oconaluftee and Little Tennessee flow through the county and form the sprawling Lake Fontana, west of Bryson City, the county seat.
Native Americans have been living and hunting in the vicinity of what is now Bryson City for nearly 14,000 years. The Qualla Boundary, which comprises the bulk of the reservation of the Eastern Band of Cherokee Indians, dominates the area to the east.
In 1988, a scenic line known as the Great Smoky Mountains Railroad was established with its depot and departure point in Bryson City.