Sag Harbor Business District

The entire business district of the whaling port and writer's colony is listed as Sag Harbor Village District on the National Register of Historic Places.

Sag Harbor is about three fifths in Southampton and two fifths in East Hampton. The dividing line is Division Street which becomes Town Line Road just south of the village. Most of the defining landmarks of the village — including its Main Street, the Whalers Church, Jermain Library, Whaling Museum, the Old Burying Ground, Oakland Cemetery, Mashashimuet Park, and Otter Pond are in Southampton. However, almost all the Bay Street marina complex, including Sag Harbor Yacht Club and Breakwater Yacht Club, at the foot of Main Street, is in East Hampton, as are the village's high school, the Sag Harbor State Golf Course, and the freed slave community of Eastville.

The median income for a household in the village was $52,275, and the median income for a family was $70,536. Males had a median income of $41,181 versus $34,750 for females.