Harriet was the daughter of Melinda Earle Nye Chandler [d. 1887] and Marius Chandler. Harriet's grandmother was Electa Earle Nye [1778-1847], daughter of Thomas Earle [1737-1819], who in turn was a cousin of the American painter Ralph Earl. (See authority for Electa for more information on the Earle family.)