Mary Paschall Young Doty was the daughter of John and Martha Eliza (née Carr) Young, and the granddaughter of Eugenia Paschall (Mrs. Walter) Carr. She and her sister Katharine Campbell Young Keck inherited a portrait of their great-grandmother, Martha Eliza Stevens Edgar Paschall (widow of General John Edgar), which had been passed down through their grandmother and mother. The picture (now in the NGA) was painted by an unknown American artist, probably commissioned around 1823 on the occasion of Martha Stevens' first marriage to Gen. Edgar. Mary Doty's mother had brought it from St. Louis to Geneseo, New York, where it remained in the family home with Mrs. Doty until 1970, when it came to her sister Mrs. Keck in Washington, D.C.