Dr. Paine inherited through his father, John Merrick Paine, two portraits by Winthrop Chandler, painted around 1780. One depicted Captain Samuel Chandler, the artist's brother, the other, the Captain's wife, née Anna Paine. The Chandlers had no children of their own and had raised Anna's nephew, John Paine, who was Dr. Robert Paine's great-grandfather. John Paine thus inherited the Chandler portraits (along with the Captain's 300-acre farm), which descended from father to son in the Paine family. After Dr. Paine's death, his widow inherited the pictures (now in the NGA); she in turn passed them jointly to their three children--Mrs. Prudence Paine Kwiecien, Hamilton Child Paine, and Agnes C. Paine.