Cleve Ginsberg was born in New York on September 22, 1918. (The family changed its name to Gray in 1936.) He attended the Ethical Culture School in New York, and completed his college preparatory studies at the Phillips Academy in
Andover, Mass., where he won the Samuel F. B. Morse Prize for most promising art student. In 1940 he graduated summa cum laude from Princeton with a degree in art and archeology. He wrote his thesis on Chinese landscape painting.