The son of James, 6th Duke of Hamilton and 3rd Duke of Brandon, and his wife Elizabeth (created Baroness Hamilton in 1776), James George Hamilton succeeded his father as Duke in 1758, when he was not quite three years of age. He became successor also to the Marquessate of Douglas and the Earldom of Angus upon the decease of the last Duke of Douglas in 1761. When James George died at the age of fourteen, he was succeeded by his brother, Douglas.