Emilie was the daughter of Jenny Mather who was part of the Mather family in lowland Scotland which emigrated to America and France. The French branch of the family ran a foundry which manufactured cannon balls, some of which are exhibited in Les Invalides in Paris. The American immigrants settled in Boston.