Thomas Jefferson, the second President to occupy the White House, directed that hundreds of small trees be planted in groves around the grounds (none of them exist today).
John Quincy Adams was the first to add ornamental trees. Andrew Jackson brought in the first sycamores, elms, and maples.
There’s more about the history of the White House gardens and grounds here.
What kind of small tree/large shrub do you think this was?
I never before knew the full value of trees. My house is entirely embossomed in high plane-trees, with good grass below; and under them I breakfast, dine, write, read, and receive my company. What would I not give that the trees planted nearest round the house at Monticello were full grown.
—Thomas Jefferson to his daughter, Martha, (from Philadelphia), 1793