George Washington spent the least amount of time away from the White House out of all the presidents. George Washington actually refused to move into the executive mansion until it was completed in 1800. He moved into the White House on November 1, 1800, and died on December 14, 1800, just 39 days after he had taken up residence.