How could Eliezer discuss returning Yitzchak to there? Yitzchak had never been there!
Riva #1: Since (one of) the Avos 1 was once there, it is called returning.
Riva #2, Rosh, Hadar Zekenim (Pasuk 8): It would be a return for Eliezer, so if he would bring Yitzchak there, it is appropriate to call this returning.
Malbim: There are two reasons why she would not want to come. (a) Going with a slave [is not honorable]. (b) She must leave her father's house to go far away. Perhaps I may take Yitzchak there to marry her, stay a few years 2 , and then return here!