Malbim: (here and Melachim II, 25:24): There he meant 'do not fear lest the Bavliyim that he appointed will be their masters and Yisrael will need to serve them. Also here, do not fear lest you need to serve the Bavli commoners that he left in the land, who are Avdei Umos ha'Kasdim, and you will be slaves of slaves. When you serve Melech Bavel, it will be good for you. He will cover you in the shade of his wings from the commoners.