What is "Oy Lecha Mo'av" referring to?
Rashi: It refers to Bil'am's curse of Mo'av - that they would be delivered into the hands of Sichon. 1
See Sifsei Chachamim.
What is the meaning of "Am Kemosh"?
Rashi and Ramban: It means the people of Kemosh, the god of Mo'av (to whom the people were totally devoted - Ramban). 1
Who turned Mo'av's sons into refugees and its daughters captives?