Why can't a Jew buy an animal from a nochri?
What is the question, if the animal gave birth before the Jew bought it then it belonged to the nochri and the Jew is exempt from redeeming the first born.
1) A Jew is allowed to buy an animal from a Nochri. What is stated in the Mishnah, "even though he is not allowed to," refers to a Jew selling a large animal to a Nochri. This is forbidden because the latter might do work with it on Shabbos.
2) Heshy, the Mishnah says like you are arguing. On the baby animal that the Jew bought from the Nochri, the Jew is exempt from redeeming the firstborn.