At the top of Daf 77 Amud bet (and at the end of amud alef) there is a whole series of questions about whether words are spelled with an alef or with an ayin. From this we can perhaps deduce that in the time of the mishna no distinction was made between the pronunciation- right? But what about the Jews of Yemen for instance whom we know were there at the time? Clearly in many communities the difference in pronunciation would have been preserved- and these are normal words, not strange words which nobody knew. Why were they not asked?
What then is the pshat? Why is there a question here?
Daniel Moskovich, Kyoto, Japan
Excellent point. There is a similar Gemara in Eiruvin (53b), which discusses other words which their spelling was in doubt. The Gemarah explains that in the area of Yehuda people expressed things accurately, while the Galileans did not. The Gemara then asks so why don't we ask the Judeans how they pronounce these words. The reply is that the Judeans were approached but it seems that they also were divided on the issue, some pronouncing it one way and other pronounced it differently.
So as you can see your idea was already raised in the Gemara.
Not only is there a difference in the pronounciation of the alef and ayin but it's noted in Shulchan Aruch that one should make the distinction between the two in davening (and in layning). The Mishne Berura agrees and no one says that they are the same non sound. The ayin is a guttural sound closer to a nun than an alef which has no sound but takes a vowel. Most of the edot hamizrach make the distinction. The fact that we Ashkenazim have the nickname Yankel is probably a vestigial remnant.
It's somewhat hard to believe that the distinction wasn't made in Eretz Yisrael and Bavel in the Gemara's time.
As you mentioned, there definitely was a distinction between the alef and the ayin. However, even in those days it turned somewhat into a gray area. The Gemarah Megilah (24b) states that the residents of Beis Shean and Tiv'on did not distinguish properly between the two. The Gemarah Eiruvin I quoted earlier mentions that the entire Galil had difficulty distinguishing between the two.
One must keep in mind that before modern media, different regions had different dialects of the same language. Aramaic was spoken throughout the entire area, but the Gemara in Nedarim (66b) relates that Eretz Yisroel and Bavel had communication problems, despite the fact that they were both using Aramaic. The very fact that the Gemara Eiruvin tells us that the spelling of certain words were in question even in Yehuda, is clear evidence that even in those days things were not black and white.