Mamzer prohibition forbids only a definite Mamzer from
entering the congregation
1) I find the Logic behind the Limud used in this Gemora hard to understand.
1a) Because the Torah states the word Mamzer - the prohibition applies only
to a definite Mamzer, not a Safek Mamzer which is permitted.
1b) Why don't we use this logic and apply the same Limud for all other Chayvay Lav'vin explicitly mentioned in the Torah ?
How would the Rishonim who hold that "Safek d'Oraisa l'Chumra" is a Halachah d'Oraisa in Chayvay Lav'vin explain this ?
2) Why in fact don't we use "Roiv and/or Chazakah" to say that every Safek Mamzer is not a Mamzer unless Proven otherwise ?
Actually, see Rashi that the second Lashon of the Gemara uses your reasoning.
The first Lashon learns from the extra word "Kahal" two Limudim, see Tosfos Yeshanim that we make a Hekesh from the word "Kahal" to the word "Mamzer" and therefore both are Vadai.
All other Lavin do not have an extra Pasuk nor this Hekesh.
The Rishonim use this Gemara as a proof that the rule of Safek d'Oraisa l'Chumra is mid'Oraisa, for otherwise why would we need a Pasuk to be Matir a Safek Mamzer. The first part of Sefer Shev Shmaitsa and Sefer Sha'arei Yosher are dedicated to answering this question on the opinion of the Rambam.
If we have a Rov, then we will definitely apply it. Chazakah does not apply in most cases, but where it does (see Kesuvos 14-15) we will also apply it. See Ritva for cases where there is no Rov nor Chazakah. These are the cases of Safek to which the Gemara is relating.
D. Zupnik