Gemara

(Ula): To check for good lineage, find a family that is silent when confronted. In Eretz Yisrael, when families quarrel, the first family to be silent is considered to have better lineage.

(Rav): The best lineage in Bavel is the families that are silent.

Kesuvos 14a (Beraisa #1): A widow of an Isah is (a widow of) a family with no doubtful Mamzerim (only Safek Chalalim, i.e. Pasul Kohanim);

R. Meir says, if a family has none of these, its women may marry Kohanim;

R. Shimon ben R. Elazar says in the name of R. Meir, it is when a Safek Chalal was mixed into a family. Yisraelim recognize Mamzerim, but not Chalalim.

(R. Yochanan): They argue about one who protests when he is called a Mamzer, but is silent when he is called a Chalal. The first Tana says that if one is silent to any accusation, a widow of the family may not marry a Kohen. R. Meir agrees if one was called a Mamzer. If was called a Chalal and is silent, he doesn't care. The widow may marry a Kohen;

R. Shimon ben R. Elazar is Machshir one who is silent when he is called a Mamzer. Yisraelim recognize Mamzerim, so he knows that people will not believe it, since there has been no talk about him. If he protests when he is called a Mamzer, but is silent if called a Chalal, we assume that he is a Chalal. He is silent because he can still marry into Yisrael.

(Beraisa #2 - R. Yosi): One who is silent when he is called a Mamzer is Kosher. One who is silent if called a Chalal is a Chalal.

(Beraisa #3): One who is silent when he is called a Chalal is Kosher. One who is silent when he is called a Mamzer is a Mamzer.

R. Yosi holds like R. Shimon ben R. Elazar. Beraisa #2 holds like the first version of R. Meir in Beraisa #1.

Rishonim

Rambam (Hilchos Isurei Bi'ah 19:22): If one who is quiet when he is called a Mamzer or Chalal, we suspect him, and one may not marry him until checking.

Ra'avad: This was only in previous generations, when Beis Din would excommunicate Yosef for calling David a slave, or lash him if he called him a Mamzer. If David did not take Yosef to Beis Din to be punished, he admitted that it is true. Nowadays, it is praiseworthy to be quiet in a quarrel, unless Yosef said so not during a quarrel.

Question (Taz EH 2:4): Since the Ra'avad distinguishes between whether or not it was during an argument, why must he distinguish between nowadays and then?

Magid Mishneh: The Rambam rules like the first Tana in Kesuvos. He holds that silence is like a family about which there was protest. Checking helps, for it is no worse than if witnesses testified that a Pasul became mixed in. The Ramban and Rashba say that silence is like admission only if we suspect his lineage, e.g. a Pasul or Safek Pasul was mixed in. If a family has Chezkas Kashrus, whoever is silent has better than lineage. There is no proof in the Gemara for their opinion or for the Ra'avad's.

Ran (Kesuvos 5a DH Tanan): Some say that silence is not an admission if he is always silent. If he screams about other Pesulim, and is silent about this one, this shows that it is true. Perhaps this is even if his family has Chezkas Kashrus.

Poskim

Shulchan Aruch (EH 2:4): If one who is quiet when he is called a Mamzer or Chalal, we suspect him and his family, and one may not marry from them until checking like we said (four mothers who are eight, or 12 for a Levi or Yisrael).

Gra (17): We learn from Kesuvos that the entire family is suspected.

Beis Shmuel (10): The Rambam requires checking lest he is a Chalal even without silence! We must say that we must check four mothers only when he was silent.

Rema: Some say that this refers only to a family into which a Mamzer or Chalal was mixed. There is no problem if someone else is quiet when called one of these.

Gra (20): Kidushin 71b says that silence shows good lineage.

Bach (7 and Chelkas Mechokek 6): This is when there are rumors that a Pasul was mixed in. If there are witnesses, one must check even without silence.

One could say that this is with witnesses, and we assume that the one who was silent is the Pasul, and until he is checked, the others need not be checked.

Rebuttal #1 (Prishah 21): The Tur does not say that this opinion argues with the Rambam, and the Rambam came to teach about Isur, not Heter! Rather, a Safek Pasul was mixed in. One who was not silent need not be checked.

Rebuttal #2 (Beis Shmuel 12): The Rema comments on the Mechaber, who requires checking the entire family if one was silent! The Rema cites the Ri's opinion. The RI explained R. Yehoshua, who permits Almanas Isah. The Halachah follows R. Gamliel, who forbids. Therefore, silence does not forbid! However, if a Safek Mamzer was mixed into a family, one may marry women from it. However, if she was silent, we establish her to be a Mamzeres. Also, if a Safek Chalal was mixed in, if a widow from the family married a Kohen, she need not leave unless she was silent.

Rema: Some say that this was only in previous generations, when Beis Din punished for falsely slandering another's lineage. Then, silence was like admission. Nowadays, it is praiseworthy to be quiet in a quarrel. If his lineage was slandered not during a quarrel, we are concerned.

Gra (21): Kidushin 71b says that silence shows good lineage in Bavel, for there they did not collect fines (for slandering lineage or anything else). Nowadays, everywhere is like Bavel. In Eretz Yisrael, the lineage of the first to be silent was preferred. He was not totally silent!

Rema: Some say that silence is like admission only if he screams about other Pesulim, but not if he is always silent. This refers to insulting his lineage. If one slanders his children's lineage in front of him, silence is not admission, but one must be a bit concerned.

Gra (24): Maharam Padavah (14) brought the three opinions (that the Rema brought) why silence does not disqualify. Also, Bahag (cited in Tosfos Bava Basra 127b DH Kach) and R. Tam (Yevamos 47a DH v'Hilchesa) say that a man can disqualify his seed only through identifying his firstborn. Also, the Gemara said that one is silent about Chalal, for it does not disqualify himself to marry a Bas Yisrael. The same applies when his children's lineage is attacked. However, one should not be lenient l'Chatchilah in practice.

Rema: If one is shamed about other matters and is silent, this shows that he has proper lineage.