Gemara

Kidushin 65b (Abaye): If Reuven told Shimon 'you ate Chelev' your Taharos (Tahor food or Kelim) became Tamei', or 'a man had Bi'ah with your ox (so it is Pasul for a Korban)' and Shimon was silent, Reuven is believed.

A Mishnah and Beraisos support these teachings. All three cases were needed;

Had we taught only about Chelev, one might have thought that there silence is an admission (lest we force him to bring an invalid Korban Chatas), but silence about Tamei food is not, since he can eat it when he is Tamei;

Had we taught only about Tum'ah, one might have thought that there silence is an admission, for now he cannot eat it when he is Tahor, but silence about his animal is not, since anyway, not all his animals will be Korbanos. (Rather, we learn that in all cases, when he is silent the witness is believed.)

(Abaye): If Reuven told Shimon 'your wife was Mezanah (forbidding her to you)' and Shimon was quiet, here also, Reuven is believed.

Answer #2 (Rava): He is not believed. Matters of Ervah (Kidushin, divorce, and adultery) require two witnesses.

Support (Abaye, for himself): A blind man (Levi) used to learn in front of Shmuel. One day, they sent a messenger to get him. Levi came on a different path. The messenger returned and said that Levi's wife was Mezanah. Shmuel told Levi to divorce her only if he believes the messenger, i.e. that he is not a wanton liar.

Rejection (Rava): No, he must divorce her only if he trusts him like two witnesses.

66b (Beraisa - R. Akiva): One witness can testify that a Mikveh is deficient, or that a Kohen is blemished. Two witnesses are needed to testify that he is a Chalal;

(Abaye): Normally, if the Kohen denies the blemish, one witness would not be believed. Here, if the Kohen is correct, he must show us that he has no blemish.

Gitin 54b (Beraisa): If Reuven was working with Shimon's Taharos and he told Shimon that they became Tamei, he is believed. If he says 'your Taharos that I worked with on a previous day became Tamei, he is not believed.

(Abaye): He is believed only in the Reisha, for then it is b'Yado (he could make it Tamei) at the time he tells him.

(Rava): Reuven is believed only when he told Shimon the first time he saw him (after, according to Reuven, it became Tamei).

Rishonim

Rif and Rosh (Kidushin 28b and 3:15): Only two witnesses are believed about Devar sheb'Ervah. However, if one witness testified and one believes him like two, he must divorce his wife to be Yotzei Yedei Shamayim.

Rosh (Gitin 5:8): R. Tam says that Levi contradicted the witness of Zenus or said 'I do not know'. In Gitin, Levi was silent, and Reuven says that Levi knows or saw supporting evidence, similar to 'you ate Chelev'. If not, he is silent because he does not know. The Ritzva says that even when he should have known, and he was initially silent and later protested and said that he never admitted, rather he was silently thinking about whether or not it is true, it is not an admission.

Ran (Kidushin 28b DH Garsinan): R. Tam holds that Rava and Abaye agree that the witness is believed only when Levi should have known and was silent. They argue only about Devar sheb'Ervah. In Gitin, Rava answered unlike Abaye because he holds that in the entire Beraisa, it was b'Yado. If not, the witnesses would not be believed. Even though it was b'Yado, he is believed only if he told him at the first opportunity.

Rebuttal: In Yevamos we could not prove that one witness is believed from a case when silence is like admission! Also, if so silence should forbid even Devar sheb'Ervah, for he admits that she is forbidden! Rather, one witness is believed whenever the owner does not contradict him, even if the owner does not know. We did not need to teach that in all three cases that silence is like admission. Rather, we teach that we never say that his silence is truly denial, but he is silent because he will not lose. In Gitin, a worker is believed only if it was b'Yado, because a worker is obligated to guard. The Chazakah is that he guarded from Tum'ah. If it was not b'Yado, we assume that he wants to pain the owner. Alternatively, had it become Tamei, he needed to tell the owner immediately. One who did not work for the owner is believed even if he did not tell him at the first opportunity.

Defense (Ran): Really, we taught that silence is like admission in all three cases, but it depends on the witness' credibility. "O Hoda Elav" teaches that silence to a Kosher witness is like admission. Silence to a thief is not. This is a Chidush in Isurim. Even if one knows that his silence was not an admission, he is forbidden, for the witness is believed. We do not say so about money! It suffices if he was initially silent, even if nothing suggests that he knew.

Rosh (Gitin 5:8:13): Silence is like admission only if the witness says that the owner saw it or saw supporting evidence, like the case of eating Chelev. If not, he is silent because he does not know; one witness can forbid only if it was b'Yado.

Poskim

Shulchan Aruch (YD 127:1): If Reuven told Shimon 'your wine was poured to idolatry', and Shimon was silent, this is like an admission.

Rema: Since he is silent, it is forbidden even if another witness contradicts Reuven.

Beis Yosef (DH u'Mah she'Chosav Rabeinu she'Melamdim): R. Tam says that one need not be stringent even if he believes the witness like two. Rabbeinu Meir says that a Ba'al Nefesh should be stringent when the witness is trustworthy and we know that he does not seek to make him lose. R. Meshulam says that one should be concerned if the witness is trustworthy. Shmuel did not tell the blind man to ignore the witness. Avi ha'Ezri says that one with Yir'as Shamayim should not invent pretexts to discredit the witness and be Metaher one's Taharos. Tosfos (Kidushin 66a DH Nitme'u) says that if one makes a Shali'ach to guard or bring wine, it is b'Yad the Shali'ach and the owner trusts him like two. Even when the owner does not rely on the witness, it is forbidden to the witness.

Rema (3): Some say that if the matter can be verified, e.g. 'come, I will show you that a Nochri is pouring your wine to idolatry', one must be concerned.

Shach (24): All agree to this. The Gemara (Kidushin 66b) explicitly says so about a Ba'al Mum! This is even if the owner, or, according to Tosfos, one witness, contradicts him.

Rema: A minor is not a witness to be believed for Isurim.

Shach (31): The Rivash (245) says that the owner's silence is like admission. He believes one witness even if the owner does not know, like the Rashba. The Rema does not believe him in this case. He teaches that a minor is not believed, even when a witness is believed without admission. E.g., he says 'you know supporting evidence', or one witness contradicts him and the owner is away, or he says 'come see a Nochri pouring your wine to idolatry', or when there is no Chezkas Isur, or the witness says which of two pieces is Heter. However, we believe even a minor against Chezkas Isur if it is b'Yado.

Rema: However, if he is sharp and proficient in the matter, and there is supporting evidence for his words, one should be stringent if he testifies about Isur. Chachamim believed him to testify leniently about an Isur mid'Rabanan, such as Bedikas Chametz. If the Isur was Muchzak, he is not believed at all.