Gemara

17a (R. Avahu): When we add judges (because one judge could not decide), we can make a Beis Din with an even number of judges l'Chatchilah.

Question: This is obvious! (An even number decided, and we add two more.)

Answer: One might have thought that we count the indecisive judge, and if he later gives a reason, we heed him. R. Avahu teaches that this is not so. It is as if he is not, and we do not heed him, even if he gives a reason.

40a: If 12 judges are Mechayev and 11 are Mezakeh, or even if 11 are on each side and one is unsure, or even if 22 agree either way and one is unsure, we add more judges, two at a time, until we reach 71.

If 36 are Mezakeh and 35 are Mechayev, he is exempt. If 36 are Mechayev and 35 are Mezakeh, they debate until a Mechayev changes his mind.

Rishonim

Rambam (Hilchos Sanhedrin 8:2): If in a Beis Din of three, one judge is unsure, whether the others agree or argue, we add two judges. The five debate. If at least four decide, we follow the majority, even if now the same or a different judge is undecided. If two acquit and two convict and one is unsure, we add two judges. We keep adding like this until 71. If 35 acquit and 35 convict and one is unsure, they debate with him until he decides. If he does not decide and none of others retracts, the Din is a Safek; we leave the money in its Chazakah.

Rebuttal (Ra'avad): The Gemara says that the undecided judge is not counted. If he gives a reason, we do not heed him. After we add, there are only four judges. The Rambam erred about the Tosefta, or he saw a mistaken version. According to him, if one of the four is unsure, we follow the majority of the other three, for a Beis Din remains. If all decided, two for Chiyuv and two for Zechus, and no one retracts, we leave the money in its Chazakah. If so, we never reach 71 judges in monetary cases. The Tosefta says that we do! We need a verdict, which is only when there is a majority. The Rambam omits from the Beraisa 'if one is Mechayev and one exempts and one is unsure, we add judges', for until now we only add one, i.e. the undecided judge is one of the two added. This is a good reason.

Kesef Mishneh: The Gemara says that we do not count or heed reasons of the undecided judge. The Ra'avad holds that this is even for monetary and capital cases. After we add, there are only four judges. (The Rambam holds that this is only in capital cases, for then he cannot give a reason for Chiyuv.) The Ramach asks, how are there five judges to debate? It is as if the undecided one is not! He answers that if he gives a reason for Chiyuv in monetary cases, we heed him. He is no worse than a Talmid. Monetary cases are (always) Zechus, for Chiyuv for one is Zechus for the other. Do not say that when we add two judges, now we must finish with four or five, for initially they sat with intent to finish with three. This requires investigation. The Rashba (3:356) and several Gedolim hold like the Ra'avad.

Rashba (3:356): If one was undecided and we added two judges, even if one of them is undecided, if three decided, we follow the majority. We cannot count the one who was initially undecided, for it is as if he is not. This is even for monetary cases, unlike the Rambam, who says that it is only for capital cases. A Mishnah lists the differences between monetary and capital cases, and this is not among them. If a Beis Din of three remains, we follow the majority. For capital cases, at least 23 must remain. It seems that Rashi agrees. The Mishnah discusses adding to a Beis Din until there are 71 only when they were evenly divided at each stage (or there was a majority of one for Chiyuv in a capital case, which is not enough to convict).

Rambam (9:2): If a small Sanhedrin judged a capital case, and 12 judges convict and 11 acquit, or one was unsure, even if the other 22 agree, we add two judges. It is as if the undecided judge is not, for he cannot give a reason to convict. After adding, there are 24 judges other than the undecided one. If 12 (some texts say 13) acquit and 12 acquit, he is acquitted. If 13 convict and 11 acquit and one is unsure, he is liable, for two more are Mechayev. If 12 convict and 12 acquit and one is unsure, we add two. We keep adding until there is one more Mezakeh or two more Mechayvim. If they were even and one is unsure, we keep adding until 71.

Kesef Mishneh: The Ra'avad says 'this is impossible.' I.e. since it is as if the added judge is not, we do not have five opinions! Also, the Rambam said that if 22 convict and one is unsure, we add judges... If 13 convict and 11 acquit and one is unsure, he is liable. We should add judges, just like when 22 agreed and one was unsure! The Rambam holds that he is counted among the judges. Even one who convicted may retract to give a reason to acquit, all the more so one who was unsure may retract to give a reason to acquit, and he is counted with the judges! The Gemara meant that if he give a reason to acquit for Chiyuv, we do not heed him.

Nimukei Yosef (3a DH Omar): We learn from here that one who was undecided cannot later judge the case. The Rambam says that this is only in capital cases, but in monetary cases he can retract and judge. Only in capital cases he cannot later be Mechayev, so he is not part of Beis Din. There is no Mechayev in monetary cases, for Chiyuv for one is Zechus for the other. Rashi says that one who was undecided cannot later convict, but he can exempt. Even the murderer can give a reason to exempt himself!

Poskim

Shulchan Aruch (18:1): If in a Beis Din of three, one judge is unsure, whether the others agree or argue, we add two judges.

SMA (4): If one cannot decide, it is as if he did not sit with them to judge. We say that had a third sat with them, perhaps the others would have agreed with him. Once we add, it suffices that three decide and agree, for initially we intended that three decide.

Shulchan Aruch (ibid): The five debate. If at least four decide, we follow the majority, even if now the same or a different judge is undecided. If two acquit and two convict and one is unsure, we add two judges.

SMA (6, citing the Maharshal): If three agree and two are unsure, we follow the majority. If two agree, one dissents and two are unsure, we add two, for there is not a majority against those who are unsure.

Rebuttal (Bach DH Kosav ha'Rambam): The Beis Yosef says that the Mishnah and Rambam hold that we add only when the judges who decided are equally divided. If so, why didn't the Rambam teach that when two of five judges are undecided, we follow the majority of the three who decided, even if when all three do not agree? Rather, since one was unsure when we added two, we intended to have a verdict with four, therefore, four must decide. Whenever we add two, we require a verdict with two more than the number who decided before we added. This is unlike Rashi. The Yerushalmi says so. It asked why it is not enough to add one judge. One opinion said that perhaps he will be undecided, but if we add two, even if one is undecided, we will have (a majority among) three who decided. This was rejected. Rather, when we add two, we require a verdict with two more than the number who already decided. The Rambam and Tur rule like this.

Defense (Shach 2): Also the Rambam rules like the first opinion in the Yerushalmi. This minimizes the argument between the Rambam and Rashba.

Ohr Some'ach (9:2): The Rambam holds that in capital cases we can make an even Beis Din, for when half acquit and half convict, we acquit due to "v'Hitzilu ha'Edah". We do not make an even Beis Din for monetary cases, for then we do not get a verdict. If so, we must say that when we add two there are five, including the judge who was unsure. The Ra'avad holds that also for monetary cases we can have an even Beis Din. If there is no majority, we add judges. The Rashba says that we do not make an even Beis Din l'Chatchilah. When we add, we need only three to decide, like initially. The Rambam rules like the Yerushalmi. No one argues there. It asks why we must add more judges when (in addition to the initial undecided judge) one of the two added judges is undecided, but omitted the Tosefta that says this. R. Ila answered that once we added, we need a verdict with four. The Ra'avad holds that the Tosefta say that we add when there is no majority. Really, the Tosefta discusses a party who requests to add judges (see Rashi Devarim 1:12) before the verdict. The Gra explains this way.

SMA (7): When one of the new judges is unsure, we do not say that it is as if the initial undecided judge is still unsure, and there is not a majority against those who are unsure.

Shulchan Aruch (2): If the judges were half-half and one is unsure, we add two, until 71. If 35 acquit and 35 convict and one is unsure, they debate with him until he decides. If he does not decide and none of the others retracts, the matter is a Safek and we leave the money in its Chazakah.

Beis Yosef (DH u'Mah she'Chosav Omar): It seems that the Rambam learned from 40a that we add two at a time.

Bach (DH u'Mah she'Chosav Yosifu): The Mishnah says Stam that we add judges, which connotes two. The Beis Yosef says that we learn from capital cases. Semag brings from the Yerushalmi that we add two, and a Tosefta explicitly says so.

Ohr Some'ach (9:2): The added judges debate with each of the old judges. If we added only one, an old judge might slacken from delving into the case again, for 'the new judge is no better than me.' Ba'al ha'Itur says each party picks another judge.