The Mishna told us that a person who comes to Bet Din and admits that he was mefateh someone is believed and owes the k'nas.
Why don't we apply the rule: "Ayn adam oseh es atzmo rasha," a person can not make himself into a rasha by his own admision?
The Mishnah writes that one does not pay the Kenas on his own admission for Modeh b'Kenas Patur. However he does pay the Boshes and the Nezek, since that is compensation for damages and not a Kenas.
The Acharonim discuss why we ever believe someone who admits to a liability (Hoda'as Ba'al Din) for one cannot testify against himself. The Ketzos haChoshen (34:4) brings in the name of Mahari Ibn Lev this rationale to prove that a person is not actually "believed that he owes money." Rather he creates a new liability with his admission.
The Ketzos disagrees and says that when it comes to an admission of liability one is believed. He cites as the source Shemos 22:8.
Dov Zupnik
Tosfos in Bava Metzia (3b DH Mah) explains that admitting to a liability is not called making oneself a Rasha if such an admission is necessary to prevent himself from sinning further. Under such circumstances, his admission is an act of Teshuvah rather than an act of declaring himself a sinner.
This certainly can be applied to a person who admits to owing money to his friend, since his Teshuvah is not full until he returns the money that he rightfully owes his friend. (This logic cannot be applied to obligations of Kenas, since he has no obligation whatsoever to pay a Kenas until Beis Din rules that he is obligated to pay it.)
(See also Insights to Kesuvos 19:1, where we discuss the opinions that rule "Ein Adam Meisim Atzmo Rasha" only applies when a person disqualifies his future testimony in court by being branded a "Rasha," see Rashi 18b DH Ela she'Amru.)
-Mordecai Kornfeld