The Gemara records an argument between Reish Lakish and Rebbi Yochanan involving a person who transgresses a "Lav ha'Nitak l'Aseh." One opinion maintains that the transgressor never receives Malkus for transgressing the Lav as long as he can still perform the Aseh ("Bitlo v'Lo Bitlo"). The other opinion argues that the person is in full violation of the Lav and can receive Malkus for it even if it is still possible for him to fulfill the Aseh. However, if he does fulfill the Aseh he can avoid the punishment of Malkus ("Kiyemo v'Lo Kiyemo").
The Gemara asserts that the only case in which Malkus can be given, according to the opinion that maintains "Bitlo v'Lo Bitlo," is a case of "Shilu'ach ha'Ken." The Gemara goes on to explain why Malkus cannot be given in other cases. In the case of a man who rapes a woman, marries her (as required by the Torah), and then divorces her (which he is prohibited to do, and he must remarry her), the Gemara explains that it is not possible for the sinner to receive Malkus since he cannot prevent himself from fulfilling the Aseh at some future date, for various reasons.
Why is it not possible to prevent himself from fulfilling the Aseh at a future date in the following manner: When a man makes himself a Kerus Shafchah, he is prohibited from marrying a Bas Yisrael (Devarim 23:2). The Mitzvas Aseh of marrying the woman he raped does not override this Lav, as the Gemara says in Kesuvos (40a). Therefore he can no longer fulfill the Aseh. Why does the Gemara not say that Malkus will be given, according to those who hold "Bitlo v'Lo Bitlo," in the case of a man who raped a woman and then made himself a Kerus Shafchah?
This is a good question. It is addressed by the Aruch la'Ner in Makos there.
1. The Aruch la'Ner answers that according to Rashi in Kesuvos (40a), the Aseh is Docheh the Lo Sa'aseh, but we force the woman to say "I do not want to marry him." According to Rashi, the question is answered, because if she says that she does not want to marry the man, then it is not he who annulled the possibility of doing the Aseh (Bitlo), but rather she was Mevatel it. (We may add that she was not actually Mevatel the Aseh, but rather since she does not want to marry him, the Mitzvas Aseh for him to marry her does not take effect at all, and thus it is considered as though he fulfilled the Aseh.)
However, the Aruch la'Ner points out that according to the other Rishonim there who hold that the Aseh of marrying the woman does not override the Lo Sa'aseh for a Kerus Shafchah to marry even when she says that she wants to marry him, the question remains.
2. The Aruch la'Ner answers that since the man was already commanded not to mutilate himself and make himself into a Kerus Shafchah, the Lo Sa'aseh of divorcing the woman does not take effect when he mutilates himself, because "Ein Isur Chal Al Isur." Therefore he will not receive Malkus for violating the latter Lo Sa'aseh.
However, the Aruch la'Ner himself points out that this does not seem to be consistent with the Gemara, because the Gemara says that if he kills the woman, he should receive Malkus, but the principle of "Kam Lei bid'Rabah Minei" prevents Malkus from being administered. We see from the Gemara that we do not apply "Ein Isur Chal Al Isur" and say that the Isur of "Lo Yuchal Shalchah" does not take effect on the Isur of Retzichah (see also Tosfos here).
3. It seems, therefore, that the answer is as follows. The action that is Mevatel the Aseh must include an action that is directly associated with the object with which he must fulfill the Aseh. Therefore, it is considered an act of Bitul of the Aseh only when he kills the woman (for she is the object of the Aseh), as the Gemara mentions, or when he divorces her, or when he makes a Neder prohibiting this woman from marrying him.
If, however, he mutilates himself, then even though this causes him to become prohibited to marry the woman, nevertheless since the action of making himself a Kerus Shafchah is done to him and not to her and relates to her only indirectly, it is not considered a Bitul of the Mitzvah Aseh to marry the woman. (A similar line of reasoning is mentioned by the Chemdas Shlomo in Pesachim 25a.)