What did Er do that was evil in the Eyes of Hashem?
Rashi and Targum Yonasan: After penetrating Tamar, he poured his seed on to the ground, 1 like his brother (in 38:9). 2
Ramban: He died on account of his unspecified sin or sins.
Rashi: Which he did in order not to spoil Tamar's beauty, should she fall pregnant.
Riva, Moshav Zekenim (to 38:9), Hadar Zekenim (to 38:9): Er and Onan had Bi'ah Lo k'Darkah (Yevamos 34a). Tamar removed her Besulim by hand, therefore she was able to become pregnant through her first normal Bi'ah [with Yehudah]. Alternatively, they had Bi'ah in the normal place, but without breaking the Besulim (Riva, Hadar Zekenim). R. Menachem asked why Er did not have normal Bi'ah the first time. It cannot impregnate! (Perhaps he was concerned lest she was not a virgin, or she removed her Besulim by hand! - PF)
Why were Er and Onan punished? They were not old enough to be liable [bi'Yedei Adam], and Shamayim does not punish below the age of 20 (Shabbos 89b)! Rashi (to 5:32) writes that prior to Matan Torah, one was liable to [Divine] punishment only from the age of 100!
Moshav Zekenim: Shamayim judges a person according to his Chochmah. If a minor has Chochmah like a 20-year-old, 1 he is punished. 2 We find that Eli wanted to punish (kill) Shmuel for ruling in front of him, even though Shmuel was only two years old. Also here, even though Er and Onan were only eight or nine, since they were able to father children, they are considered adults. 3
Oznayim la'Torah: They did not die only because of what they did, but predominantly 4 because of the sin of Yehudah, who did not finish the Mitzvah that he began. 5
Chacham Tzvi (49) and many others challenge this (that Shamayim does not punish below 20). Teshuvas Chasam Sofer (YD 155) limits it to special cases, e.g. Onesh Dor ha'Midbar. Tzitz Eliezer (14:20,2) brings many Poskim, and proves that the Halachah does not follow this Agadah.
Gur Aryeh (to 5:32): This exemption applies only to Divine judgment in a circumstance that might warrant it, such as at the Flood. However, when the Torah mandates the penalty of Misah bi'Yedei Shamayim for a specific transgression, such as that of Er and Onan, no judgment is necessary, and that minimum age does not apply. 6
Hadar Zekenim (to 38:10): The nation is punished bi'Yedei Shamayim based on its majority only from the age of 20 (e.g. for heeding the Meraglim), but individuals, even below 13; and similarly for good. (For good, minors are rewarded even as a nation. The world stands on the Torah of children (Shabbos 119b). - PF)
Riva (to 38:10) and Hadar Zekenim (to 38:10) support this from Shlomo. He was at most 12 when David told him, "If you abandon Hashem, He will abandon you forever!" (Divrei Hayamim I 28:9).
The Rosh asks so also about Tamar (even though she was a minor, Yehudah planned to execute her) and answers for all three, since they could have children, they are like adults. (How could Yehudah know that she was old enough to bear children? Perhaps she was pregnant with a Nefel! - PF. The Rosh's text in Sotah (10a) must say "I am a minor," like Hadar Zekenim (to 38:14) says; unlike our text. According to Rashi (to 38:24, she was Shem's daughter), she was an adult, since Yaakov returned to Eretz Yisrael in 2207, 49 years after Shem died.)
Refer to 38:12:1.1:1 and the note there. Also compare to 38:1:1:2.
See Oznayim la'Torah DH 'Bechor Yehudah' in the name of the Maharsha.
Gur Aryeh (ibid.): The same distinction applies today, when the minimum age for Divine punishment is 20 (Shabbos 89b). Would a young man age 13-20, who eats Chelev, be exempt from Kareis?! Rather, he is only exempt from Divine judgment for particular circumstances (e.g. the generation of the Meraglim - Rashi to Shabbos 89b).
Why does it say that Er was Yehudah's Bechor?
Malbim: Malchus Beis David was proper to come from him. If not [for his great potential], he would not have been bad in Hashem's eyes.
Ha'amek Davar: Malchus Beis David should have come from Yehudah's Bechor. Er was not worthy of this, so Hashem killed him.
QUESTIONS ON RASHI
Rashi writes: "'Evil in the eyes of Hashem' - ... Why [did Er do this]? So that [Tamar] would not become pregnant, diminishing her beauty." But elsewhere (Rashi to Sotah 10a), Rashi writes that Tamar was a minor; if so why was this a concern?
Gur Aryeh (to 38:15): When Tamar was first married to Er, she was a minor; but then she reached maturity.
Although today we assume that a woman cannot conceive before age 11 or 12 (Yevamos 12b), in earlier generations a woman could give birth at 8 or even 6 years old (see Sanhedrin 69b). (CS)