To whom does "mi'Kol Asher Hayu Lefanecha" refer?
Malbim: It is Shlomo, that I took the kingship from him due to his sin. He built Bamos for his wives; you made gods for yourself! He did not protest against his wives due to love of them. You did not do so due to love of women, rather, "to anger Me"!
What is "v'Osi Hishlachta Acharei Gavecha"?
Radak: You did not turn to Me this way or that way, like one who casts an object in back of him. He does not face it at all.
Radak citing Targum Yonasan: You distanced My Avodah from opposite your eye.
Malbim: You abandoned Hashem and His Avodah.
Brachos 10b: This is one who eats and drinks, and afterwards prays. We read Gavecha like Gei'echa (your pride). After he is haughty, does he accept Ol Malchus Shamayim?!