Divrei Eliyahu: A man needs (a) to serve Hashem, and (b) to learn Torah. He should not turn his heart to other matters. They tell Hashem "veer from us" - we will not serve You, and we do not desire Your ways (Torah). This is why the coming verse calls Him Shakai - there is Dai (enough) in His service alone, and then man will not lack anything. "His'halech Lefanai ve'Hye Samim" - when you go in front of Me, you do not lack anything.

Malbim: You cannot say that they succeed because they are not Resha'im - they tell Hashem to go away, for they do not desire to know His ways (Chesed, mercy and Chaninah). They are evil Bein Adam l'Chavero, and also Bein Adam l'Makom.