Rashi: Every Aniyah is an expression of a loud voice. The 'father' of such expressions is "v'Anu ha'Leviyim v'Amru" (Devarim 27:14).

Radak: It is an expression of speech, like "v'Anisa v'Amarta" (Devarim 26:5). I will say Your words to others, and teach them that all Your Mitzvos are Tzedek.

Radak citing ha'Mesorah: It is an expression of praise; my tongue will praise Your words, for all are Tzedek. Also two others are an expression of praise - "va'Ta'an Lahem Miryam" (Shemos 15:21), "[Al] Ta'an Kesil" (Mishlei 26:4).

Malbim: Simple speech is attributed to the lips. The tongue refers to elevated speech of understanding. After my lips will express praise, my tongue will speak about Your words. After You teach to me the Chukim whose reason is not known, I will understand via my reasoning the reasons for the Mitzvos, and I will answer that all of them are Tzedek. Before, he said "Kol Mitzvosecha Emunah" (86); now he recognizes their reasons, that they are Tzedek based on intellect, and not only Emunah.