Why will he thank Hashem with a straight heart?
Radak: [The Mitzvos] will teach me to thank You with a straight heart, when I learn and understand that they are all Your righteous Mishpatim.
Malbim: For Mishpatim, I will thank with a straight heart. "Yashar" refers to Binah. Yosher Levav is one whose heart's understanding is vast, to the point that he grasps everything via his Binah. It is a sharp power that passes through all matters to understand a matter amidst a matter.
What are "Mishpatim"?
Radak (1): Mishpatim are monetary laws Bein Adam l'Chavero.
Malbim: They are Mitzvos that are totally understood.
Why does it say "Mishpetei Tzidkecha"?
Radak (1): Tzedek is the justness of Mitzvos; they were said justly.
Malbim: This refers to when he will understand their justness and straightness.