Rashi: Their hearts are unlike their mouths. Even so, Hashem is merciful and always atones for their sin, and does not destroy them.

Radak: It would be proper not to pardon them at all, since their hearts are not proper. Everything follows the intent of the heart. [He pardons] because people see the revealed Teshuvah, and do not know the hearts. If He will not pardon them at all, they will say that Teshuvah does not help.

Malbim: Also the thoughts of their hearts were not proper with Hashem. They were not faithful to His Bris, just they thought that Hashem does not know their thoughts, and they trick Him with their mouths - "b'Fiv uvi'Sefasav Kibduni v'Libo Richak Mimeni" (Yeshayah 29:13).