What is the significance of Hashem remembering the Bris?
Rashi (59-60): This is the Bris that I had with you in your youth. I am unlike you - I will not annul the Bris and not remember it.
Radak: You annulled Medinas ha'Yam Bris, but I will remember it and not annul it. It is the Bris of Har Sinai.
Malbim: Anything that Hashem does cannot be totally Batel; it must be forever. Similarly, every good Midah that came from His mouth, even on condition, He does not retract.
What is "Yemei Ne'urayich"?
Radak: It is from Egypt until Yisrael came to Eretz Yisrael.
Malbim: It is when I took you out of Egypt.
Why does it say "va'Hakimosi Lach Bris Olam"?
Radak: Just like you accepted it forever - "Lanu ul'Vaneinu Ad Olam" (Devarim 29:28), so I will establish it forever. It will not cease. This will be after you receive your punishment, remember your ways and be ashamed of them.
Malbim: I will make a new Bris not dependent on any condition. It will last forever - "v'Charati Es Beis Yisrael... Bris Chadashah; lo cha'Bris Asher Karati Es Avosam... Asher Hemah Heferu Es Brisi; Ki Zos ha'Bris...; Nasati Es Torasi b'Kirbam...; Ki Kulam Yed'u Osi" (Yirmeyah 31:30-33). Via the great exiles and problems that they will have, and the great wonders that I will do for them, Toras Hashem will be engraved on the tablet of their heart. The Yetzer will not rule over their hearts, and they will have no Safek about His Hashgachah and His constant clinging with them. There will not be any sin at all.