This is a very important point. Reb Baruch Ber Leibowitz (Bircas Shmuel Kidushin 1) quotes Reb Chaim Soloveitchik to answer this question. Reb Baruch Ber explains Reb Chaim at length, and here I will try to write down the basic points of his explanation.

We learn from the Pasuk of Ki Yikach Ish that the act of Kidushin must be done by the man. We also know that for the Kidushin to be valid we need to know that the act of Kidushin (Kesef, Shtar or Bi'ah) was done for the purpose of Kidushin.

Since the Kidushin is only valid if we know that the act of Kidushin was done for Kidushin, Reb Baruch Ber explains that the way that we are made to know that the act of Kidushin was for Kidushin is also considered to be a part of the act of Kidushin. This being the case, we can understand that we must become aware of the fact that the act of Kidushin is for Kidushin through the actions or words of the man.

It follows, therefore, that in the case of Asukin b'Oso Inyan, the Kidushin is only valid if the man says or does something that through his acts or words we understand that the ensuing act is for Kidushin. In our case, the man said nothing that gives us to understand that the act of Kidushin was for Kidushin, so the Kidushin is not valid.

Reb Baruch Ber explains further, that even if the man agreed verbally with the woman's suggestion that he be Mekadesh her, we may still not have a valid Kidushin. This is because he was not Mekadesh her relying on the fact that we know that the act is a Kidushin from his verbal agreement to her suggestion that he be Mekadesh her, rather he is Mekadesh her relying on the speech of the woman. This being the case his confirmation of his agreement is not considered a part of the Kidushin, and we are left with a problem of Ki Yikach that the man must make the act of the Kidushin not the woman.

I hope this is clear, if not please let me know,

Dov Freedman