Regarding Sukah decorations, one may make a Tenai to prevent the decorations from becoming Asur ("I am not exclusively designating this material for the Sukah, for I retain the ability to use it during Bein ha'Shemashos"). Thus, before hanging up the fruit (or even after hanging up the fruit, but before Bein ha'Shemashos of the first day of Yom Tov), one may make a Tenai. Then, no matter what, one may remove the fruit.

Regarding one who did not make such a Tenai, the MISHNAH BERURAH (OC 638:24) cites the TAZ who suggests that when a person hangs up embroidered sheets to decorate his Sukah, we assume that his intention was not to leave them there all the time (lest they be stolen, or rain come and ruin them), and he was not "Mekatzeh" them for the exclusive purpose of Noyei Sukah, even though he did not stipulate as such. It seems that the same logic applies to fruit which can spoil or attract bees; we assume ("Anan Sehadi") that it was not his intention -- when he hung the fruit before Yom Tov -- to leave them there at all times if they spoil or attract bees, and it is as if he explicitly stipulated that they are not exclusively designated for the Sukah (and he may remove them even if they do not spoil).