Why do the rabbanan take the chamor since it's not klei umnes. Lemaisa, this is how he makes a living? Without it he has no way to live. What is the mekor for davka klei umnes? See Rambam erechin 3:18. Have you seen anyone discuss this?
Lulei demistephina I would say, the derasha of hachayehu as Rashi learns is hachaye- "hu". An uman is only as good as the tools that he has. Without them he isn't an uman, he is like anyone else. However, a chamor etc. just has nechasim, those we take away, since it doesn't change the persons value, just what he has in physical value.
Keep up the great work of being marbits torah berabim,
Rabbi Uri Neumann
The Sefer ha'Cchinuch writes that although a Chacham cannot teach without Sefarim, we still take his Sefarim. It seems that the distinction of Klei Umnus is an item that is used exclusively for the Umnus.
However, I find your Sevara interesting, and I did not find any other explanation in the Mefarshim.