Why is the criterion for returning an aveida to a Talmid Chacham based on his reversing his shirt? Is it not possible that a non-Talmid Chacham could simply be fastidious?
It is definitely possible that a non-Talmid Chacham can be fastidious. However, this is merely a sign of whether or not this person is indeed a Talmid Chacham, when we already think that he is, based on his vast Torah knowledge. A similar Gemara is mentioned in Bava Metzia 23b, where it says that we only return an Aveida b'Tvius Ayin to a Talmid Chacham who is careful to only lie about three things (which one is permitted to lie about). This does not mean that only Talmidei Chachamim tell the truth! It merely means that when someone is a Talmid Chacham and we see he is scrupulous about his appearance to others and tells the truth, this is the kind of man that one can return a lost object to b'Tvius Ayin.
All the best,