You can please some of the people some of the time
“Criticism is something you can easily avoid — by saying nothing, doing nothing, and being nothing.”― Aristotle
As the modern saying goes "you can please some of the people some of the time, but not all of the people all of the time.”, and the principle is the same as Aristotles; the more you do, the more you can expect someone to have an opinion about it. In that case, why would you be fearful of critisicm at all.
If anything, it probably means you are doing something meaningful for many and displeasing at least someone is just a natural part of that process. The other option is to try and please everyone and up achieving nothing.
But the biggest disservice you can do to yourself is to withdraw from life because you are afraid of someones opinion. Nothing good ever came from doing that!