As a Westerner this does not bother us, we are not bound by the "social" rules Indians have. For us it is easy to just brush gossip off of our shoulders and go on live our lives. You want to talk about me behind my back... just go ahead if it makes you happy!
But for the people here it is an every day struggle. What will they say... what will they think... As a woman for example you cannot just stop and talk to a random man on the street. If you do, someone will notice and by the time you get home the whole neighborhood will know. When you buy something expensive people will see and wonder what you did to get the money together. And so forth, and so on.
The whole concept of helping your neighbors is still very important and if there are nasty stories going around about you and your family, people might not be willing to help you anymore. And being alone, without a community is for an Indian one of the worse things they can imagine.