Do you remove these small stones, pebbles and what nots on your tire treads?

Will letting these be damage the tires?