Technically there is a big difference, but people mostly mean the same thing: getting rid of the stump. The difference is how. Removal is much more intrusive and involves digging up the whole ground to remove the extensive root system. This involves more time, work and leaves a big hole affecting a wider area until it’s filled in.
Stump grinding is much less intrusive, a lot quicker and also less harmful to the surrounding environment. This process involves arborists using a special machine to grind the stump down to wood chipping’s that can be used as mulch elsewhere.
The short answer is no. If the tree was removed for simply cosmetic reasons (for example blocking the light or you just don’t want it there anymore), or if it was removed for safety reasons (like it was encroaching on property or falling down) then you are generally free to leave the stump behind but slowly over time it will decay and start to rot. If the tree was unhealthy and had disease then then rot and decay will start much quicker and there are lots of problems when this happens.
As mentioned above, grinding is much less obtrusive requiring much less of the ground to be accessed. The chipping generated by the grinder can be used as mulch, and are obviously organic so cause no problems. Stump removal can require going into other businesses or residents property, affect nearby water and other plants and bushes.
Over time any stump will rot. Rotting attracts pests and fungus which can be harmful to us as well as other plants. Carpenter Ants and Termites can set up home and be damaging to your property. Fungus and also smells and can cause human health issues so grinding the stump away completely bypasses all these problems permanently.