If it's too big and too many I would take them out. Otherwise, they're good to have for cleaning up uneaten food and waste. It's only when you have a big one that ran out of food to eat and start tasting your corals.
I'll leave them alone sometimes, but if they're about an inch long, then I'll take them out with tweezers.
Thanks guys. Dong have fun taking your worms for a walk lol