Yeah I've done it. If you select one of your own settlements than Shank will double check with you first if you want to go through with it, and if the settlement has nobody living there than your raiders will show up and be a little disappointed that there's nobody to kill. Bear in mind though that claiming even a single settlement for the raiders will immediately turn Preston against you and irreparably damage your relationship with him, so if you're a minuteman at heart and plan to always be one than don't commit to the raider path. I personally complete the Nuka World questline and then betray and wipe out the raiders after getting their perks before even meeting Preston, so he never finds out about it.

^ Thank you @MadMikeRyan

That's exactly my plan: do the job, get the perks and the achievements, clear the area from the raiders and avoid Preston in the meantime.

Unless the dark side prevails in the meantime... XD

If you decide to add turrets to the raider camps, make sure to remove them if you want to eliminate the raiders later. If you don't then it won't be possible and you'll never complete the quest.

^ Yes, I read that in the wiki, but THX anyway :)