Such a settlement will attract pot-grow hippies and anarchists and generally people who may reduce our chances of being taken seriously
We can limit (not ban) the inclusion of such people in our initial set. We can limit the ways we recruit to limit this issue. We can build the settlement around other ideas (such as a collaborative live-in learning and project centre, often informally called "The Monestary") that are of greater interest to more serious people.