I have no doubt whatsoever that there are millions of peopl…

I have no doubt whatsoever that there are millions of people who are willing and able to self-organize. It's very hard to make the case that they shouldn't be allowed to.
Are we as a species anywhere near a level where most people are willing and able? Except for the predators, obviously?
Far from it.
But that doesn't matter. We didn't wait for everyone to think that flying was possible until we allowed someone to try. People just tried it in spite of the masses screaming that it is impossible.
I think the trick is to have EXCLUSIVE societies that take members based on reputation, skill and merit - and kicks them out if they break the rules.
The beauty of freedom is that it's not a one size fits all where everyone is forced under one roof.