I, for example, am unrefined, but rude. I won't claim to know about intelligence or lack thereof. Am I "uncultured"?
You say you're an outsider and don't fit in with the crowd, but there are plenty of such outsiders. Does that make you not "uncultured"? If so, then there's a very large number of people who aren't "uncultured," even though there should be very few of them. I suppose there's your group, an "uncultured" group, and "normal people" who don't fall into either of these extremes, but they all live and think in a similar way. They study and work, live off their relatives, have preferences and dislikes. And what's the difference then?