Good old Haley Barbour, governor of Mississippi. He helps keep life in perspective. When he defended Virginia Governor Robert McDonnell’s infamous recollection of the Confederacy that somehow failed to mention slavery, Barbour called the issue a “nit” — merely an insignificant matter.

But while Barbour’s dismissive view of slavery may be popular in Mississippi — he’s the governor, after all — it’s nonetheless a little raw for most of the country. One hundred and fifty years after the Civil War began, America’s entrenched segregation is more refined, but no less real. We accomplish racial separation simply through zoning regulations and law enforcement. Zoning keeps poor people, especially when they’re not white, out of middle-class and wealthy towns by concentrating them in jurisdictions laden with cramped housing and under-funded schools.

This technique accomplishes two things. First it assures that poor minorities won’t live in affluent white neighborhoods. Second, since lower property values mean less property tax revenue to fund public schools, the kids raised in those homes aren’t as likely to get a good education. That in turn means that when they’re adults, their own kids will probably live in areas with under-funded schools too.

Another systemic problem is that across the nation the police tend to target African-American and Latino youths at traffic stops. Prosecutors, likewise, tend to throw the book at black suspects, thus guaranteeing a criminal record for potential employers to peruse.

And so blacks, and many Latinos, grow up poor and stuck in the segregation cycle. The median net worth of white households at last count was $97,900. For black households, the median net worth was $2,200, or next to nothing. Minorities account for more than half of the 50 million Americans who lack health insurance, and one in four blacks and Latinos live below the poverty line.

The Great Migration that brought so many African Americans up North, lured by plentiful industrial jobs, has now reversed. Those good jobs at unionized factories are gone, thanks to technological advances and corporate America’s predilection for shipping jobs to China and other low-wage nations with feeble labor laws. Liking to eat occasionally, and not having many other choices, many blacks are now moving from Northern to Southern states.

Detroit, once the hub of black upward mobility, has shriveled. Atlanta’s more promising now. And New York City, the single biggest center of African-American culture, has for the first time seen its black population decline.

Maybe racial discrimination will truly fade someday, but don’t count on it for now. Even the United Nations recently gave us low marks for our segregated schools. From disproportionately harsh prison sentences to schools that are separate and not equal, our society tilts invariably against Americans of color. In such a setting, even Haley Barbour is just another nit.