A strong case can be made that racial relations in the U.S. are deteriorating, whereas for most of the period after slavery’s end in 1865 they were improving, particularly in the middle of the 20th century and for some time thereafter. I liked Gerard Baker’s recent Wall Street Journal column where he suggested our obsession with defining everything in terms of race has led to problems, such as the rise of extremists of several racial backgrounds. Instead of moving toward a society where “race does not matter,” we are moving closer to one where “only race matters.”

This all came to mind recently when the Justice Department demanded that Yale University end what it considers admission preferences for African-American and to a lesser extent Hispanic students, thereby disadvantaging whites and Asians. This led me to ask the question: what would happen if it were illegal for a college to ask students or employees wishing to join its community about their race, or gather racial information in other ways, such as through mandatory in-person interviews, or required photographs from applicants?

My thinking has evolved from over 50 years of researching and writing about American race relations (much of my early scholarship related to the economics of slavery). Although vicious, hateful racial discrimination was rampant for generations after 1865, it started disappearing at an increasing rate in the mid-twentieth century: for example, lynching essentially stopped and Blacks became prominent in such high-paying, visible fields as sports and entertainment.

In a forthcoming essay, I argue that the period of greatest post-Emancipation rise in Black incomes relative to whites was between 1940 and the early 1970s—most of it before civil rights legislation had been passed or had become fully effective. In a competitive market-based economy, true racial discrimination, for example favoring a white job applicant over a more qualified Black one, can be costly—the employer is getting less output per dollar spent. Market principles in fact worked slowly but surely to lower discrimination. By the early 1970s, the median income of black households was around 70% of that of white ones, compared with about 50% as late as 1940.

By 2000, the black to white household median income had risen further, to about 80%. Moreover, racial assimilation has grown rapidly—the number of interracial marriages, for example, has soared. But since 2000, by some indicators, such as household median income, the previous narrowing of inequality has stagnated, at the very time universities increasingly moved away from “affirmative action,” involving strongly nudging campus decision-makers to encourage minority participation, to more militant forms of promoting “diversity and inclusion,” where race became a primary criterion in such things as admissions.

Hence schools like Harvard, Yale and many others are increasingly facing a real legal dilemma. The zeitgeist of universities, the set of ostensibly shared values, increasingly deviates from a traditional American ideal: achievement and rewards in life should depend more on individual accomplishments rather than one’s station in life at birth or on group characteristics which cannot be controlled, like skin coloration. The magisterial admonition of Martin Luther King that what is important is not the color of one’s skin but the content of one’s character is out of fashion, at least on college campuses.

If the courts start ruling against schools like Harvard or North Carolina in current lawsuits, one logical remedy of alleged racial bias would be to remove race completely from consideration by eliminating racial information. If we truly want a society where “race does not matter” in assessing human qualities, perhaps we need to make it difficult to take race into account. Arguably ignorance is bliss.

This is not to say “ignore the disadvantaged.” There are other ways to aid disadvantaged persons in our society, disproportionately members of minority groups, and give them an extra boost, besides looking at their skin coloration or other physical attributes, ways that on balance help the most disadvantaged minorities. Giving generous scholarship assistance to those who are poor is an obvious traditional one. Assisting struggling kids, many of them in racial minorities, with after school tutoring and enrichment programs is another. Eliminating preferential treatment of collegiate legacies in admissions (predominantly favoring whites) is a third.