Measuring attitudes towards African-Americans using machine learning and textual data

Identifying determinants of cultural distance to overcome barriers

Historically, ethnic tensions have proved a significant barrier to economic and financial development; for example, by impacting public-goods provision, reducing social-capital formation and increasing the likelihood of conflict. By applying machine-learning methods to a collection of digitised newspapers dating from when North America was a developing country, this project aims to uncover both the determinants of ethnic tensions and reasons why attitudes change; the findings will have important lessons for the economic development of countries more generally.

NORTHERN AMERICA

The challenge

Cultural distance between groups has important economic consequences. Ethno-linguistic fragmentation, for example, has been shown to reduce the provision of public goods, reduce social capital formation and increase the likelihood of conflict. This could ultimately reduce the gains to ethnic minorities of ‘moving to opportunity’ if the gains from neighbourhood effects are reduced by the negative economic consequences of increased social tensions. In addition, these outcomes worsen when cultural differences across ethnic groups are greater.

The intervention

Measuring cultural distance is crucial but has received limited attention in literature. This research focuses on attitudes of ethnic groups towards other ones; more specifically on attitudes towards African-Americans and other ethnic minorities in the USA over a long period of history. The study does this by examining a historical collection of digitised newspapers, using machine-learning methods to identify attitudes from textual data. The measures will show how attitudes evolve across states and counties over time and the main determinants in their shifts.

The impact

Historically, ethnic tensions have been an important barrier to economic and financial development. This research hopes to shed light on ethnic frictions at a time when the US was a developing country, as well as the determinants and shifts in these attitudes in relation to the economic and financial development of the nation. The findings will have important lessons for the economic development of countries more generally.

Publications