Measuring attitudes towards African-Americans using machine learning and textual data
Identifying determinants of cultural distance to overcome barriers
Historically, ethnic tensions have proved a significant barrier to economic and financial development; for example, by impacting public-goods provision, reducing social-capital formation and increasing the likelihood of conflict. By applying machine-learning methods to a collection of digitised newspapers dating from when North America was a developing country, this project aims to uncover both the determinants of ethnic tensions and reasons why attitudes change; the findings will have important lessons for the economic development of countries more generally.
NORTHERN AMERICA
Publications
Gender Norms Do Not Persist But Converge Across Time
Abstract
We investigate the evolution of gender norms for 160 years in the US. Socioeconomists have posited two fundamental and widely debated theories on the evolution of cultural norms across time. One argues that cultural norms should converge across time as economies become more advanced and integrated, whereas the other states that cultural traits are highly persistent, passed down from generation to generation. These theories remain untested due to a lack of granular and high-frequency data over a longer time period. We develop a novel unsupervised machine learning methodology and apply it to 193 million pages of local newspaper text to produce localised attitudes towards women on four dimensions: career vs family, attitudes towards abortion, attitudes towards feminism/suffrage, and violence against women. We establish novel facts on the evolution of attitudes across time. First, attitudes are less persistent than the existing literature hypothesises. Second, the persistence varies considerably across regions and dimensions. Third, attitudes exhibit cyclical patterns. Fourth, regional variation in attitudes decreases considerably over time and has fallen between 64% to 79%. Fifth, a decrease in transport costs that allows for easier information sharing is associated with a homogenisation of the norms.