Big Data Accountability and the Challenge of Institutional Discrimination

Discrimination can include comments, actions or decisions that make people feel unwelcome or uncomfortable, based on their identity or ability. It can also include policies, rules, and ways of doing things that knowingly or unknowingly disadvantage some groups of people, while privileging others. The unfair treatment does not have to be on purpose – it can happen when a person or organization does not mean or intend to discriminate against someone else. It exists presently, and the future remains a concern. The data-driven systems of the future, privileging automation and artificial intelligence, may normalize decision-making processes that “intensify discrimination, and compromise our deepest national values” (Eubanks 2018). The policy determined in the coming years about the role of big data in our lives will speak volumes to how we ensure the rights of the most vulnerable among us, in particular, strive to respect the rights of minorities.

Institutional or systemic discrimination is captured in everyday thinking at a systems level: taking in the big picture of how society operates, rather than looking at one on one operations. This includes ideas of mistreatment of an individual or a group of individuals by society and its institutions as a whole through unequal bias or selection, intentional or unintentional, as opposed to individuals making a conscious choice to discriminate. The achievement gap in education is an example of institutionalized discrimination. The achievement gap refers to the observed disparity in education measures between the performance of groups of students, especially groups defined by gender, race/ethnicity, and socioeconomic status. This disparity includes standardized test scores, grade point average, dropout rates, and college enrollment and/or completion rates. “Defund the police” programs are efforts to trigger change and introduce police reform to address institutional racism.

Socioeconomic status, whether measured by income, education, or occupational status is among the most robust determinant of variations in health outcomes in virtually every society in the world (WHO 2008). Race still matters after socioeconomic status is considered. In particular, health is also affected by exposure to adversity throughout the life course. Early life adversity – poverty, abuse, traumatic stress – influences multiple indicators of physical and mental health later in life, including cardiovascular, metabolic and immune functions. There is a community cost to inequality. The non-equivalence of socioeconomic indicators across racial groups, for example, compared to Whites, Blacks and Hispanics receive less income at the same education levels, have markedly less wealth at equivalent income levels, and have less purchasing power due to higher costs of goods and services in residential environments where they are disproportionately located.

In the US racism puts you at higher risk for COVID-19. It does so through two mechanisms: People of color are more infected because they are more exposed and less protected. Then, once infected, they are more likely to die because they carry a greater burden of chronic diseases from living in disinvested communities with poor food options and poisoned air, and because they have less access to health care – testing is located in more affluent neighborhoods. Institutional discrimination and socioeconomic status disadvantages lead to over-representation of minorities in toxic residential and occupational environments that leads to elevated exposure to major hardships, conflicts, and disruptions such as crime/violence, material deprivation, loss of loved ones, recurrent financial strain, relational conflicts, unemployment and underemployment. Also, residential segregation by race is an example of institutional racism, has created racial differences in education and employment opportunities which, in turn, produces racial differences in socioeconomic status.1

Neoliberalism advances the view that economic and political freedom are inextricably linked. Milton Friedman preached that through the elimination of centralized power whether in government or private hands, each can serve as a counter balance to the other. Friedman feels that competitive capitalism is especially important to minority groups since impersonal market forces protect people from discrimination in their economic activities for reasons unrelated to their productivity. On the other hand, economist Paul Krugman has argued that “laissez-faire absolutism” promoted by neoliberals “contributed to an intellectual climate in which faith in markets and distain for government often trumps the evidence.” Other scholars have argued, in practice, this “market fundamentalism” has led to a neglect of social goods not captured by economic indicators, an erosion of democracy, an unhealthy promotion of unbridled individualism and social Darwinism and economic inefficiency. This has been associated with a widening inequality gap between the rich and middle class.

In articles published in the 1930s, Erich Fromm considered the criminal justice system as an important legitimating institution within the capitalist social order. The state uses the criminal justice system to enhance itself, Fromm claims, by treating the criminal as a scapegoat instead of confronting society’s deep social problems. In dwelling on crime and punishment, the state manipulates society into becoming less attentive to the social and economic inadequacies and oppressions in daily life. That is, a punitive criminal justice system was employed to divert the anger of the masses from the oppressive social conditions that required government remedies. In brief, the criminal rather than state policy became the social scapegoat for social ills, economic inequality, and government corruption and callousness. Did this “criminal system” at least deter crime? Fromm observes that evidence consistently demonstrated that imprisonment, harsh conditions, and even capital punishment had no salutary effect on the crime rate and thus did not protect the public.

Fromm notes the criminal justice system has a decided class bias. Whereas the propertied class has opportunities to sublimate their aggressive propensities into a socially acceptable channel, the disadvantaged lacked these channels and were consistently more likely to commit crimes. Therefore, the reform of social inequities through the redistribution of wealth constitutes a more effective plan for combatting crime than a harsh system of incarceration and punishment that offered little protection to the public. The psychanalyst Fromm observes wars, revolt, and other signs of social discontent are not rooted in infantile fixations, but in external economic structures, concrete social conditions, and shared ideologies and emotions. Social cures or at least reforms could be implemented by changing these collective structures through political actions. The writings of men like Fromm have taught us that pure and absolute freedom is an illusion – freedom is given rather than achieved.2

With nearly 2.3 million prisoners behind bars, the United States has the highest incarceration rate in the world. Rachel Barkow notes that people have a sense that, while you lock them up; we never throw away the key. Ninety-five percent of the time, the person comes back out, and you are just kicking the can down the road. Incarceration is buying you some time, but the underlying issues the person might have, or the underlying cause of the crime in the first place, you’re just putting off. While you’re incarcerating people, not only are you not making them better, you’re often putting them in environments where they are likely to become worse. With respect to recent reform: The First Step Act established “earned time credits” that allows inmates time off their sentencing if they participate in programming while incarcerated. The people who need it most are high risk, while the only people eligible are low risk.

An Obama-era White House report entitled Big Data: Seizing Opportunities, Preserving Values states: “big data analytics have the potential to eclipse longstanding civil rights protections in how personal information is used in housing, credit, employment, health, education, and the marketplace”. To this list, we can add immigration, public safety, policing and the justice system as additional contexts where algorithmic processing of big data impacts civil rights and liberties. Automated, data-driven decision making requires personal data collection, management, analysis, retention, disclosure and use. At each point in the process, we are all susceptible to inaccuracies, illegalities and injustices. We may all be unfairly labelled as “targets or waste”, and suffer consequences at the bank, our job, the border, in court, at the supermarket and anywhere that data-driven decision making determines eligibility. While this threatens us all, the research is clear: vulnerable communities are disproportionately susceptible to big data discrimination.

The quickly changing procedures for determining and implementing labels from myriad data points and aggregations must be scrutinized, as policy struggles to keep up with industry practice (Obar and Wildman 2015). People of color; lesbian, gay, bisexual, transgender and queer communities; Indigenous communities; the disabled; the elderly; immigrants; low-income communities; children; and many other traditionally marginalized groups are threatened by data discrimination at rates differing from the privileged. The maintenance of biased policing techniques to generate new data, raise considerable concerns for civil rights in general, and automated criminal justice efforts in particular. Addressing such challenges involves a combination of strategies for eliminating biases in historical and new data sets, being critical of data sets from entities not governed by law and developing policy that promotes lawful decision-making practices (i.e., data use), and mandating accountability for entities creating and using data sets for decision making.3

1 David Williams, Naomi Priest, Norman Anderson. (April 2016)  Understanding Associations between Race, Socioeconomic Status and Health: Patterns and Prospects

2 Lawrence J. Friedman. The Lives of Erich Fromm: Love’s Prophet Page 35-36.

3 Jonathan Obar and Brenda McPhail. Preventing Big Data Discrimination in Canada: Addressing Design, Consent and Sovereignty Challenges. (12 Apr 2018)

This entry was posted in economic inequality and tagged , , . Bookmark the permalink.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.