Algorithmic Bias and its Impact on Society (2024)

Algorithms have a significant impact on our daily lives, ranging from social media feeds to search engines and navigation apps. They influence our actions, methods, and motivations through digital tools and interactions with our built environment.

With algorithms touching arguably almost every part of our daily lives, is it increasingly crucial to acknowledge when — and how — these algorithms may be biased.

First, it is important to define what exactly an algorithm is. In its simplest terms, an algorithm is a set of steps to follow in order to accomplish a goal. For example, an algorithm for making a peanut butter and jelly sandwich would consist of the following steps:

1. Grab two pieces of bread, peanut butter, jelly, and a knife.
2. Using the knife, spread about 2 tablespoons of peanut butter on the top of one of the slices of bread.
3. Repeat step 2 with jelly instead of peanut butter.
4. Put the two slices of bread together with the sides with peanut butter and jelly on them facing each other.
5. Cut the sandwich in half and enjoy!

While we can use the term “algorithm” to describe the above process of making a peanut butter and jelly sandwich, algorithms are generally used in the context of computers or technology. In the field of technology, an algorithm is essentially a recipe for a computer to perform a task. Specifically, algorithms are helpful because they can break down tasks into smaller subtasks for computers to understand and execute.

Thus far, algorithms seem to be extremely useful tools that help computers accomplish complex tasks without the need for human intervention. However, their widespread use has led forms of bias, or unfounded preferences, to seep in.

This occurs when algorithms, which are theoretically considered to be fair and unbiased by nature, produce discriminatory results through the weightage assigned to certain factors or through unforeseen interactions between discrete outcomes. Algorithmic bias is crucial to recognize and mitigate because it can have far-reaching negative effects on individuals and society if unchecked or mismanaged.

There are several areas in which algorithmic bias leaves its mark, including finance, criminal justice, and healthcare.

Algorithms profoundly impact the financial industry, as businesses everywhere use algorithms to analyze reams of data in decision-making processes about consumers and customers. In most cases, this approach is useful is it increases speed and efficiency when making decisions. However, it can prove to be harmful when these algorithms start to make discriminatory decisions towards certain groups, such as people of color, women, and low-income communities.

For example, the Apple Card algorithm recently came under scrutiny for giving male users higher spending limits than their female counterparts, despite females having comparable or higher credit scores and incomes. One of the main issues with this was that the algorithm was essentially a black box, which means that no one, not even Apple customer service representatives, could explain exactly why it made the decisions it did. Therefore, it is difficult for customers to understand why they received a certain spending limit, which could prove to be disastrous as it could penalize them for something out of their control, which is what happened with one Apple Card user and his wife.

While this is only one instance, there are several other examples of biased algorithms negatively affecting society through loan discrimination and inaccurate investment recommendations. Therefore, it is crucial to attempt to reduce this bias in the industry as much as possible.

Finance isn’t the only industry impacted by algorithmic bias. The criminal justice system has also been afflicted by biased algorithms making decisions about recidivism, unfair sentencing, and inequity in access to bail.

A prominent example of this is documented in the ProPublica report published in 2016, which demonstrated that the COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) algorithm, which is used to inform decisions about jail sentences and recidivism rates, was biased. The authors reported that the algorithm found that Black defendants were found to be at a higher risk of recidivism, or likelihood of reoffending, than white defendants, even when comparing people with similar criminal histories, ages, and genders.

This fact is particularly alarming due to the widespread use of the COMPAS algorithm in the United States in pretrial and sentencing settings. If a biased algorithm is being used to determine recidivism, which in turn is considered when determining jail sentences, it can perpetuate existing biases and even destroy lives. Therefore, it is crucial to constantly evaluate and monitor algorithms used in criminal justice settings because they have a profound impact on individuals’ livelihoods.

Finally, algorithmic bias is also evident in the healthcare industry– another sector that has the power to radically affect the standard of living. It can worsen health disparities by making inaccurate diagnoses and treatment plans for patients.

For example, researchers at the University of California-Berkeley found that a widely used algorithm assigned Black and white patients the same risk score, or risk of experiencing a negative health issue, but the Black patients were significantly sicker than white patients in that threshold. In other words, the algorithm was underestimating the level of healthcare that Black patients needed.

Algorithmic errors like this have serious consequences because they prevent people from receiving adequate treatment. American healthcare has always been plagued with racial and income biases, but if algorithms in the space are developed and used incorrectly, it can result in an even deeper entrenching of these prejudices. In situations when healthcare professionals must rely on technology more than their intuitions, such biases could have widespread implications for the future of healthcare.

The effects of biased algorithms seem daunting, but the status quo can be changed. There are several different ways to implement more ethical algorithms, beginning with:

• Train algorithms on more representative data sets

When an algorithm is being trained, or taught to recognize patterns and make predictions based on data, one way to make it less biased is to expose it to a large and representative dataset. For example, if one were training a facial recognition algorithm, they would not only expose it to lighter-skinned men but also darker-skinned people of color and women. By following this approach, the algorithm is more likely to produce accurate results when deployed for commercial use.

• Diversify the teams developing the algorithms

The field of computer science and technology has historically been dominated by white men, potentially contributing to the prevalence of algorithms that display biases toward this demographic. However, adding diversity to the teams that develop algorithms may be impactful because it would bring in a variety of new perspectives and backgrounds, which would make the teams more likely to identify and address potential biases before they escalate. This will, in turn, ensure that the algorithms that these teams develop will be beneficial to a wider range of people rather than a select few.

• Use explainable algorithms

Currently, most algorithms, particularly those related to artificial intelligence, are considered black boxes because it is impossible to determine the reasoning behind their decision-making processes. However, there is a growing field with a focus on creating more explainable algorithms that can help mitigate algorithmic bias by enabling researchers to identify and resolve biases more quickly. Essentially, the explainable algorithms provide a more transparent decision-making process, which allows for more preventative measures against algorithmic bias.

• Regularly evaluate and monitor the algorithms

Finally, while the above steps are helpful, it is good practice to keep regularly monitoring and checking the algorithms to ensure that every detail is noticed.

In conclusion, while algorithms are ubiquitous in our daily lives in almost every industry (including finance, criminal justice, and healthcare) and often provide numerous benefits, they can also perpetuate biases, a fact which we must all be aware of.

By working to reduce algorithmic bias, we can ensure that algorithms are being used to serve us, not harm us.

• Neil Vigdor. “Apple Card Investigated After Gender Discrimination Complaints.” The New York Times. Accessed at: https://www.nytimes.com/2019/11/10/business/Apple-credit-card-investigation.html
• Jeff Larson, Surya Mattu, Lauren Kirchner and Julia Angwin. “How We Analyzed the COMPAS Recidivism Algorithm,” ProPublica. Access at: https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
• Jamie Condliffe. “Algorithmic Bias Is Bad. Uncovering It Is Good,” New York Times. Accessed at: https://www.nytimes.com/2019/11/15/technology/algorithmic-ai-bias.html?searchResultPosition=2
• Andre Esteva, Brett Kuprel, Roberto A. Novoa, Justin Ko, Susan M. Swetter, Helen M. Blau, Sebastian Thrun. “Dermatologist-level classification of skin cancer with deep neural networks,” Nature. Accessed at: https://www.nature.com/articles/nature21056
• Dhruv Khallar, “A.I. Could Worsen Health Disparities,” The New York Times. Accessed at: https://www.nytimes.com/2019/01/31/opinion/ai-bias-healthcare.html