The most popular post on this site is “Emergent Biases” where I talk about Tay, Microsoft’s AI which went quite sour as she learned neo-Nazism and misogyny from Twitter users.
Due to the post’s popularity, I will be doing a brief series on biases in computer systems, based on Batya Friedman and Hellen Nissenbaum’s “Bias in Computer Systems.” They define three types of biases – pre-existing biases, technical biases, and emergent biases. This post details one of these biases.
Pre-existing biases always seem the most nefarious to me. As though some software developer somewhere sat in their basement, chuckling maniacally as they put societal prejudices into code. Obviously, that’s rarely ever the scenario. Many of us just can’t think out of the societal box.
Friedman and Nissenbaum define pre-existing bias as stemming from “social institutions, practices, and attitudes.” These biases “exist independently [of], and usually prior to the creation of the system” (333). Unlike emergent biases, these are inherent. And unlike technical biases, these have nothing to do with the limits of technology. No, these come from how specific groups of humans view the world.
My best examples of pre-existing biases appear in dating apps and websites. One of the first things a user does is pick their gender. Unfortunately, there are usually one of two options: “Man” and “Woman.” For those of us who strongly identify as one or the other, this isn’t a problem. But only having these two options ignores individuals who don’t identify as either. And then there are those of us who stress about what the system means by “Man” or “Woman.” Do they mean male anatomy and female anatomy? Or purely gender performance? Or both? And if it’s gender performance, that means a lot of things to a lot of different people. The developers creating only two options for gender creates a pre-existing bias in the online dating systems against non-binary individuals.
There is also another example of pre-existing bias in online dating apps. Tinder is one of the most used dating apps. (I think we all know someone who met their partner on Tinder.) Tinder requires users to have a Facebook account and links the Tinder to the Facebook account. I’m sure this requirement is to ensure that Tinder users are verified and “real.” But it also makes the assumption that everyone has a Facebook account. Thus it’s a pre-existing bias against people without Facebooks. Obviously, not a matter for legal recourse but an interesting example.