I love the current series of commercials put out by esurance. They all involve people encountering someone that is “sort of” like someone else they were expecting. Watch them up on YouTube:
- Sorta Mr. Craig – Parents visit an unlikely teacher in the classroom. You’re not Mr. Craig. Well, sort of; “we’re both between 35 and 45, both like to save on car insurance, and both really good at teaching people a lesson”.
- Sorta Your Mom – An unlikely mom pulls up to pick up two kids at school. That’s not my mom. “I’m sorta your mom. We’re both 25 to 35 years old, both women on the go, and we both clocked a lot of miles.”
- and my favorite, Say My Name – A woman approaches a pharmacy counter which is manned by Walter White. You’re got Greg. “I’m sorta Greg. We’re both over 50 years old, we both used to own a Pontiac Aztec, and we both have a lot of experience with drugs.”
I don’t love these commercials just because they are funny and one includes Walter White, but because they brilliantly demonstrate the folly with the commoditization of us through classification algorithms. By commoditization, I mean we’re lumped into a group for which the members share characteristics mostly determined by “letting the numbers speak”, stripping away all those little things that seems irrelevant, but in aggregate are what makes each of us unique and special. The theory is that folks sharing a set of characteristics would likely share one or more that are of interest. It eases a business’ efforts in dealing with people if we can carve away that nasty complexity of real people and deal with a nice, clean entity who we will now assume works as the categorization suggests. Things are simplified at the expense of our individuality.
This subject has touched a nerve in me lately through a couple of recent conversations with colleagues on personality tests. I’m not comfortable with being just a number and thought to be that simple of a creature. I’ve had to take personality tests such as Myers Briggs in the past for work. In fact it would be very hypocritical of me as a BI practitioner to not appreciate the value of such tests since it is in fact the result of a classification algorithm, which is a big part of what I do for a living through my Business Intelligence practice.
Therefore, I want to be clear that this blog isn’t a hit piece on such tests. I very much see the value as a tool that helps to smooth out working relationships within teams, which is a worthwhile thing. My point is to provide a friendly reminder to everyone that predictive analytics is just a reasonable guess based on statistics. It’s easy to get overly hung up on such numbers as the simplification of decision making is seductive. Clustering, or any data mining algorithm for that matter, provide heuristics. They are simple rules that at least kick us off in some direction, freeing us from analysis paralysis. They toss out the noise. They theoretically should be better than a shot in the dark, at least for the person doing the shooting.
The problem is when we become married to that initial impression – which of course, brings up that “first impression” thing, which seems to be a real thing. And that is an easy thing to happen. Our brains are constantly and desperately seeking out patterns to tame the complexity of the real world. We hate to give up old patterns because we’re in the groove with those current patterns and now we need disrupt things with new ones.
Whenever I take those personality tests, I really, truly struggle to answer those multiple choice questions. With almost all of them, I think, “Well, it depends!” “It’s not as simple as that!” The problem I have answering those questions probably pegs me into some hole in itself. Extravert or Introvert? It depends. For me anyway, it’s not a one dimensional continuum. Sometimes you can’t shut me up, sometimes I just want to sit in the back row and watch. I don’t recall exactly which of the sixteen categories I fell into, but I do remember it wasn’t what I would have thought for myself.
Like any statistics in the hands of those not skilled with statistics (or with too much skin in the game), things can be taken out of context and/or continue to be applied long after it’s no longer valid. The irony of all this is that as we strive so hard to remove old stereotypes and prejudices from society, we replace them with a much larger number of newly invented ones. Granted, these new ones usually appear to be more benign than those old ones.
I rarely ever see a cluster created in which the vast majority of the members are in it with a high probability. Meaning the clustering is right most of the time, but when it’s wrong, it can be merely rude to significantly troublesome as the esurance commercials humorously illustrates. Every now and then we’ll be tossed into a group for which we are not really well suited, but it’s the closest fit (shoving a trapezoid into a square hole). Yet, we’ll be treated the same way as those who fit very nicely into the categories.