In a world of plenty, algorithms may be our saving grace as they map, sort, reduce, recommend, and decide how airplanes fly, packages ship, and even who shows up first in online dating profiles. But in a world where algorithms increasingly determine what we see and don’t see, there’s danger of filtering gone too far.
The global economy may be a wreck, but data volumes keep advancing. In fact, there is so much information competing for our limited attention, companies are increasingly turning to compute power and algorithms to make sense of the madness.
The human brain has its own methods for dealing with information overload. For example, think about millions of daily input the human eye receives and how it transmits and coordinates information with our brain. A task as simple as stepping a shallow flight of stairs takes incredible information processing. Of course, not all received data points are relevant to the task of walking a stairwell, and thus the brain must decide which data to process and which to ignore. And with our visual systems bombarded with sensory input from the time we wake until we sleep, it’s amazing the brain can do it all.
But the brain can’t do it all—especially not with the onslaught of data and information exploding at exponential rates. We need what author Rick Bookstaber calls “artificial filters,” computers and algorithms to help sort through mountains of data and present the best options. These algorithms are programmed with decision logic to find needles in haystacks, ultimately presenting us with more relevant choices in an ocean of data abundance.
Algorithms are at work all around us. Google’s PageRank presents us relevant results—in real time—captured from web server farms across the globe. Match.com sorts through millions of profiles, seeking compatible profiles for subscribers. And Facebook shows us friends we should “like.”
But algorithmic programming can go too far. As humans are more and more inundated with information, there’s a danger in turning over too much “pre-cognitive” work to algorithms. When we have computers sort friends we would “like”, pick the most relevant advertisements or best travel deals, and choose ideal dating partners for us, there’s a danger in missing the completely unexpected discovery, or the most unlikely correlation of negative one. And even as algorithms “watch” and process our online behavior and learn what makes us tick, there’s still a high possibility that results presented will be far and away from what we might consider “the best choice.”
With a data flood approaching, there’s a temptation to let algorithms do more and more of our pre-processing cognitive work. And if we continue to let algorithms “sort and choose” for us – we should be extremely careful to understand who’s designing these algorithms and how they decide. Perhaps it’s cynical to suggest otherwise, but in regards to algorithms we should always ask ourselves, are we really getting the best choice, or getting the choice that someone or some company has ultimately designed for us?
Question:
Rick Bookstaber makes the case that personalized filters may ultimately reduce human freedom. He says, “If filtering is part of thinking, then taking over the filtering also takes over how we think.” Are there dangers in too much personalized filtering?
Tags: algorithms, Facebook, filtering, Google, Match.com, Online Advertising, PageRank, personal freedoms, security and privacy
There’s a great TED talk on this as well by Eli Pariser. He talks about how online “filter bubbles” that reduce our worldview and limit our opportunities to learn new things or find differing opinions. I think this is an important thing to be aware of.
Scott thanks for the note and the links. Eli seems to be a little ahead of the rest of the world in his thinking on this topic. Good stuff and I’ll take a look at the video!
Oh– That TED talk can be found at: http://www.ted.com/talks/lang/eng/eli_pariser_beware_online_filter_bubbles.html
Yes, this is precisely what Eli Pariser at http://front.moveon.org/what-the-internet-is-hiding-from-you-2c/ has been talking about.
Allen, thank you for commenting. I hear “Filter Bubble” is a good tome and plan on adding it to my reading list!
Hi Allen, I agree that pre-filtering is already damaging my access to open information stores. I remember when searches actually returned relevant, unique, or interesting website content that wasn’t main stream commercialize content or normalized thoughts.
Censoring and filtering by search engines prohibit finding specific or unique thought/data/information. The author/creator must commercialize it or play the commercial game of online promotion for it to be published by search engines.
My view is a generalization and an oversimplification; it also contains a large measure of truth. What the human species doesn’t need but seems to be generating is a devolution in both precognitive processing and cognitive capabilities within mass human populations.
An offsetting alternative is more R&D in cognitive tool development. Cognitive tools would help humans think more efficiently and effectively. A very powerful and simplistic example is the use of check-off lists by medical professionals to reduce errors.
When lists are used by medical staff there have been dramatic reductions in human errors. There was a corresponding significant drop in avoidable deaths and avoidable serious complications caused by human error. I don’t recall from memory a medical study to cite but I do recall reading an abstract from a prominent medical journal and that additional medical research studies support the dramatic benefit to human life accrued by medical staff using check-off lists.
The main barrier to widespread use of check-off lists was cited as the stigma that the medical staff associated with “having to use a check-off list” and other was their resistance to forming new habits. The stigma was most prevalent, and carried the strongest emotional charge, in high status medical jobs e.g. surgeons who felt they looked stupid or that checklists implied a deficiency in memory or in intelligence. They rationalized they smart enough to remember require flow of medical procedures and all the steps in each medical process.
-Neville Reynolds
Hi Allen, I agree that pre-filtering is already damaging my access to open information stores. I remember when searches actually returned relevant, unique, or interesting website content that wasn’t main stream commercialize content or normalized thoughts.
Censoring and filtering by search engines prohibit finding specific or unique thought/data/information. The author/creator must commercialize it or play the commercial game of online promotion for it to be published by search engines.
My view is a generalization and an oversimplification; it also contains a large measure of truth. What the human species doesn’t need but seems to be generating is a devolution in both precognitive processing and cognitive capabilities within mass human populations.
An offsetting alternative is more R&D in cognitive tool development. Cognitive tools would help humans think more efficiently and effectively. A very powerful and simplistic example is the use of check-off lists by medical professionals to reduce errors.
When lists are used by medical staff there have been dramatic reductions in human errors. There was a corresponding significant drop in avoidable deaths and avoidable serious complications caused by human error. I don’t recall from memory a medical study to cite but I do recall reading an abstract from a prominent medical journal and that additional medical research studies support the dramatic benefit to human life accrued by medical staff using check-off lists.
The main barrier to widespread use of check-off lists was cited as the stigma that the medical staff associated with “having to use a check-off list” and other was their resistance to forming new habits. The stigma was most prevalent, and carried the strongest emotional charge, in high status medical jobs e.g. surgeons who felt they looked stupid or that checklists implied a deficiency in memory or in intelligence. They rationalized they are smart enough and have enough memory recall to remember the required flow of medical procedures and all the required steps in each medical process.
-Neville Reynolds
Neville – thanks for commenting. Interesting you mention check off lists. I’ve looked at decisioning techniques in the medical field, specifically use of decision trees to arrive at best outcomes. There was even some efforts to codify these into decisioning algorithms. But as you know, the human body is quite complex and medical diagnoses can go a million different ways, so in the end these techniques haven’t proven esp valuable. That said, I think there’s a lot of capability and value on the way in advanced decisioning systems like IBM’s Watson. There are efforts ongoing to port Watson’s probablistic decisioning to the medical field and I look forward to the possibilities!
Signals, incoming messages to shuck and jive, read keep or delete. Sorting through what is worth while, what is spun, what is raw, real and transparent means everyone’s processing mental dog and pony is stepped up. Several notches. Like John and Sue Public being lashed to a very fast turning merry go round. Too many messages bombarding asteroid like. But a few standing out as helpful, worthwhile, sincere and tightly put together consistently to reach further, with amazing frequency. Great post and helpful links to expand on the filtering and how it all works in the mind. Or is suppose to.
Andrew, you bring up some good points about the constant information bombardment. Our brains have to filter – we have no choice. And hence, the reliance on compute power and algorithms to help us filter the million billions of possibilities. And while these algorithms can certainly help us, I think it behooves us to understand more about how they decide and even motives behind their decision making so we can effectively determine if we are truly getting the best result set. Definitely something to noodle and I appreciate that you’ve joined the conversation!
[...] based on one’s language, location, and other criteria is no secret. Although personalization may or may not be a bad thing from the searcher’s viewpoint, what it basically means for SEO is the inability to [...]