Brocker.Org: Infinite personalization is making us dumber



As news about “fake news” reaches a fever pitch, all the noise is
drowning out what I think is a much greater threat, something
related but more subtle—and far more destructive: infinite

Fake news consists of the absurd, sometimes amusing, and often
revolting stories that are easy to spot and dismiss. Infinite
personalization comprises the artificial intelligence-driven,
big-data based tools that allow algorithms to build a
personalized Internet echo chamber customized just for you,
designed to make you feel great. Infinite personalization feeds
you the real, the fake, and everything in between, with the
simple goal of holding your attention and getting you to come
back for more. It is the process by which companies can measure,
match, and predict consumers’ individual preferences with amazing
accuracy and then tailor offerings to maximize revenue.

Infinite personalization is attractive to many types of companies
for obvious reasons. After all, if they understand precisely the
types of things you want to consume, the probability of making a
sale (or getting a click) is greater. For digital goods and
services, personalization tends to be very economical—practically
cost-free, in fact, even on a massive scale. And it tends to work
well. When it comes to suggesting a streaming music playlist, for
example, artificial intelligence does a pretty remarkable job.
Importantly, if such an algorithm makes a mistake in this
context, no real is harm done; the program learns from user
feedback and continually seeks to improve. At worst, the
algorithm deprives users of the opportunity to stumble randomly
upon something very different that they would really like,
resulting in less diversity in one’s musical tastes.

Particularly when it comes to information, however, infinite
personalization has a dark side. Not long ago, Americans tended
to watch more or less the same nightly news on TV, read the same
newspapers, magazines, and bestselling books. Today, each and
every one of us has a custom-designed experience based on our
past preferences. Our shopping experiences on major online
retailers is designed just for us, individually. We have
seemingly endless video choices on streaming media sites,
presented to us in a meticulously customized way. Even search
engines personalize results.

David Siegel Two Sigma
Siegel, cofounder and co-chairman, Two Sigma


One result is that Americans seem to be losing a certain
commonality of experience—even, in some cases, within the same
household. Social network feeds are a fantastic way to keep in
touch with your friends. There is no doubt these services are
very useful. However, not everyone may be aware that postings in
a news feed are carefully selected by algorithms in a very
proprietary way, mainly to get us to use the service more. This
makes good business sense, but it subliminally impacts our
thinking. Like streaming music recommendations, these algorithms
are very good at filtering out postings that we’d dislike,
potentially robbing us of alternative points of view. This is the
echo chamber at work.

Thanks to infinite personalization, we’re losing commonality of
experience just as a record-high 77% of Americans polled by
Gallup see the nation as fundamentally divided about “the most
important values.” Correlation isn’t causality, but common sense
certainly points to a connection. Even with the same raw facts
available (to those who would search for them), infinite
personalization likely contributes to increasingly slanted and
divided views about the issues of the day. Algorithms will find
news (valid or otherwise) and suggest friends likely to validate
one’s views, shielding us from alternatives and stealthily
reinforcing our differences.

The problem is likely to get worse. The technology of infinite
personalization is getting so good that it’s debatable whether we
choose our information sources, or the other way around. Clearly,
that’s good business for the providers of these algorithms and
the companies that use them to advertise and sell. But it dumbs
us down as individuals and weakens us as a society.

When it comes to information about current events (and even
history), what these algorithms really are designed to provide us
with is pure, mainlined confirmation bias—which itself leads to
overconfidence in our own beliefs. It’s easy to argue that these
algorithms are simply giving us exactly what we want. They
certainly do learn what we like; after all, we keep coming back
for more. Further, their designers might argue that the lack of
human editors actually removes biases and give us something
better or purer than was possible before. Don’t buy it. In the
very best of cases, infinite personalization simply replaces one
set of biases (reporters’ and editors’) with another one

fake newsEv Williams/Business Insider

Does this mean that algorithms themselves can be biased?
Definitely. AI doesn’t think, it just scrapes knowledge and ideas
from data and finds patterns to present back to us. AI is as
biased as the data and rules that are used to train it. In
addition, one should ponder this critical question: what exactly
is the AI trying to optimize? In many cases the answer is simply
company profits, not some broad societal objective.

Combining the insult of fake or intentionally slanted news with
the injury of infinite personalization, what we’re talking about
is a vast machine designed purely to earn money by confirming
biases on a mass scale, regardless of the collateral damage. It
is, unfortunately, a very powerful tool for subliminal
manipulation of thinking on a grand, societal scale. It provides
a way to manufacture “truth” out of fiction at very low cost and
very rapid speed, and the results cannot be anything other than

The scientific method and the advent of artificial intelligence
offer us the promise of greater empiricism and a more
evidence-based understanding of reality. Perversely, infinite
personalization seems to deliver the exact opposite. Worse yet,
we are still in the early days of this information revolution.
The ability to use such tools to shape our thinking will only
grow more powerful in the coming years. As individuals and as a
society, we should be very wary about their potential to warp our

David Siegel is cofounder and co-chairman of Two Sigma Investments, a
systematic investment manager with more than $40 billion in