Is Big Data a Good Thing?

Data star trek 8470691 1024 782
Image via thealphaquadrant.blogspot.com

Big Data is a crazy reality that we have created with society’s many digital input devices, from street cameras to the common smartphone (sorry, Trekkies). There is so much data available that computing algorithms are needed to extrapolate and contextualize the information. Companies are actively looking at ways to mine and extrapolate Big Data for analytics and market use.

McKinsey & Company’s Business Technology Office says Big Data will become a key basis of competition, underpinning new waves of productivity growth, innovation, and consumer surplus. The report goes on to list five ways Big Data can be used by companies and nonprofits:

1) Big Data can unlock significant value by making information transparent and usable at much higher frequency.

2) Organizations create and store more transactional data in digital form, they can collect more accurate and detailed performance information on everything from product inventories to sick days, and therefore expose variability and boost performance.

3) Big Data allows ever-narrower segmentation of customers and therefore much more precisely tailored products or services.

4) Sophisticated analytics can substantially improve decision making.

5) Big Data can be used to improve the development of the next generation of products and services.

Given the incredible amounts of data available about people, will companies abuse the data to take advantage of people and society in general? This is a tough issue because generally, Big Data will improve our ability to serve each other with better, more qualitative information, product and service offerings. Semantic information is already making search infinitely better.

However, there will be repercussions including further polarization and perhaps an unhappy realization of the picture that Big Data shows of ourselves as a society. Society may not be ready to see itself in the mirror.

Further, the continuing trials of Facebook illustrate just how serious of an issue Big Data has become. Facebook’s consistent use of user data to benefit its corporate customers in the face of privacy has triggered investigation requests to the FTC, and continues to get exposed by the media. Yet Facebook continues its practices in the face of media protests and potential lawsuits or worse.

For every Facebook that data issues become well known (and the company suspect), there are dozens who get away with Big Data abuses, oft under the radar. Really, in every technology, in every sector, there are abuses. Big Data is and will always be no different.

Will we accept Big Data’s negatives as a trade off for better results. Or do we even have a choice? What do you think?

How Social Semantic Search Defines People

BORKER-2-popup.jpeg
(Cartoon by David G. Klein from the New York Times)

Search is the underpinning of the Internet today, from the 1 billion traditional searches everyday on Google to providing references about a person on Twitter and delivering their stream feed on Facebook. Search has moved from simple page rank to an increasingly complex algorithm that weight’s social and semantic data points to deliver the outcomes most likely to please you. Personalization of search continues to evolve, but in turn it defines people and their choices.

Search — the technology itself — doesn’t bear responsibility for this. People do. People who use the Internet and its many free tools without understanding how the information is provided to them. They blindly accept search results or the search-based content feed without considering the source.

Consider the DecorMyEyes fiasco broken by the New York Times. Owner Vitaly Borker explained how he used intentionally created negative complaints about DecorMyEyes to game search results and place himself as a top ranked eyeglasses vendor. To Google’s credit, they promptly changed their algorithm to include more semantic weight (all negative or all positive disqualifying you), and the Department of Justice followed up with charges.

Social networks and applications also use search to source preferred content. Facebook’s activity feed is designed to source the most “interesting” content to people in your friends network are using the Open Graph API and likes. Search on Facebook is completely driven by the Open Graph (Like) protocol.

Of course, hashtags have demonstrated the power of search on Twitter. Twitter search was originally based on the acquired Summize search technology, and has been used to reference mentions and trends, too. Now Twitter (and other services) suggests people like you using semantic data.

The Danger of Homogeneous Definition

Google Organic.jpg

The danger in all of this personalized search — particularly when it’s largely based on peer interests — is creating a society of homogeneous sycophants that blindly accept the content sourced to them, either via search or feeds. Lest we think that people actually think through the click, consider organic click through rates on Google (as pictured above by SEO’s Neil Walker). Clicking through on the first few search terms is and has been the norm.

The addition of local semantic data to search only further complicates concepts of popularity. Algorithms tell people which burger joints, music venues, theaters, etc. are most likely to meet their interests.

When popularity is defined by an algorithm and served to people, homogeneos or mob thinking becomes the norm. This thinking feeds on the popular. Society is not currently trained to question the information presented to it. Thus algorithms — designed to create the output that will generate the most click throughs — become a critical determinant in defining people’s lives, and society as a whole.

That’s not necessarily a bad thing. Semantic information can weigh in when a system is gamed, and social search can provide the latest information based on people’s actual use and check-ins. However, idea markets are increasingly influenced by the popular, and not necessarily in a good way. Algorithms can keep bad ideas popular for longer periods of time.

It all points back to the need for society to teach better information skills. In an information economy, the ability to question and discern quality data presented via a plethora of media is an essential quality for democracy and individualism. It’s important to look deeper at online search, whether that’s because a search provided direct information or because an algorithm sourced a friend or influencer touting an idea or product. Quoting Doug Haslam, “Think for yourself. …you needn’t be part of some pack that can’t brook disagreement with your heroes.”

An educated Fifth Estate creates an evolutionary society, a mindless one creates results like Kim Kardashian as the number one search term on Bing for 2010. While many people find Kardashian attractive, should social semantic search tell every person — man and woman alike — what the icon of attractive is? Parents across America may object.

What do you think about how search and algorithms are defining our society?