Analytics needs to walk around some holes in the
sidewalk.
A wonderful book of poems by Portia Nelson, There's
a Hole in My Sidewalk: The Romance of Self-Discovery, addresses the struggle
to stop falling into the same psychological/behavioral hole, how to walk around
it and “go down another street”…and grow as a person.
The field of analytics has fallen into a few big holes lately
that represent both its promise and its peril.
These holes pertain to privacy, policy, and predictions.
Privacy. $1B. Target, the retailer, was the poster
child for using big data for customer analytics to pump up sales. It unabashedly collected lots of data on its
customers, from a variety of sources, integrated it, and used it for predictive
modeling to identify segments that are experiencing “moments that matter” when
habits can be influenced to buy new products.
Target
touts that “we’ll be sending you coupons for things you want before you
even know you want them.” For example,
it developed algorithms about the probability of pregnancy and the delivery
date to sell specific products that women buy at different times during their
pregnancy. It identified the women, sent
them coupons, and opened its cash registers to amazing profits. However, as we have learned, it also opened
its cash registers, credit card machines, and databases to cybercriminals who
stole the personal data of tens of millions of customers. It is estimated that this error will cost
Target over $1B in fraud claims. Its
stock price has fallen over 25% since the incident.
The “hole” is a comfortable one for analytics. The habit
is to uncork technology before its time.
For example, the NSA exploited
the technology to tap telephone calls and scrape peoples’ metadata into a
database before it confronted the likelihood that world leaders and the public
at large would condemn it and it could not defend it in terms of averting terrorism. Similarly,
there was a lot of talk about the “creepiness” of retailers collecting personal
data on customers by whatever means possible.
The big appetite for the data to improve sales may have
blinded companies from thinking about the consequences and “forgetting” the
basic responsibility to protect it. In the
Target case, there are known credit card technology safeguards, including the
use of a security microchip, that were ignored. Additionally, there must be encryption
protocols and firewalls to decouple data so that cybercriminals would not find
personal identity information. The simple lesson is that just because the
technology exists does mean that it should be used. Perhaps one route around the “hole” is to “count
to ten” before technology genies are let out of the bottle.
Predictions: 43-8.
The great hope to demonstrate the value of analytics is (advanced)
predictions. It uses all the breadth
and depth of big data to go beyond reporting on the past to predicting the future. So, how could the predictions about the 2014
Super Bowl game between the Sea Hawks and the Broncos be so far off? The point spread was 3 points but the actual
spread was more than 10 times that as the Sea Hawks routed the Broncos and
Peyton Manning from the first (mis) play of the game. Perhaps there is a tribe of analytics
“sharps” who are making it big in sports wagering but the facts are that the
best of them only win about 53% of the time.
The irony perhaps is that football, like baseball and
basketball, is a fully digitized industry unlike most others including healthcare
which still struggles to use electronic medical records to capture its key
transactions information. In sports, every
play action on the field is captured, recorded, and discussed, resulting in a
rich performance database of players in almost every conceivable context, e.g.
how a baseball hitter performs relative to a specific pitcher, playing field, regular
or post-season game, and so forth.
But, it is clear from the big-miss prediction of the Super
Bowl game that some important data that would improve the precision of the model are missing. The “squares”, who rely on softer data (intuition),
think they know this realty of the shortcomings of quant data, although their
win rate is no better than that of the sharps.
My personal insight on this is when I was 16 years old I worked as a dog
handler at a greyhound racing park. I
took a dog from its pen, to the viewing stand, into the starting gate, and
picked it up at the conclusion of the race.
I knew when the dog was nervous, sick, and hyped up. And I knew when they hit their head going
into the gate that they would not recover to win the race.
The “hole” here is the reliance on the big data that is under
the lamppost. In this case, it is the
big sports data, most of which is collected…because it can be… without a model
in mind and mostly for its entertainment value.
The big data presumption is that if you build it (the database), the
predictions will come. That ain’t
necessarily so, even if one runs zillions of simulations on all the
yottabyte of big data. The data have to be right
for the model to work. In the case of
sports, there are lots of (“soft”) untapped personal data such as health, resilience,
and response to certain threats (and more) that may be important factors in big
game performance. It’s a real short circuiting of predictive
modeling to be carried away with the technologies of the yottabytes while
avoiding a full understanding of the phenomena under study.
Policy. 2.2/7.
The biggest analytics project in recent history is the $6 billion
federal investment in the health exchanges.
The goals of the health exchanges are to enroll people in the health
insurance plans of their choice, determine insurance subsidies for individuals,
and inform insurance companies so that they could issue policies and
bills. The project touches on all the
requisites of analytics including big data collection, multiple sources,
integration, embedded algorithms, real time reporting, and state of the art
software and hardware. As everyone
knows, the implementation was a terrible failure. The CBO’s conservative estimate was that 7
million individuals would enroll in the exchanges. Only 2.2 million did so by the end of
2013. (This does not include Medicaid
enrollment which had its own projections.)
The big federal vendor, CGI, is being blamed for the mess. Note that CGI was also the vendor for the
Commonwealth of Massachusetts which had the worst performance of all states in
meeting enrollment numbers despite its long head start as the Romney reform
state and its groundbreaking exchange called the Connector. New analytics
vendors, including Accenture and Optum, have been brought in for the
rescue.
Was it really a result of bad software, hardware, and
coding? Was it that the design to enroll and determine
subsidies had “complexity built-in” because of the legislation that cobbled
together existing cumbersome systems, e.g. private health insurance
systems? Was it because of the incessant politics
of repeal that distracted policy implementation? Yes, all of the above.
The big “hole”, in my view, was the lack of communications
between the policy makers (the business) and the technology people. The technologists complained that the business
could not make decisions and provide clear guidance. The business expected the technology
companies to know all about the complicated analytics and get the job done, on
time. This ensuing rift where each group did not
know how to talk with the other is recognized as a critical failure point. In fact, those who are stepping into the
rescue role have emphasized that there will be management status checks daily
“at 9 AM and 5 PM” to bring people together, know the plan, manage the project,
stay focused, and solve problems. Walking
around the hole will require a better understanding as to why the business and
the technology folks do not communicate well and to recognize that soft people
skills can avert hard technical catastrophes.
In summary, these
three holes in the sidewalk of analytics are recurrent themes and threats to
fulfilling the promise of analytics.
First, the technology cannot zoom ahead of the sociology. The need for business results cannot err on
the side of the creepy use of personal data to increase sales without a full
respect of the need to protect privacy and to honor customers. Second, big data is not the answer if it is
not the right data. The full potential
of predictive modeling requires more thinking and less data processing. And lastly, the big failures in analytics
have less to do with bad machines and buggy software and much more to do with
people on either side of the business and technology fence just not talking
with one another.
