The Science of Prediction

Big business today knows what you are going to buy and where...before you do. Predictive analytics is the science of predicting the future based on historical patterns, and predictive analytics has truly come into its own in the second decade of the 21st century.

The idea of using data to create models projecting trends into the future has been around for decades, but the huge amount of relatively easily accessed data and the sophistication of the analytics tools available today,  together with the growing ubiquity of the use of these tools by organizations, is creating a stealth revolution in organizational management right under our noses.



Image by

The Beginnings of Modern Predictive Analytics

Wal-Mart was among the first to apply sophisticated statistical analyses to customer and sales data, with their initial efforts stretching back as far as the mid-1990s. Even in the early days Wal-Mart recognized that data analytics was not just about identifying what people were buying, but why, how much they would pay, and how to most efficiently purchase and deliver the specific products that the people who shopped in those particular Wal-Marts were most likely to purchase. And it worked like a charm, with Wal-Mart continuing to increase sales and firmly cementing its domination of the retail sector in the late 1990s and early 2000s.

It didn't take long for Wal-Mart's competitors to see the value of what Wal-Mart was doing, but it did take a few years for the retail industry as a whole to work out the kinks in their predictive analytics programs to reach the same level of sophistication. With the retail industry leading the way, it has not taken  long for predictive analytics to become de rigeur in big business in almost every industry, as it is difficult to compete without a real-time understanding of the evolving market landscape.

Predictive Analytics in Practice

The use of predictive analytics has become common in industries such as telecommunications, financial services and retail, and it is expanding rapidly into new industries and new areas.

Professional sports teams were also early adopters of sophisticated data analysis systems, particularly the statistics-oriented game of baseball. Modern player statistics analysis really began with Bill James (of MoneyBall fame) who developed the now famous Sabermetrics system in the 70s, some variant of which is used today by all 30 Major League Baseball teams. The Oakland Athletics and the Boston Red Sox led the pack with their sophisticated use of player metrics in the 90s, and most of the rest of MLB jumped on the bandwagon around the turn of the century.

Professional football and basketball teams also noted the trend and obvious value of player metrics, and by 2008 almost all major sports teams had some form of player statistical analysis program in place. The increasing sophistication of predictive analytics software has also led sports teams to take advantage of its potential in other areas such as ticket sales. According to Anthony Perez, director of business strategy for the NBA's Orlando Magic, the team enjoyed  their largest ticket sales ever in 2012 after implementing a predictive analytics-based ticket pricing system, and this despite missing 11 home games of the season due to the NBA lockout.

Academic institutions of all sizes are also rushing to develop data-driven analytical processes.  Colleges and universities are looking at declining enrollment over the next few years. A 2012 survey by Moody's Investors Service reported that over half of the colleges and universities project  enrollment of full-time students to decline over the next few years. Furthermore, 35% project tuition revenue to decrease or to increase at less than the inflation rate.

This impending decline in revenue has forced academic institutions to start to reconsider their operational models and budgets, and search for efficiencies.  College administrators need the answers to questions such as how many freshmen will drop out following their first semester or what will total revenue will look like next fall or in Fall 2014? Or even how many dorm rooms, classrooms and dining halls will we need five years from now?

The trend toward the use of predictive analytics in academia is just in its infancy in 2013 -- just a half a dozen schools (including the University of Kentucky and Taylor University) have programs in place to date. The good news is that the success of these programs is enabling college administrators to find out just how much useful information a thoughtfully developed near-real-time data-driven predictive analytics program can offer, and many other institutions have begun laying plans to implement similar or even more advanced programs.