We'll respond shortly.
Whether companies refer to results, outcomes, ROI, or case studies, big data and data science are moving beyond the hype and proving to show more and more benefits over time. The world now believes in the concept of smart cities, smart cars, and smart homes, and there are now many real-world examples of where big data and data science is making an impact, including the early development of data lakes and application of machine learning.
According to research from Deutsche Bank, the expectations for big data have become very real as Apache Hadoop® is expected to be prominent in CIO’s analytics investments in 2015. Today, the industry agrees that the third platform is here, as our CEO, Paul Maritz, discussed last year, and IDC expects apps with predictive analytics and machine learning to grow by 65% as a unified data platform architecture takes shape.
To highlight how real these results are across the industry, Pivotal published a popular post last year, and it cited 20 examples of what companies were doing to achieve benefits from big data. Below, there is a new, recent roundup of research and big data examples, showing results and support for the next phase of big data adoption.
According to the most recent surveys by Accenture, GE, and IBM, there are strong conclusions on big data. For companies that are using big data, 92% of executives are satisfied with the results and 89% rate big data as “very” or “extremely” important. Similarly, Accenture researchers found that 89% of respondents who have implemented at least one big data project see it as a way to revolutionize business operations, and 85% believed big data would dramatically change the way business is done.
In a collaborative report with GE, the Industrial Insights Report for 2015, 84% of those surveyed believe big data analytics will “shift the competitive landscape for my industry” within a year and 87% believe so in three years. In addition, 89% believe a lack of big data adoption will create a risk of losing marketshare, and 75% cite growth as the key value of analytics. It’s no wonder that, according to CIO.com and CEB, CMOs now spend more on technology than other departments and the biggest driver of their tech spending is big data at 37% of the marketing budget, but marketing is far from the only department seeing results—other departments are getting results as well.
Venturing over to the cost side of the equation, the well-known “Internet Trends Report” from KPCB’s Mary Meeker showed several general trends for the past decade or two—reduction of compute cost by 33% annually, storage cost by 38% annually, and bandwidth costs by 27% annually. These statistics point to the ability of big data to provide greater value for companies by lowering the cost of investments and operations. In one life sciences example, the cost-per-bit of biologic information “is coming down faster than Moore’s Law,” and the cost of genome sequencing has dropped as much as 90% for several years in a row.
For two specific examples of both value and cost elements of big data, the work of EMC data scientist Pedro Desouza is a perfect example. One of his team’s churn algorithms helped a company predict and prevent account closures whereby attrition was lowered 30%. He also “helped reduce an organization’s cost of big data analytics from $10 million to $100 thousand per year.” In the latter case, the customer was shocked because the change was non-disruptive.
The bottom line is that businesses are seeing big value and reaping the benefits of big data, and it is clear that technologies like Apache Hadoop®, running on commodity hardware, can store massive amounts of data at a fraction of the cost of traditional data warehouses.
Big Data implementations have now existed long enough to show results beyond the internet juggernauts and early adopters that started off applying Hadoop to solve innovative problems. From automotive and healthcare to logistics and retail, there are strong results with big data and data science across virtually every industry.
At Walgreens, big data is being used by clinicians at in-store health clinics. The company is delivering advanced analytics at the point of care to better assess patient conditions and provide recommendations that improve health overall and avoid future medical costs. For example, a current system can catch an unfilled prescription to help people stick to their healthcare plans and avoid further, unnecessary costs. Over 7.5 billion medical events for 100 million people power the big data system with information like demographics, enrollment, diagnoses, procedures, and data from managed-care plans.
Kroger is leveraging big data as well through its joint venture with Dunnhumby. The company accesses, collects, and manages data for about 770 million consumers. For Kroger, the analytics output from big data has helped them see greater, more actionable insights on customer loyalty and profitability. Claiming 95% of sales are rung up on the loyalty card, Kroger sees an impact from its award-winning loyalty program through nearly 60% redemption rates and over $12 billion in incremental revenue by using big data and analytics since 2005. According to one report, Kroger credits their loyalty and data driven buying pattern analysis programs for staying profitable during the recession of 2009.
For companies focused on customer relationships, providing great service is top of mind via social channels and other interactions. Southwest uses speech analytics to help improve the interactions between customers and personnel. As well, Southwest uses big data to understand online behaviors and actions, improving offers for customers and leading to growth in loyalty year over year.
Delta has used big data to help with one of the most uncomfortable travel situations that exists—lost baggage. With over 130 million bags checked per year, the company held a lot of tracking data about bags and became the first major airline to allow customers to track their bags from mobile devices. To date, the app has been downloaded over 11 million times and gives customers much greater peace of mind while traveling while also differentiating Delta as a customer-centric company.
In the fast-paced world of online media, Huffington Post grew last year into the number one online news site in the United States. According to this report, the company’s leadership believes in running the business based on data. This includes improving the user experience in real-time from social trends, recommendations, moderation, and personalization—they optimize the site many ways, and their analytics platform powers the entire analytical process.
According to the same report on Big Data for Media, FT.com also uses data to understand and serve the customer better, create targeted advertising, and design new products based on information collected. Their CEO claimed that big data transformed their business. The company uses many data points to analyze customer content preferences, increase relevance in their communications, and personalize the content—all to keep visitors and traffic. The data also helps the company understand time of day consumption based on both PC and mobile channels.
On a daily basis, UPS makes 16.9 package and document deliveries every day and over 4 billion items shipped per year through almost 100,000 vehicles. With this volume, there are numerous ways UPS uses big data, and one of the applications is for fleet optimization. On-truck telematics and advanced algorithms help with routes, engine idle time, and predictive maintenance. Since starting the program, the company has saved over 39 million gallons of fuel and avoided driving 364 million miles. The next steps include completion of the roll-out and applying the operational efficiency to their airplanes.
Sprint spoke about using big data analytics to improve quality and customer experience while reducing network error rates and customer churn. They handle 10s of billions of transactions per day for 53 million users, and their big data analytics put real-time intelligence into the network, driving a 90% increase in capacity.
Similarly, EMC data scientists looked at 5 billion cell user CDR signals per day to save tens of millions of dollars with analytics. The project helped identify service issues and avoid needless, costly repair work.
The American Express Company looked to shift traditional business intelligence-based hindsight reporting or trailing indicators of how business was doing to predict loyalty. Their sophisticated predictive models analyzed historical transactions with 115 variables to forecast potential churn. In the Australian market, they now believe they can identify 24% of accounts that will close within four months.
American International Group (AIG) uses big data and data visualization to help fight fraud. The system takes structured and unstructured data from claims databases and handwritten adjuster notes to identify potential fraud. Besides listing priority claims to investigate, charts and visualizations, like heat maps, inform teams of other insights and also help them make improvements to machine learning algorithms.
Of course, Tesla is the poster child for instrumenting vehicles with sensors and sending all the data back to the mother ship for analysis, using an Apache Hadoop® cluster to collect the data. The data is used to improve the company’s R&D, car performance, car maintenance, and customer satisfaction. For example, the company is notified if the car is not functioning properly and consumers can be advised to get a service. These capabilities have helped Tesla create market share in a difficult environment where charging stations are not widely deployed.
As shared this past CES, Ford’s CEO explained how the company is using big data and transportation analytics to become more of a technology company. They are looking to use big data to address vehicle quality, insurance costs, transportation, vehicle intelligence, driving patterns, and more. For example, the company wants to use the data to help lower insurance costs for the driver and even has approximately 200 big data and analytics experts supporting major decisions throughout the company. In a marketing example, the company analyzes multiple data streams on what was built, sold, in inventory at the time of sale, and what customers are searching for on websites along with economic data such as housing starts and employment rates—all of this is used to help sell more cars.
The healthcare industry has historically used data in many ways, but big data and data science is changing the way businesses operate as showcased in our recent case study.
At Kaiser Permanente, big data from electronic health records for 9 million people have been used to help improve care and reduce costs, improve recommendations for care, support adherence to prescriptions, and more. Recently, Kaiser used big data to study the incidence of blood clots within a group of women taking oral contraceptives. The analysis revealed that one formula contained a drug that increased the threat of blood clots by 77%—understanding these types of patterns can help many people avoid visits to the doctor or emergency room. Similarly, Kaiser doctors are studying preterm and newborn babies who visit NICUs and determine the type of care needed for diagnosing infections like sepsis or others within blood streams as well as reducing the number of days infants spend in NICU.
At Emory University Hospital, a new system is underway to help identify and alert clinicians to danger signs in patents. Typically, streams of data like heart physiology, respiration, brain waves, and blood pressure stream out of a dozen systems and monitors in an ICU. Doctors and nurses monitor the information to make decisions. New systems are helping correlate the massive streams of data in real-time to provide analysis much faster and help save lives.
In professional, online education, Skillsoft is using big data to learn and apply knowledge across 19 million users and 60,000 learning assets. Content has been individualized based on direct email response behavior and surveys. Since applying big data approaches, there has been a 128% improvement in user engagement and recommendations have proven to be much more relevant and actionable. The company’s leadership see that the analysis of big data has generated substantial results and trusted advanced machine learning and optimization algorithms to deliver. Similar results in the education space have also been produced by Pivotal.
As shared at the Strata + Hadoop conference, online game developer Jagex is using big data to cull through 10 years of game content and 220 million player accounts to recommend relevant, in-game content to users. One of their top games, RuneScape, was a free, massively multiplayer online role playing game (MMORPG). With more relevant content, the game could actually improve revenue instead of damage it. In fact, the deployed data science model not only increased revenue, it improved player engagement and quest completion rates.
For Red Roof Inn, big data has created a new way of marketing and helped produce 10% growth year over year, even with the worst winter in years. The marketing department started with an inventory of potential data to use, including government stats and nationwide, historical weather information, and they began a plan to target stranded airport passengers. With an estimated 2-3% of flights cancelled daily, 500 planes don’t take off and 90,000 passengers get stranded. The company uses big data to identify the areas of demand and uses search advertising, a focus on mobile communications, and other methods to drive digital bookings with personalized messages like ‘Stranded at O’Hare? Check out Red Roof Inn.’ The ads provided a timely and contextual offer—showing up at the right time and right place for the right user’s search criteria.
The IRS is using big data to stop identity theft, fraud, and improper payments, such as those who are not paying taxes and should. The system also helps to ensure compliance with tax rules and laws. So far, the IRS has stopped billions of dollars in fraud, specifically with identity theft, and recovered more than $2 billion over the last three years.
In Chicago, public sensors are being placed around the city, and these are expected to measure air quality, light intensity, sound volume, heat, precipitation, wind, and cell phone traffic to count pedestrians (without sacrificing privacy laws). The ultimate goal is to make the city a safer, more efficient, and cleaner place to live while attracting technical researchers and, rolling out in phases, to offer the city a new public utility.
In an example recently published in the Economist, a data analysis firm helped a multinational engineering firm improve team performance. When they looked at dozens of variables on team composition, they could see how a set of small improvements would add up. By looking at the number of members on a team, the time-zone spread, the relationship between workers, and other factors, the improvements ultimately added up to a 22% increase in productivity.
As showcased, there are a number of companies across industries who are using big data to inform their business and take actions that produce results.
Editor’s Note: Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.