November 13, 2016 by
Polls are a record of public opinion. Here are polls taken on Monday, November 7, the day before election day, from very reliable organizations.
- Clinton 44%, Trump 41%, Johnson 4%, Stein 2% (Bloomberg)
- Clinton 44%, Trump 39%, Johnson 6%, Stein 3% (Reuters/Ipsos)
- Clinton 45%, Trump 41%, Johnson 5%, Stein 2% (Economist)
- Clinton 48%, Trump 44%, Johnson 3%, Stein 2% (FOX News)
Polls have been a part of elections since the country was founded. The language of the Declaration of Independence requires we function with “the consent of the governed.” But this election shook up a lot of things. One of them was our faith in polls.
Should we conclude polls and the people who conduct now don’t know what they’re doing? Or, is it that good analysis is always depends on quality data and a sound methodology.
Judge for yourself. Here are 10 very real reasons polls get it wrong.
- SAMPLING: Probability sampling is the fundamental basis for all polls. The basic principle: A randomly selected small sample of a population represents the attitudes, opinions and projected behavior of all people. But random samples almost never occur organically.
- SAMPLE RESPONSE RATES. For example, women and older Americans tend to answer the phone more often. This is how most polls are still conducted. This throws off the sex and age ratios of the sample. Instead of relying exclusively on random number dialing, pollsters take the extra step of adjusting or weighting results to match the demographic profile of likely voters.
- NON-RESPONSE RATES: Adding to problem of creating a random sample, response rates are way down. In 1997, Pew Research, a very well respected research and polling organization, saw telephone response rates were 36%. By 2012, Pew reported a downward trend to an average response rate of 9%.
- WEIGHTING: Since it is virtually impossible for a company conducting polls to expect a random sample much less that participants even answer their phones, weights are assigned to demographic characteristics of the total sample of respondents to match the latest estimates of demographic characteristics available from the U.S. Census Bureau. Weighting has a major impact on the results of polls.
- CENSUS RESULTS: Census results reflect hard facts such as age, race, address and family size. They do not reflect characteristics like religion and group affiliations. Beliefs and values that are more likely to determine people’s actions.
- BRADLEY EFFECT: We don’t always say in polls what we do. It’s called the Bradley Effect, after Tom Bradley, an African-American candidate for governor of California in 1982. Polls incorrectly predicted he would win. Looking back, experts think that’s because people told pollsters they would vote for Bradley, even though they didn’t plan to, in order to avoid sounding racist.
- PHONE SURVEYS: The majority of political polls are still surveys done by phone. That’s because someone’s email is more private and protected than their phone number. Surveys conducted over the phone are a pretty antiquated way to conduct research in the computer age. On the phone, the Bradley effect is more likely to occur than online because someone else is hearing and recording your answers. CNET reported Trumps polls a lot better online than in a polls conducted over the phone.
- GROUPS: Census numbers can tell us how many Asian-Americans live in a particular state. They can’t reliably tell us how many conservatives or evangelicals are in that state or groups that systematically exclude themselves from polls at higher rates than other groups. There’s no easy way to fix the problem and know the group that someone belongs.
- MULTIPLE AFFILIATIONS: Even if pollsters could reliably align weighted samples with groups, none of us are singularly dedicated to one group. We have multiple affiliations. We belong to a particular religion, participate at a certain level in community affairs and have specific views on the environment. So, even if polls could accurately correlate Census information with groups, there are multiple factors and sub-segments to consider.
- EXIT POLLS: In any race, there is a fascination with who is likely to be the winner. So there are exit polls to gauge how the race is going. They’re usually based on a sample of a few dozen precincts or so in a specific state, sometimes not even including many more than 1,000 respondents. Like every other type of survey, they’re subject to a margin of error because of sampling and additional error resulting from various forms of response bias.
Did these reasons explain to you how polls get it wrong? Does your organization need guidance understanding data and its results?
November 07, 2016 by
Data mining is: 1) The practice of examining large databases to generate new information and 2) the process of analyzing data from different perspectives to make it insightful and useful.
Data mining is used by companies to increase revenue, decrease costs, identify customers, provide better customer service, listen to what others are saying and do competitive intelligence. And that’s just some of the ways.
Here’s are 20 companies that do data mining and prove it makes their business better.
- AMAZON: With $5 off, for those who use the Amazon Price Check Mobile App – to scan the products in store, take a picture of the product or perform a text search to find the lowest prices, the app also prompts the customers to submit the in-store price. Amazon is collecting intelligence and valuable pricing information from its competitors.
- ARBY’S: The fast food company uses data mining to help them determine the best targets for their advertisements. They can see which advertisements are most effective, while seeing the channels that are most receptive to each ad pitch. This allows them to ensure every advertisement utilizes the appropriate channel to increase the number of leads from their marketing.
- CAPITAL ONE: Data mining and big data management to help them ensure the success of all customer offerings. By analyzing the demographic data and spending habits of customers, they’re able to determine the most optimal times to present various offers to clients, thus increasing the conversion rates of their offers and gaining more leads from their marketing budget.
- DELTA: Large airlines like Delta, monitors tweets to find out how their customers feel about delays, upgrades and in-flight entertainment. When a customer tweets negatively about his lost baggage, the airline forwards to their support team. The support team sends a representative to the passengers destination presenting him a free first class upgrade ticket on his return along with the information about the tracked baggage promising to deliver it as soon as he or she steps out of the plane.
- DUETTO: Known online for their “hotel optimization,” Duetto makes it easier for companies to personalize data to individuals searching online for hotels. Duetto makes it easy for hotels to personalize their prices by taking data such as how much you typically spend at the bar or casino to incentivize you with a lower price for your room. Therefore the hotel can give you a better price, knowing you’ll spend money on other services. The hotel can give you a better price, knowing you’ll spend money on other services.
- EXPRESS SCRIPTS: Which processes pharmaceutical claims, realized that those who most need to take their medications were also those most likely to forget to take their medications. So they created a new product: Beeping medicine caps and automated phone calls reminding patients it’s time to take the next dose.
- FREE PEOPLE: The more bohemian segment of Urban Outfitters, uses millions of customer records (reviewed by an in house analytics team) to shape the next season’s offerings. Information like what sold, what didn’t, what was returned and more fuels the brand’s product recommendations, the look of its website and what kinds of promotions customers see to improve Free People’s bottom line.
- GOOGLE AND CENTER FOR DISEASE CONTROL (CDC): Google proposed a different approach. Using historical data from the CDC, Google compared search term queries against geographical areas that were known to have had flu outbreaks. Google found spikes in certain search terms where flu outbreaks occurred and identified forty-five terms that were strongly correlated with the outbreak of flu. Google then started tracking the use of those terms and is now able to accurately predict when a flu outbreak is occurring in real time. Using this data, the CDC can act immediately.
- KOHL’S: Customers are more likely to respond to an offer when it’s at the moment of purchase. That’s why Kohl’s does real-time, personalized offers. Shoppers can opt in for offers via their smartphones. So if a shopper lingers in the shoe department, for example, they can receive a coupon on the shoes they looked at online but never bought,
- KREDITECH: European company, uses more than 8,000 sources including social media, to create a unique credit score for consumers, which is then sells to banks and other lenders. And they have discovered some surprising correlations between social media behaviors and financial stability. For example, if your Facebook friends use all capital letters, your score is docked.
- MACY’S: Through sentiment analysis of big data, Macy’s finds out that people who are sharing tweets about “Jackets” are also making use of the terms “Michael Kors” and “Louis Vuitton” frequently. This information helps the retailer to identify what brands of jackets should be offered discounts in their future advertising campaigns to attract customers.
- MCDONALD’S: With more than 34K local restaurants serving 69 million customers across 118 countries , 62 million daily customer traffic, selling 75 burgers every second, $27 billion annual revenue- McDonald’s is using big data analytics to gain lot more insight to improve operations at its various stores and enhance customer experience. McDonald’s analytics system analyse data about various factors such as wait times, information on the menu, the size of the orders, ordering patterns of the customers.
- NETFLIX: To create data models and find what makes show or movie popular among consumers, according to the insights they gained from their data, House of Cards was the ultimate entertainment experience. They went all out, winning a bidding war with other companies over the rights and immediately scheduled two seasons of content before showing a thing. It was a huge success, and the best part is they almost knew it would be.
- NORDSTROM: With 225 retail outlets, Nordstrom generates petabytes of data from its 4.5 million Pinterest followers, 300,000 Twitter followers and 2 million likes on Facebook. Their analytics system monitors customer behaviour by tracking – How many people enter the store, which section they walk in, how long they stay at the store and for how long they shop in a particular section. This helps Nordstrom decide what products should be promoted to which customers when and through what advertising channel.
- PANDORA: With 72 million users and the data for approximately 200 million users’ listening habits, Pandora is a name to reckon with in the music industry for providing music recommendations that people really love. Apart from the data like gender, age, zip code that users provide at sign up, Pandora tracks all the songs that a particular user likes and dislikes, from which location they listen, from which devices they listen and more – to provide customers with curated music catalogue based on interests and demographics.
- PREDPOL: The Los Angeles and Santa Cruz police departments, a team of educators and a company called PredPol have taken an algorithm used to predict earthquakes, tweaked it and started feeding it crime data. The software can predict where crimes are likely to occur down to 500 square feet. In LA, there’s been a 33% reduction in burglaries and 21% reduction in violent crimes in areas where the software is being used.
- STARBUCKS: As the leading coffeehouse company in the world, Starbucks manages to open new stores in very close proximity with their other stores, while still guaranteeing a high success rate. Normally, when expanding a company, it’s needlessly risky to open a new location just a block from another location.
- T-MOBILE: Data mining helps reduce customer turnover rate. By analyzing big data, T-Mobile can determine the core causes for turnover, allowing them to implement effective solutions that will keep more clients on board. As a telecom company, they accrue boundless quantities of data every year, and without big data management, the ability to analyze the data would be greatly inhibited.
- THOMSON REUTERS: Financial experts can gain competitive advantage by analysing the twitter sentiment data by tracking specific tweets from various companies and people. This helps financial professionals get an overview on the number of positive and negative sentiments related to any given company. Sentiment analysis along with other advanced big data analytics solution helps the financial professionals spot the financial market and any events impacting the company as they happen.
- WALMART: The mega-retailer’s latest search engine for Walmart.com includes semantic data. Polaris, a platform that was designed in-house, relies on text analysis, machine learning and even synonym mining to produce relevant search results. Wal-Mart says adding semantic search has improved online shoppers completing a purchase by 10% to 15%. “In Wal-Mart terms, that is billions of dollars,” Laney said.
Do these companies prove how data mining could make your business better? Could your company benefit from data mining?
April 18, 2016 by
Data analytics as a business asset is changing dramatically. That’s because companies are applying data analytics to new problems and making data a driver of innovation.
Data analytics is the science of examining raw data to drawing conclusions that help companies make better business decisions. How?
Here are 14 companies seizing data analytics as a business asset.
- AMERICAN EXPRESS: Started looking for indicators that could really predict loyalty. They developed sophisticated predictive models to analyze historical transactions and 115 variables to forecast potential churn. The company believes it can now identify 24% of accounts that will close within the next four months.
- BOEING: Used data and devices to transform the company from an aircraft manufacturer to an aero-health service provider. To reduce its airline customer’s total cost of ownership, Boeing offered customers a data-based Airplane Health Management service. Performance data can be wirelessly transmitted from each Boeing aircraft directly to the fleet operator for real-time fault management, performance monitoring and customized alerts. The data service allowed Boeing customers to make fix-or-fly decisions quickly, which in turn, helped the airline improve maintenance efficiency and reduce servicing costs.
- CITY OF BOSTON: Introduced Street Bump, an app which enables people to use their smartphone’s accelerometer—a motion detector in the device—to record road conditions and send data to public works employees. With the Street Bump app, citizens simply turns on the app and, as the drive, data is automatically collected and sent to the city. Street Bump was expected to identify the location of potholes—a top concern of Boston residents. Thanks to analytics, the early data has provided some unexpected insights: trouble spots are eight times more likely to be “castings,” those manhole covers, grates and other cast metal lids that are supposed to be flush with the roadway surface but instead heave up due to the extreme cold of a New England winter. Hundreds of these castings have been repaired as a result.
- CONAGRA: Faced the challenges of figuring out the optimal pricing for its products in an environment where consumers are hyper-sensitive, while coping with the ever fluctuating costs for 4,000 raw materials used in some 20,000 products. The company turned to in-memory computing, which loads extremely large volumes of data from multiple sources into one database and enables users to answer questions almost instantly. ConAgra decision-makers had real-time insight into the company’s costs and consumers’ demands. ConAgra also shared data-driven insights with retailers.
- EXPRESS SCRIPTS: Processed pharmaceutical claims and realized that those who most need to take their medications are also those most likely to forget to take their medications. So they created a new product: Beeping medicine caps and automated phone calls reminding patients it’s time to take the next dose.
- INFINITY PROPERTY & CASUALTY CORP.: Realized it had years of adjusters’ reports that could be analyzed and correlated to instances of fraud. It built an algorithm out of that project and used the data to reap $12 million in subrogation recoveries.
- JOHNSONVILLE SAUSAGE: On average, consumer products (CP) companies spend 13.7% of their gross sales on trade promotions, according to a recent study by AMG Strategic Advisors. Immediate insights from analytics enabled Johnsonville to track special offers, such as coupons. With real-time data, Johnsonville correlated the offers with sales performance to adjust the programs based on those insights.
- KAISER PERMANENTE Made a multi-billion dollar investment to build its HealthConnect® health information system. The system securely connects 8.6 million people to their healthcare teams, stores their personal information and provides the latest medical knowledge. In an industry known for chronic high costs and quality issues, the system allowed Kaiser to not just identify and rollout best practices but it also gives the healthcare company a data-driven edge in providing lower-cost and higher-quality care.
- MERCEDES: In the automotive industry, manufacturers must manage a greater number of both models and customization options as they adjust to fragmented consumer demand, while also managing shorter product life cycles. They used real time sensor data from the engines being tested to enable engineers to identify possible problems more quickly. That change, in turn, creates more engine-testing capacity each week and a focus for engineers of refining engine quality. The strategy worked. Mercedes AMG sold more than 32,000 automobiles in a year, their most successful year ever.
- NORWEGIAN CRUISE LINES: Knew it needed to better understand who its customers are, what they value and what they most want to do when onboard the ship. Norwegian implemented a business intelligence (BI) system aimed at delivering insights to business users. From the data, they created onboard food and entertainment packages based on common spending patterns. The data also made it possible for Norwegian to introduce all-inclusive cruising, the first of the major cruise lines to do so.
- TESCO: The supermarket chain collected 70 million refrigerator-related data points coming off its units and fed them into a dedicated data warehouse. Those data points were analyzed to keep better tabs on performance, gauge when the machines might need to be serviced and do more proactive maintenance to cut down on energy costs.
- T-MOBILE: 77% of the most successful U.S. companies use Twitter to communicate with customers and 70% use Facebook. Wireless campaign typically get 1% to 3% negative reactions but sometimes response can go to 12%. T-Mobile used social media for better listening to: 1) Identify trending issues and monitor customer sentiment, 2) Allow the tough conversations to take place, 3) Offer quick responses in real time 3) Make positive connections that deliver on the brand promise. The end result is better customer service. Customer complaints went down by 50% in 3 months because they have been able to come up questions and issues relating to how customers might respond.
- WALMART: The mega-retailer’s latest search engine for Walmart.com includes semantic data. Polaris, a platform that was designed in-house, relies on text analysis, machine learning and even synonym mining to produce relevant search results. Wal-Mart says adding semantic search improved online shoppers completing a purchase by 10% to 15%.
- ZILLOW: provided publicly available mapping of individual home values across the US by combining existing data sources such as transactional history from public records and listings from real estate brokerages with new data from Microsoft-based maps. Its media-based business model continued to expand with nearly 12 million visitors per month and an increasing array of services as Zillow mines the data on visitor actions and continuously builds its data assets.
Do these examples show you how companies are applying data analytics in new and innovative way. Does your organization need guidance using data analytics as a business asset?