10 very real reasons polls get it wrong 0

Posted on November 13, 2016 by Rob Petersen


Polls are a record of public opinion. Here are polls taken on Monday, November 7, the day before election day, from very reliable organizations.

  • Clinton 44%, Trump 41%, Johnson 4%, Stein 2% (Bloomberg)
  • Clinton 44%, Trump 39%, Johnson 6%, Stein 3% (Reuters/Ipsos)
  • Clinton 45%, Trump 41%, Johnson 5%, Stein 2% (Economist)
  • Clinton 48%, Trump 44%, Johnson 3%, Stein 2% (FOX News)

What happened?

Polls have been a part of elections since the country was founded. The language of the Declaration of Independence requires we function with “the consent of the governed.” But this election shook up a lot of things. One of them was our faith in polls.

Should we conclude polls and the people who conduct now don’t know what they’re doing? Or, is it that good analysis is always depends on quality data and a sound methodology.

Judge for yourself. Here are 10 very real reasons polls get it wrong.

  1. SAMPLING: Probability sampling is the fundamental basis for all polls. The basic principle: A randomly selected small sample of a population represents the attitudes, opinions and projected behavior of all people. But random samples almost never occur organically.
  2. SAMPLE RESPONSE RATES. For example, women and older Americans tend to answer the phone more often. This is how most polls are still conducted. This throws off the sex and age ratios of the sample. Instead of relying exclusively on random number dialing, pollsters take the extra step of adjusting or weighting results to match the demographic profile of likely voters.
  3. NON-RESPONSE RATES: Adding to problem of creating a random sample, response rates are way down. In 1997, Pew Research, a very well respected research and polling organization, saw telephone response rates were 36%. By 2012, Pew reported a downward trend to an average response rate of 9%.
  4. WEIGHTING: Since it is virtually impossible for a company conducting polls to expect a random sample much less that participants even answer their phones, weights are assigned to demographic characteristics of the total sample of respondents to match the latest estimates of demographic characteristics available from the U.S. Census Bureau. Weighting has a major impact on the results of polls.
  5. CENSUS RESULTS: Census results reflect hard facts such as age, race, address and family size. They do not reflect characteristics like religion and group affiliations. Beliefs and values that are more likely to determine people’s actions.
  6. BRADLEY EFFECT: We don’t always say in polls what we do. It’s called the Bradley Effect, after Tom Bradley, an African-American candidate for governor of California in 1982. Polls incorrectly predicted he would win. Looking back, experts think that’s because people told pollsters they would vote for Bradley, even though they didn’t plan to, in order to avoid sounding racist.
  7. PHONE SURVEYS: The majority of political polls are still surveys done by phone. That’s because someone’s email is more private and protected than their phone number. Surveys conducted over the phone are a pretty antiquated way to conduct research in the computer age. On the phone, the Bradley effect is more likely to occur than online because someone else is hearing and recording your answers. CNET reported Trumps polls a lot better online than in a polls conducted over the phone.
  8. GROUPS: Census numbers can tell us how many Asian-Americans live in a particular state. They can’t reliably tell us how many conservatives or evangelicals are in that state or groups that systematically exclude themselves from polls at higher rates than other groups. There’s no easy way to fix the problem and know the group that someone belongs.
  9. MULTIPLE AFFILIATIONS: Even if pollsters could reliably align weighted samples with groups, none of us are singularly dedicated to one group. We have multiple affiliations. We belong to a particular religion, participate at a certain level in community affairs and have specific views on the environment. So, even if polls could accurately correlate Census information with groups, there are multiple factors and sub-segments to consider.
  10. EXIT POLLS: In any race, there is a fascination with who is likely to be the winner. So there are exit polls to gauge how the race is going. They’re usually based on a sample of a few dozen precincts or so in a specific state, sometimes not even including many more than 1,000 respondents. Like every other type of survey, they’re subject to a margin of error because of sampling and additional error resulting from various forms of response bias.

Did these reasons explain to you how polls get it wrong? Does your organization need guidance understanding data and its results?


10 reasons agencies and consultancies battle over data 0

Posted on May 23, 2016 by Rob Petersen


agencies and consultancies

An agency is an organization established to provide a particular service. A consultancy is a professional practice that gives expert advice within a particular field. Agencies and consultancies have always tried to do what the other does. But it’s accelerating at a more rapid pace these days.

Consultancies like Deloitte, Accenture, KPMG and PwC. Even McKinsey are building agency arms. Tech companies like Adobe, Oracle and Epsilon have added a service component in the form of an agency to their core product offering. Publicis bought technology consultancy and agency services network Sapient a few years ago.

What’s behind the activity? Why?

Here are 10 reasons the battle between agencies and consultancies is over data from 10 experts.

  1. MARKETING AND BUSINESS PERFORMANCE CAN NOW BE CONNECTED: “Agencies have never had a bigger opportunity to differentiate themselves from the competition via high-value, strategic services. Digital provides hard-wired connectivity between marketing activity and business performance. As a consequence, it enables agencies to assume an absolutely integral role in the fabric of their clients’ businesses.” – Damian Burns, Director of Global Agency Sales, Google
  2. DATA REQUIRES SOMEONE TO DIGEST IT: “There’s so much more information available about business performance, consumers, what’s happening with marketing campaigns. The expectation is that [marketers] would be able to digest all that and be able to know what to do next and do that very quickly. That is incredibly complicated.” – Jason Harrison, CEO, Gain Theory
  3. DATA ESTABLISHES RESULTS-ORIENTATION: “Previously, a consulting services firm would be hired based on their reputation and relationship. While older agency directors and C-suite executives still hire consultants and government contractors according to legacy criteria, Millennial (and Millennial-minded) leaders will pass over these candidates in favor of more results-oriented professional advisers.” – John Diller, President, Big Sky
  4. DATA ANALYTICS IS COMPETITVE ADVANTAGE: “Many agencies are now beginning to consider consolidating with businesses that offer consulting capabilities. data and the rise of multi-channel media consumption have completely disrupted the way businesses deliver marketing messages. Emerging digital marketing challenges have also spurred a ‘data analytics arms race,’ where whoever has the most robust business intelligence solution is thought to have a competitive advantage.” – Jay Sampson, Vice President of Partner Sales, Adobe
  5. DATA BREAKS DOWN SILOS WITH STRATEGY THAT IMPACTS SALES: “Companies have been talking about customer-centricity forever but with the emergence of roles like the chief data officer and the chief customer officer they finally have someone internally who can work across divisions and truly break down those silos. They now need a new type of service provider that can help create an integrated strategy that will impact sales and market share.” – Gene Hartman, Managing Director, Accenture
  6. BRAND PREFERENCE HAPPENS WITH A CLICK: “The time between consumption of a brand story through mass media to the actual experiencing of the brand has shrunk to a single click. It is not difficult to understand what is attracting management consultants into marketing services.” – Avi Dan, CEO, Avidan Strategies
  7. DISRUPTION OF CLASSIC STRATEGY: “New competitors with new business models arrive. Although these upstarts are as yet nowhere near the size and influence of big-name consultancies like McKinsey, Bain, and Boston Consulting Group (BCG), the incumbents are showing vulnerability. For example, at traditional strategy-consulting firms, the share of work that is classic strategy has been steadily decreasing and is now about 20%, down from 60% to 70% some 30 years ago.” – Tom Rodenhauser, Managing Director, Kennedy Consulting Research & Advisory
  8. STRATEGY REQUIRES DELIVERY CAPABILITIES: “It is no coincidence that management consultancies are acquiring the more technically skilled digital agencies – they are coming from the other direction and want to be able to determine the strategy and deliver the higher margin activities that flow from it. They need to prove delivery capability but will not be interested in the lower margin activities – they can simply buy these in as needs be.” – Green Square, Corporate Financial Advisors
  9. CMO IS NEW CIO: By 2017, Gartner, the technology research company, estimates that the largest portion of a company’s IT spend will be controlled by the CMO instead of the CIO, from data and analytics to front and back-end IT spend.” – Avi Dan, CEO, Avidan Strategies
  10. AGENCIES AND CONSULTANCIES HAVE TO BE DATA DRIVEN: “Agencies are more responsible for marketing and customer management data, but as data becomes more of a driver, agencies need to be more savvy data analysts and integrators. Clients need to hold consultancies to the same standards as agencies. But agencies shouldn’t try to become consulting companies. Instead, they should continue to do what they do best: Specialize and encourage a culture that breeds smart marketing people.” – Peter Figueredo, Head of Client Services, Kaizen

Do these reasons show you why agencies and consultancies battle over data? Does your organization need a data-driven company working in your behalf?

9 non-technical steps to better data-driven decision making 0

Posted on June 14, 2015 by Rob Petersen



Avinash Khausik's Glorious Executive Dashboard

Avinash Khausik’s Glorious Executive Dashboard

Data and algorithms have a tendency to outperform human intuition. As a result, companies are vacuuming up data to make better decisions about everything from product development, marketing, advertising to hiring.

But it’s not the data, it’s what you do with it. Because data doesn’t make decisions, people do. Insights and actions are as important as statistical analysis.

Here are 9 non-technical steps to better data-driven decision making.

  1. UNDERSTAND THE REAL BUSINESS QUESTIONS: Ask key stakeholders who will be making decision from the data: “What are the question you want the data to answer? What’s the context and who are the impacted segments?”
  2. PICK THE RIGHT METRICS: There is a difference between numbers and numbers that matter. Good metrics are consistent, cheap, and quick to collect. They have to capture capture what your business cares most about.
  3. ASK THE RIGHT QUESTIONS OF THE DATA: Managers have a critical role to play framing questions before the analysis before the analysis begins. What is the source of the data? How well does the data represent the population or the audience that is most important for the analysis?
  4. CENTRALIZE WHERE DATA RESIDES: A firm needs to take inventory of its key data properties, as well as identify the specific business needs and functions. A critical success factor is a proper data architecture and pairing of data within a business function for impacts and inter-dependencies.
  5. PROCESS TRUMPS TECHNOLOGY: Data-driving decision making involves gathering, analyzing, reporting and taking action. It’s a that happens to happen regularly and involve a team of people who have specific functions to be effective.
  6. IT’S GOING TO COST MORE MONEY AND TIME: The process itself demands a serious commitment to training, data cleanup, and maintenance. End users have to trained on tools, how to use data for analysis, how the data is structured and what is data-warehouse security.
  7. KNOW THE BASICS OF DATA VISUALIZATION: For data to tell a story, it has to be visualized effectively. Before considering the technique or software for data visualization, look at who is the audience for the analysis; how much do you want to show them to convince them of the insights, actions and financial implications to be taking from the data
  8. CORRELATION IS NOT CAUSE AND EFFECT: Just because a correlation exists doesn’t mean there is a cause and effect. The frequency and length of time for the correlation plays a big role. This minimizes the risk of taking or not taking action, but never ensures cause and effect and the benefits of being right or wrong.
  9. DISCOVER INSIGHTS, MAKE RECOMMENDATIONS AND TAKE ACTIONS: Reporting is not analysis and analysis is not results. To get to productivity and profitability rates cited above, data-driven decision can’t happen without insights, recommendation and actions taking all along the way.

Do these steps help you understand how to achieve data-driven results? Does your company operate this way? Are you ready to begin?



  • About

    BarnRaisers builds brands with proven relationship principles and ROI. We are a full service digital marketing agency. Our expertise is strategy, search and data-driven results.

↑ Top