Surveys Are Mostly Bullshit

You’ve probably all experienced something like this: You purchased something you never thought about much, a Hyundai Santa Fe or a Yorkshire Terrier, for example, and now you can’t stop seeing them everywhere. It’s almost like when Instagram starts targeting ads for you, only it happens in the real world and there are no algorithms at work. This is known as the Baader-Meinhof phenomenon, more commonly referred to as frequency illusion.

My personal Baader-Meinhof are surveys. Before Animal Spirits became an anti-survey podcast, I can’t remember paying attention to any of them. Now they’re all I see. And the more I see, the stronger I feel about the fact that surveys are mostly bullshit.

Here is Seth-Stephens Davidowitz on the matter in his excellent book, Everybody Lies:

Everybody lies. People lie about how many drinks they had on the way home. They lie about how often they go to the gym, how much those new shoes cost, whether they read that book. They call i sick when they’re not. They say they’ll be in touch when they won’t. They say it’s not about you when it is. That say they love you when they don’t. They say they’re happy while they’re in the dumps. They say they like women when they really like men. People lie to friends. They lie to bosses. They lie to kids. They lie to parents. They lie to doctors. They lie to husbands. They lie to wives. They lie to themselves. And they damn sure lie to surveys.

Why is this the case? In my estimation, a survey of one if you will, there are three main reasons why people don’t tell the truth, which I’ve pie-charted below.

They don’t know the truth.

People don’t know how they really feel. Nick’s poll is a perfect example of this.

It’s a thought provoking question, but it’s a question that can’t be answered because hypothetical questions are for finding out what people think they think, not what they actually think. The results are encouraging, people value things other than money, but if push came to shove, I think more than 10% of people would make this trade for 5x their money. It’s also worth mentioning that Nick’s audience probably isn’t living paycheck to paycheck, which I’ll address in a minute.

There is no incentive to tell the truth. According to a recent survey of affluent millennials, 30% of respondents admitted to lying about their spending or investing habits. If you believe that only 30% of people admitted to lying, I’ve got a far out-of-the-money option to sell you.

They answer the way they think they’re supposed to answer. This is perfect. 37% of Americans say they would not vote for a candidate older than 70. There is a high  likelihood (>95%?) that our next President will be older than 70. ¯\_(ツ)_/¯

There is a fourth reason why surveys are mostly bullshit and it has nothing to do with the people answering the poll, but rather those asking the questions. A Bank of America survey showed that “1 in 6 millennials has $100,000 saved.” You don’t even need to know the data to know that this is nonsense.

Korina Moss read the fine print and found that this survey was taken from 1,500 people who have long-term savings accounts at Bank of America. Thanks to Korina, this tweet was ratio’d into the stone age, exposing the bullshit that are most surveys.

Not all surveys are bunk. For example what’s your favorite (Scorcese movie, Rolling Stones Song, etc). Even if my favorite Stones song might not be the same on any given day, get enough answers and you’ll get close to finding out what their most popular song is. But most surveys, at least on my experience, ask a variation of a question that cannot reliably be answered: “What will you do in the future.”

I don’t know what I’m having for dinner tonight, let alone know how much I’ll spend on my children’s education or when I’ll retire or how long I’ll be in my primary residence. *Morgan Freeman voice* I have to remind myself that some questions aren’t meant to be answered.

This content, which contains security-related opinions and/or information, is provided for informational purposes only and should not be relied upon in any manner as professional advice, or an endorsement of any practices, products or services. There can be no guarantees or assurances that the views expressed here will be applicable for any particular facts or circumstances, and should not be relied upon in any manner. You should consult your own advisers as to legal, business, tax, and other related matters concerning any investment.

The commentary in this “post” (including any related blog, podcasts, videos, and social media) reflects the personal opinions, viewpoints, and analyses of the Ritholtz Wealth Management employees providing such comments, and should not be regarded the views of Ritholtz Wealth Management LLC. or its respective affiliates or as a description of advisory services provided by Ritholtz Wealth Management or performance returns of any Ritholtz Wealth Management Investments client.

References to any securities or digital assets, or performance data, are for illustrative purposes only and do not constitute an investment recommendation or offer to provide investment advisory services. Charts and graphs provided within are for informational purposes solely and should not be relied upon when making any investment decision. Past performance is not indicative of future results. The content speaks only as of the date indicated. Any projections, estimates, forecasts, targets, prospects, and/or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others.

The Compound Media, Inc., an affiliate of Ritholtz Wealth Management, receives payment from various entities for advertisements in affiliated podcasts, blogs and emails. Inclusion of such advertisements does not constitute or imply endorsement, sponsorship or recommendation thereof, or any affiliation therewith, by the Content Creator or by Ritholtz Wealth Management or any of its employees. Investments in securities involve the risk of loss. For additional advertisement disclaimers see here:

Please see disclosures here.