You’ve probably all experienced something like this: You purchased something you never thought about much, a Hyundai Santa Fe or a Yorkshire Terrier, for example, and now you can’t stop seeing them everywhere. It’s almost like when Instagram starts targeting ads for you, only it happens in the real world and there are no algorithms at work. This is known as the Baader-Meinhof phenomenon, more commonly referred to as frequency illusion.
My personal Baader-Meinhof are surveys. Before Animal Spirits became an anti-survey podcast, I can’t remember paying attention to any of them. Now they’re all I see. And the more I see, the stronger I feel about the fact that surveys are mostly bullshit.
Here is Seth-Stephens Davidowitz on the matter in his excellent book, Everybody Lies:
Everybody lies. People lie about how many drinks they had on the way home. They lie about how often they go to the gym, how much those new shoes cost, whether they read that book. They call i sick when they’re not. They say they’ll be in touch when they won’t. They say it’s not about you when it is. That say they love you when they don’t. They say they’re happy while they’re in the dumps. They say they like women when they really like men. People lie to friends. They lie to bosses. They lie to kids. They lie to parents. They lie to doctors. They lie to husbands. They lie to wives. They lie to themselves. And they damn sure lie to surveys.
Why is this the case? In my estimation, a survey of one if you will, there are three main reasons why people don’t tell the truth, which I’ve pie-charted below.
They don’t know the truth.
People don’t know how they really feel. Nick’s poll is a perfect example of this.
Poll: How much $ would someone have to give you to live the next year of your life (i.e. 2020) with no memory?
Assume your working memory still functions, just that you cannot form any long-term memories for 1 year.
— Nick Maggiulli (@dollarsanddata) November 4, 2019
It’s a thought provoking question, but it’s a question that can’t be answered because hypothetical questions are for finding out what people think they think, not what they actually think. The results are encouraging, people value things other than money, but if push came to shove, I think more than 10% of people would make this trade for 5x their money. It’s also worth mentioning that Nick’s audience probably isn’t living paycheck to paycheck, which I’ll address in a minute.
There is no incentive to tell the truth. According to a recent survey of affluent millennials, 30% of respondents admitted to lying about their spending or investing habits. If you believe that only 30% of people admitted to lying, I’ve got a far out-of-the-money option to sell you.
They answer the way they think they’re supposed to answer. This is perfect. 37% of Americans say they would not vote for a candidate older than 70. There is a high likelihood (>95%?) that our next President will be older than 70. ¯\_(ツ)_/¯
37% of Americans say they would not vote for a candidate older than 70.
Donald Trump is 73
Elizabeth Warren is 70
Joe Biden is 76
Bernie Sanders is 77Resisting the urge to make a joke, I think this says a lot about polls / surveys.
What people say ≠ what they do. https://t.co/wwbwBD5yEo
— Christian Kaylor (@KaylorChristian) October 23, 2019
There is a fourth reason why surveys are mostly bullshit and it has nothing to do with the people answering the poll, but rather those asking the questions. A Bank of America survey showed that “1 in 6 millennials has $100,000 saved.” You don’t even need to know the data to know that this is nonsense.
It’s from a survey of 1500 people ages 23-37 who have long term savings accounts at Bank of America. So 1 in 6 of them have that saved. That’s NOT equivalent to 1 in 6 millennials. So misleading!!
— Korina Moss Author, cheese shop mystery series (@KorinaLMoss) October 21, 2019
Korina Moss read the fine print and found that this survey was taken from 1,500 people who have long-term savings accounts at Bank of America. Thanks to Korina, this tweet was ratio’d into the stone age, exposing the bullshit that are most surveys.
Not all surveys are bunk. For example what’s your favorite (Scorcese movie, Rolling Stones Song, etc). Even if my favorite Stones song might not be the same on any given day, get enough answers and you’ll get close to finding out what their most popular song is. But most surveys, at least on my experience, ask a variation of a question that cannot reliably be answered: “What will you do in the future.”
I don’t know what I’m having for dinner tonight, let alone know how much I’ll spend on my children’s education or when I’ll retire or how long I’ll be in my primary residence. *Morgan Freeman voice* I have to remind myself that some questions aren’t meant to be answered.
