Is Apple Making Implausible iPhone Satisfaction Claims?

A 99% approval rating is nearly statistically impossible. Apple says it gets one every year. 

Updated 11/3/23 to reflect the fact Apple has lied about customer satisfaction in their Q3, 2023 and Q4, 2023 earnings calls. Updated 7/14/23 to add a reference to Scott Alexander's Lizardman's Constant and further clarify that this is a satisfaction survey, not a Net Promoter score.

Key Points: 

• In its May earnings call, Apple claimed that the iPhone 14 had a 99% customer satisfaction rating citing 451 Research, a third party research firm it hired. This was a satisfaction survey, not a Net Promoter survey. (Updated 7/14/23).

• Apple has made a similar claim in every single earnings call for the past seven years. 

• A customer satisfaction rating this high is nearly impossible to find using traditional survey methods and a representative sample. 

• Independent sources find iPhone satisfaction ratings significantly lower than what Apple claims. 

• Apple’s claimed iPhone satisfaction is far higher than what other consumer companies and brands claim for beloved products. 

• We asked Apple’s press team to explain how the company arrived at these results. They opened our email 73 times, but didn’t respond. 

• Our best guess is that Apple wants to make bold customer satisfaction claims, and has hired a third-party research firm that will produce those numbers to make them seem more credible – and perhaps to provide plausible deniability.

• That firm is likely using a highly unrepresentative sample of tech enthusiasts and early adopters to generate extraordinary satisfaction numbers that Apple then presents as representative of consumers generally. 

Toward the end of its earnings call in May, Apple made an extraordinary claim: “The latest survey of US Consumers from 451 Research measured customer satisfaction at 99% for the iPhone 14 family,” Apple Chief Financial Officer Luca Maestri said. 

The problem is a 99% satisfaction rate is so high, there’s almost no way to get it using standard survey methods and a representative sample. 

“Even if every single person actually loved their iPhone it would still be nearly impossible to find near unanimous satisfaction using traditional survey methods,” said economist and PerfectRec founder Joe Golden. “A few percent of people just won’t understand the question, or will give the wrong response by mistake,” Golden said. 

And even if it’s not by mistake, a few percent of respondents like to give strange and contrarian answers to pollsters. Scott Alexander called this phenomenon Lizardman’s Constant, in reference to the 4% of survey respondents who told researchers the world was run by lizard men. (Added 7/14/2023)

If Apple’s claims of 99% satisfaction were true, it would mean the iPhone 14 was not only the most popular mass market consumer product of all time, but it would also probably make it the most popular anything of all time. To get a sense of just how extraordinary Apple’s iPhone 14 claims were, we looked at some other satisfaction numbers and approval ratings for context. 

America’s favorite holiday, Thanksgiving, enjoys 82% popularity according to a 2023 survey by Statista. Memorial Day comes in second at 77%. 

The three most popular movies of all time on IMDB are The Shawshank Redemption, The Godfather and The Dark Knight. They have ratings of 9.2, 9.2 and 9.0 respectively.

If we look at some of the most popular retailers in America, we find satisfaction ratings maxing out in the 80s. Trader Joe’s fans will stand in line for hours for groceries but still give the store just an 84% satisfaction rating, according to a survey by the American Customer Satisfaction Institute. 82% of Costco customers say they are satisfied with the retailer, and they pay $60-120 a year for the privilege of shopping there.

What about politics where tribal loyalties tend to trump facts? According to Gallup, President Barack Obama enjoyed an average approval rating across his two terms of just 83% among Democrats. Donald Trump performed a bit better among Republicans while in the White House, averaging an 86% approval rating from members of his own party. Even Syrian dictator Bashar Al-Asad only managed 95% support in the last election, and that was condemned by the United Nations for being undemocratic. 

In fact, the only other poll we could find claiming 99% support for anything was a press release  that said 99% of Americans enjoy ice cream. It was authored by a frozen dessert company. I’d note they didn’t bother sharing methodology for such an implausible claim. 

So what is going on with Apple’s claimed iPhone satisfaction rates? We reached out to Apple to find out, but didn’t get any response. When we kept digging, this is what we found. 

Third-party sources find much lower satisfaction rates for the iPhone

Independent sources do not match Apple’s claimed satisfaction rates. 

The American Consumer Satisfaction Institute regularly surveys consumers about cell phones and wireless carriers. Their latest survey found iPhone satisfaction at 81% for 5G models and 78% for other models. That was the highest combined satisfaction rate of any smartphone manufacturer, but still a far cry from the company’s own findings through 451 Research.

User reviews on Google tell a similar story. People love the iPhone, but not as much as Apple claims. 

While Apple has not published their methodology, a 2018 press release about the survey from its creator, 451 Research shows at least two categories of support: “Very Satisfied” and “Somewhat Satisfied.” If Apple and 451 Research are using a traditional five-point satisfaction scale ranging from “Not At All Satisfied” to “Very Satisfied,” those responses would correspond to four and five stars respectively.  If we map that to ratings for the iPhone on Google Reviews, which uses a five-star scale, we find that  between 87% to 90% of users are rating the iPhone 4+ stars, depending on the model. Those are great reviews, but they are not a 99% satisfaction rate. 

Even if you took the most favorable approach to Apple and said anything that’s not a one-star review counts as satisfied, you would still fall short of Apple’s claimed satisfaction rates. Six percent of users rate the various iPhone 14 models 1 star on Google as of July 6, 2023. That extremely generous scale would translate into a satisfaction rate of 94%, still not the 99% Apple is claiming. 

When other consumer companies brag about satisfaction, they cite much lower numbers 

An analysis of dozens of earnings call transcripts show that executives are usually extremely cautious about satisfaction claims. 

Companies love talking about how happy their customers are, but in the context of heavily scrutinized public earnings calls, they tend to keep their remarks extremely vague. When the word “satisfaction” is used at all, it’s usually part of broad statements like Disney CEO Bob Iger pledging to “increase park capacity while preserving guest satisfaction.” 

We combed through the last four quarters of earnings transcripts from consumer favorites like Costco, Disney and McDonald’s as well as popular electronics companies like Samsung and Sony, for claims about customer satisfaction. The only quantitative claim we found was Sony touting that the PlayStation Plus subscription service enjoyed 72% customer satisfaction in a 2022 investor presentation.

Even outside of investor calls, it’s unusual for consumer-facing companies to make concrete claims about customer satisfaction. One example of the few examples we found was the appliance company Dyson, which thought an 84% customer satisfaction rating for one of its vacuum cleaners was impressive enough to merit a press release.  

Apple, by contrast, has claimed near-universal approval in every investor call for the last half decade. 

• Q4, 2022 - “And the latest survey of US consumers from 451 Research indicates iPhone customer satisfaction of 98%” - Luca Maestri

• Q4, 2021 - “The latest survey of U.S. consumers from 451 Research indicates iPhone customer satisfaction of 98% for iPhone, and our active installed base of iPhones reached a new all-time high.” - Luca Maestri

• Q4, 2020 - “In fact, in the U.S., the last survey of consumers from 451 Research indicates iPhone customer satisfaction of 98% for iPhone 11, 11 Pro and 11 Pro Max combined.” - Luca Maestri

• Q4, 2019 - “And in the U.S., the latest survey of consumers from 451 Research indicates iPhone customer satisfaction of 99% for iPhone XR, XS and XS Max combined.” - Luca Maestri

• Q4, 2018 - “The latest survey of U.S. consumers from 451 Research indicates customer satisfaction of 98% for iPhone X, 8, and 8 Plus combined.” - Luca Maestri

You can see the full list of Apple’s earnings call satisfaction claims in the appendix. The trend goes back to at least 2011.Other research firms were hired to produce the satisfaction claims prior to 2016, finding similar results.

It’s worth noting that Apple cites third-party research firms they have hired in every single quantitative mention of customer satisfaction they make on earnings calls going back to at least 2013. Why do this? Apple is the largest company in the world and almost certainly has extremely precise in-house measures of customer satisfaction. The answer likely has to do with credibility and deniability. An extraordinary claim sounds more credible if it’s coming from an independent party, even if you pay to produce it. Furthermore, if investors or others start asking questions about something as eyebrow raising as near-universal satisfaction, it’s easier to deflect than if Apple had produced the statistic in house. 

We contacted Apple and 451 Research to try and figure out what was going on with the satisfaction numbers. They didn’t respond

We reached out to Apple and 451 Research to try and figure out how they were finding a 99% satisfaction rate for the iPhone 14.

According to Mailtrack, representatives from Apple opened our email 73 times since we sent it on June 20th, but have not responded to our request for the study methodology. 451 Research also read our message but did not respond. 

Unrepresentative samples

While neither Apple nor 451 Research responded to our questions about survey methods, one hint as to what might be going on here might be found in the footnotes to the 2018 press release from 451 Research we mentioned earlier. In it, 451 Research outlines the population they are drawing their samples from. 

A sample limited to accredited business and technology professionals who consider themselves early adopters and spend a lot on gadgets is, of course, not representative of iPhone purchasers generally. By not mentioning that the survey participants are highly non-representative, Apple’s claims about the survey’s results are misleading, and out of step with basic best practices when reporting survey results. Apple gives the impression that this survey is representative of the population by not indicating otherwise. 

Our Take 

The iPhone is an extraordinary phone. We recommend it to consumers using PerfectRec every day. But we’re 99% confident something doesn’t quite add up about Apple’s customer satisfaction claims.* 

Consider: 

• Independent sources report iPhone satisfaction levels that are significantly lower than what Apple claims. 

• The iPhone 14 did not get a great critical reception when it launched late last year, and earlier research by PerfectRec found user reviews for the model were significantly worse than previous editions of the handset. 

• Other companies with beloved consumer products tout much lower satisfaction rates when they talk about satisfaction at all.

• It would be extraordinarily hard to find near-universal satisfaction using normal survey research methods and a representative sample. 

It is likely that Apple continues to work with 451 Research not despite its unusual results but because of them. Apple wants to make extraordinary customer satisfaction claims and has found an outside research firm to produce them. 451 Research polls an unrepresentative panel of tech professionals, early adopters and gadget enthusiasts about their iPhone satisfaction, which Apple presents as reflecting the opinions of iPhone users as a whole. It doesn’t. The iPhone is perhaps the most successful consumer product in the world. Its market long ago grew beyond early adopters and gadget enthusiasts. 

It’s worth noting that Apple almost certainly has highly accurate internal measures of customer satisfaction. But instead of sharing those, the company instead highlights and likely misrepresent the findings of a third-party research firm they hired. 

“We’re not calling on the SEC to investigate these customer satisfaction claims, since they have more important cases to go after,” Golden said.  “But extraordinary claims require extraordinary evidence, and we believe Apple should review the survey results they trumpet on every earnings call. If they truly think they are valid, they should publish their methodology, and if not, they should issue retractions.” 

*In this case, it’s a figure of speech.

Appendix: Apple’s last 12 years of iPhone satisfaction claims on earnings calls

Q4, 2023 - "We continue to see extremely high levels of customer satisfaction which 451 Research recently measured at 98% in the U.S." - Luca Maestri Q3, 2023 - "This is a testament to our extremely high levels of customer satisfaction, which 451 Research recently measured at 98% for the iPhone 14 family in the U.S." - Luca Maestri Q2, 2023 - “We are very pleased by the results of the latest survey of US Consumers from 451 Research, which measured customer satisfaction at 99% for the iPhone 14 family.” - Luca Maestri

Q1, 2023 - “Our customers continue to love their experience with our products with the latest survey of U.S. consumers from 451 Research indicating customer satisfaction of 98% for the iPhone 14 family.” - Luca Maestri

Q4, 2022 - “And the latest survey of US consumers from 451 Research indicates iPhone customer satisfaction of 98%” - Luca Maestri

“And customer satisfaction for the iPhone remains very, very strong at 98%.” - Tim Cook

Q3, 2022 - “ In fact, the latest survey of U.S. consumers from 451 Research indicates iPhone customer satisfaction of 98%.- Luca Maestri

“Of course, the most important thing for us is to maintain an incredible customer satisfaction and loyalty from the customers. And we're really pleased that it's currently at 98% for the latest iPhones.” - Tim Cook

Q2, 2022 - “We set March quarter records in both developed and emerging markets and the latest survey of US consumers from 451 Research indicates iPhone customer satisfaction of 99% for the iPhone 13 family.” - Luca Maestri

Q1, 2022 - “We set all-time records in both developed and emerging markets, reached all-time high in the iPhone active installed base and the latest survey of U.S. consumers from 451 Research indicates iPhone customer satisfaction of 98%.” - Luca Maestri

“And that's all baked into some -- an enormous customer satisfaction rating of 98% and doing well throughout the geographies.” - Tim Cook

Q4, 2021 - “The latest survey of U.S. consumers from 451 Research indicates iPhone customer satisfaction of 98% for iPhone, and our active installed base of iPhones reached a new all-time high.” - Luca Maestri

Q3, 2021 - “In the U.S., the latest survey of consumers from 451 research indicates iPhone customer satisfaction of 97% for the iPhone 12 family.” - Luca Maestri

Q2, 2021 - “In the U.S. the latest survey of consumers from 451 Research indicates customer satisfaction of over 99% for the iPhone 12 family” - Luca Maestri

Q1, 2021 - “In fact, in the U.S., the latest survey of consumers from 451 Research indicates iPhone customer satisfaction of 98% for the iPhone 12 family.” - Luca Maestri

Q4, 2020 - “In fact, in the U.S., the last survey of consumers from 451 Research indicates iPhone customer satisfaction of 98% for iPhone 11, 11 Pro and 11 Pro Max combined.” - Luca Maestri

Q3, 2020 - “In fact in the US the latest survey of consumers from 451 Research indicates iPhone customer satisfaction of 98% for iPhone 11, 11 Pro and 11 Pro Max combined.” - Luca Maestri

Q2, 2020 - “In fact, in the U.S., the latest survey of consumers from 451 Research indicates iPhone customer satisfaction of 99% for iPhone 11, 11 Pro and 11 Pro Max combined.” - Luca Maestri

Q1, 2020 - “In the U.S., the latest survey of consumers from 451 Research indicates iPhone customer satisfaction of 98% for iPhone 11, 11 Pro and 11 Pro Max combined” - Luca Maestri

Q4, 2019 - “And in the U.S., the latest survey of consumers from 451 Research indicates iPhone customer satisfaction of 99% for iPhone XR, XS and XS Max combined.” - Luca Maestri

Q3, 2019 - “Our active installed base of iPhone continued to grow to a new all time high in each of our geographic segments and in the U.S., the latest survey of consumers from 451 Research, indicates iPhone customer satisfaction of 99% for iPhone 10R, iPhone 10S and 10S Max combined.” - Luca Maestri

Q2, 2019 - “The latest survey of U.S. consumers from 451 Research indicates customer satisfaction of 99% for iPhone XR, XS and XS Max combined.” - Luca Maestri

Q1, 2019 - “The latest survey of U.S. consumers from 451 Research indicates customer satisfaction of 99% for iPhone XR, XS and XS Max combined.” - Luca Maestri

Q4, 2018 - “The latest survey of U.S. consumers from 451 Research indicates customer satisfaction of 98% for iPhone X, 8, and 8 Plus combined.” - Luca Maestri

Q3, 2018 - “iPhone X was the most popular iPhone in the quarter once again, with a customer satisfaction score of 98% according to 451 Research.” - Tim Cook

“The latest survey of U.S. consumers from 451 Research indicates that across all iPhone models, customer satisfaction was 96% and combining iPhone 8, 8 Plus and iPhone X, it was even higher at 98%” - Luca Maestri

Q2, 2018 - “The latest survey of U.S. consumers from 451 Research indicates that across all iPhone models, the customer satisfaction rating was 95%, and combining 8, 8 Plus, and iPhone X, customer satisfaction was even higher, at 99%.” - Luca Maestri

Q1, 2018 - “The latest data from 451 Research indicates US customer satisfaction ratings of 96% or higher across iPhone models. In fact, combining iPhone 8, iPhone 8 Plus, and iPhone X, consumers reported an amazing 99% satisfaction rating.” - Luca Maestri

Q4, 2017 - “Customer interest and satisfaction with iPhone are very strong with both consumers and business users. In the U.S., the latest data from 451 Research on consumers indicates a customer satisfaction rating of 97% or higher across all iPhone models.” - Luca Maestri

Q3, 2017 - “Customer interest and satisfaction with iPhone are very strong with both consumers and business uses. In the U.S., the latest data from 451 Research on consumers indicates a 95% customer satisfaction rating for iPhone 7 and 99% for iPhone 7 Plus.” - Luca Maestri

Q2, 2017 - “Customer interest and satisfaction with iPhone are very strong, not only with consumers but also with business users. In the U.S., the latest data from 451 Research on consumers indicates a 96% customer satisfaction rating among iPhone 7 owners and 98% for iPhone 7 Plus. Among corporate smartphone buyers, iPhone customer satisfaction was 95%.” - Luca Maestri

Q1, 2017 - “Customer interest and satisfaction with iPhone are exceptional, not only with consumers, but also with business users. In the U.S., for instance, the latest data from 451 Research on consumers indicates a 97% customer satisfaction rating among all iPhone owners and a 99% satisfaction rating for owners of iPhone 7 Plus. Among corporate smartphone buyers, the iPhone customer satisfaction rating was 94%.” - Luca Maestri 

Q4, 2016 - “Customer interest and satisfaction with iPhone remains extremely strong. In the U.S. for instance, the latest survey fielded by 451 Research found that among consumers planning to purchase a smartphone within 90 days, 65% plan to purchase iPhone, with the current iPhone owners reporting a 97% customer satisfaction rating. Among corporate smartphone buyers, the latest survey measured a 95% iPhone customer satisfaction rating and found that of those planning to purchase smartphones in the December quarter, 79% planned to purchase iPhone.” - Luca Maestri

Q3, 2016 - No mention of iPhone satisfaction. 

Q2, 2016 - No mention of iPhone satisfaction. 

Q1, 2016 - “First and foremost, our customer satisfaction and retention rates are second to none and provide us with a long lasting foundation. For example, recent consumer surveys by 451 Research, formerly known as ChangeWave, measured an incredible 99% customer satisfaction rate for iPhone 6S and 6S Plus and an equally impressive 97% rate for the iPad Air 2.” - Tim Cook

Q4, 2015 - No mention of iPhone satisfaction. 

Q3, 2015 - “Most importantly, iPhone customer metrics are tremendous. ChangeWave's most recent survey of US customers found that iPhone has the highest customer satisfaction rate of any smartphone brand by a wide margin, and that among iPhone owners planning to purchase a new phone, 86% plan to purchase another iPhone.” - Tim Cook 

Q2, 2015 - No mention of iPhone satisfaction. 

Q1, 2015 - “A December ChangeWave survey measured a 97% customer satisfaction rating for iPhone.” - Luca Maestri

Q4, 2014 - No mention of iPhone satisfaction, but a claim about 100% iPad satisfaction. “In an August survey by ChangeWave iPad Mini with retina display earned an incredible 100% satisfaction rate, and among consumers planning to purchase a tablet within 90 days, the survey indicated a 55% plan to buy an iPad.” - Luca Maestri

Q3, 2014 - “Also based on the latest survey by ChangeWave, iPhone earned a 97% customer satisfaction rate and among responders planning to purchase a smartphone within 90 days, 50% planned to purchase an iPhone, up from 42% in the March quarter and 44% a year ago.” - Luca Maestri

Q2, 2014 - No mention of iPhone satisfaction. 

Q1, 2014 - “A December survey of U.S. customers by ChangeWave indicated a 96% customer satisfaction rate for iPhone.” - Peter Oppenheimer

Q4, 2013 - “Based on the most recently published research, ChangeWave measured a 96% customer satisfaction rate among iPhone users and Kantar measured a 92% customer loyalty rate, significantly higher than the competition.” - Peter Oppenheimer

Q3, 2013 - “The most recently published study by Kantar measured at 93% loyalty rate among iPhone owners significantly higher than our competitors. And iPhone continues to lead in terms of customer experience. Not only has iPhone earned the top spot in customer satisfaction from J.D. Power and Associates, an unprecedented nine consecutive times it has also received the top customer satisfaction ranking in a number of surveys including the recent Quality Insights survey of smartphone customers in South Korea.” - Peter Oppenheimer

Q2, 2013 - “A recent study by Kantar measured 95% loyalty rate among iPhone owners, substantially higher than the competition, and iPhone remains top in customer experience. Last month, we were very pleased to receive the number one ranking in smartphone customer satisfaction from J.D. Power and Associates for the ninth consecutive time.” - Peter Oppenheimer

Q1, 2013 - No mention of satisfaction. 

Q4, 2012 - No quantitative mention of satisfaction, no mention of iPhone satisfaction. 

Q3, 2012 - No quantitative mention of satisfaction, no mention of iPhone satisfaction. 

Q2, 2012 - No mention of satisfaction. 

Q1, 2012 - No mention of satisfaction.

Q4, 2011 - “For the sixth consecutive time, iPhone ranked highest in consumer reports, wireless consumer smartphone customer satisfaction survey, achieving the highest ratings for overall satisfaction as well as for performance, ease of operations, features and design.” - Peter Oppenheimer