Tuesday, November 4, 2008

Walmart: Where is a statistician when you need one?

Extracted from the shareholder letter of the last Walmart annual report:

Improving Customer Experience
We continue to invest in improving the customer experience in our stores - faster checkouts, friendlier service and cleaner premises. Our customers are taking  note.  We survey almost two million customers every quarter, and they 
are validating the improvements.

I couldn't fathom this.  Surveying nearly 8 million people per year means that Walmart has a better handle on what the population wants that just about anyone.  

But somewhere I learnt the central limit theorem and the law of large numbers suggests that you don't obviously need to survey that many people unless you are dealing with very fine nuances...  

So please dear readers - what is Walmart doing here?  And why does it give them a competitive advantage?

John Hempton

PS.  If Walmart were a political party surveying that many people I would think were push polling.  It is not that - so what is it?


Dan Sweet said...

i agree n=30 is sufficient to reasonably assess one value at one point in time for one target group of people. (CLT)

i'd assume much of Walmart's claimed 2 million consumers comes from those bottom of the receipt call this 1-800 number to get a chance to win a $250 gift card type of things. with an automated system to gather the data it prob makes no difference to Walmart whether they gather 250 responses or 250,000 responses.

my guess is that Walmart is also performing complex analyses to segment their customers into many separate groups. its not uncommon for these types of studies to have sample sizes more like n=5000. this allows the researcher to break consumers out by region and segment. in Walmart's case I'd be shocked if they don't have the capability to break this out down to the store level. hence the reason they need > n=30. in addition to that, I'm sure that collecting this data on an ongoing basis likely allows Walmart to evaluate emerging trends by tracking changes in responses as time-series data as well.

Anonymous said...

You want to measure and manage at the store level - in particular wrt store metrics like "faster checkouts, friendlier service and cleaner premises". So you have to survey at the store level. For which the total of 2M / qtr seems to be in the right area.

Roger Michael said...

Well, could it be that they want to monitor the satisfaction for every individual outlet, not only the company as a whole. Then they need a statistically significant sample for every individual outlet?

Anonymous said...

The survey likely consists of the cashier gal rhetorically asking, "didja find what you were lookin' fer?"

Anonymous said...

I don't know Walmart but such numbers can be achieved easily if you define surveying very loosely. Sure they will perform targeted surveys, as does every other chain, but maybe they count some customer complaint/satisfaction form also as 'survey'. Does this give competitive advantage? No.
What really does give competitive advantage is the customer loyalty cards that record your purchases against you, thus providing the chain with a consumer profile of you as a family. This IS a fine science and does give a chain that knows how to handle large datasets a competitive advantage. And the best bit of it is that the consumers don't know their supermarket knows just about everything about them. Much more than their bank. And it will use this knowledge to make people spend more!


Blank Xavier said...

What does the phrase actually mean?

What's a survey? is it a consistent thing over all two million samples or are Walmart counting *any* customer query as a survey?

Did they get two million completed surveys (whatever they are) or did they merely *ask* two million customers? if they merely asked, what was the return rate of fully completed surveys?

John Hempton said...

Ok - so the general consensus in my email is that questions are asked but only negative replies are recorded.

Some of it is marketing too - people like to know that their views are being taken seriously.


Does anybody know the process Walmart goes through with negative responses? How does this integrate to their business.

My experience is that Walmart is very deliberate as per systems - so there is something going on here...


Ryan said...

I agree with dan's point that they are almost certainly going to be interested in not only corporate-wide, but also regional and even store-level data.

So, let's see how this works out:

According to their wikipedia page, Wal-Mart has 971 Discount Stores, 2447 Supercenters, 132 Neighborhood Markets, and 591 Sam's Clubs in the United States as well as 2980 international stores. That's a total of 7121 total stores.

As a 24/7 store pretty much everywhere, if they survey 3 people per store per day that's 1.95m people per quarter.

Prospectus said...

My take: Every time I go though the checkout at WMT, on the ATM/debit card pin pad there is a touch button with one of two questions: "Was your cashier friendly today?", or "Was your store clean today?" Yes/No answer buttons and that's it. From the sound of the language from their shareholder letter, that dumb little poll could be what they are touting here as "customer surveying" of 8 million people.

Anecdotally, they also have a video at checkout that shows (in my area) that "WMT has saved Texas families XX dollars this year!" The number ends up in the 26 billions and counting each year. I say "bollocks" on that one.

Anonymous said...

I second "prospectus" above.

ngogerty said...

this may help understand the sheer IT used in wal-mart. http://www.youtube.com/watch?v=b5K1yrICMII

Anonymous said...

They are very good at playing regional differences in pricing.

I shop at several walmarts over a pretty large geographical area in upstate NY. There are noticable differences in prices on a lot of different items. They are simply trying to surmise how much pricing power they have.

Closer to urban areas, prices are much more competitive. Where they have moved into 'rural' markets, their prices are higher, in my estimation, across the board, but not by enough to justify driving to another store.

They also have a policy of not honoring online prices in the stores. If you talk with a manager you can generally get them to match, but I have to wonder if those situations are where they are getting data from.

General disclaimer

The content contained in this blog represents the opinions of Mr. Hempton. You should assume Mr. Hempton and his affiliates have positions in the securities discussed in this blog, and such beneficial ownership can create a conflict of interest regarding the objectivity of this blog. Statements in the blog are not guarantees of future performance and are subject to certain risks, uncertainties and other factors. Certain information in this blog concerning economic trends and performance is based on or derived from information provided by third-party sources. Mr. Hempton does not guarantee the accuracy of such information and has not independently verified the accuracy or completeness of such information or the assumptions on which such information is based. Such information may change after it is posted and Mr. Hempton is not obligated to, and may not, update it. The commentary in this blog in no way constitutes a solicitation of business, an offer of a security or a solicitation to purchase a security, or investment advice. In fact, it should not be relied upon in making investment decisions, ever. It is intended solely for the entertainment of the reader, and the author. In particular this blog is not directed for investment purposes at US Persons.