For most businesses, there is a measurable tie between business performance and customer satisfaction and loyalty (see "Morgan-Rego" for one compelling study). It is for this reason that the most well-known and successful companies routinely survey their customer base to ensure that their improvement efforts have the desired effect.

To "survey" implies to gather an "overview" of the customer population, and that requires special techniques. Usually, it is difficult to gather statistically reliable data via passive methods such as web or paper. It is for this reason that telephonic interviewing remains so popular among progressive companies.

We are advocates and practitioners of customer satisfaction research conducted primarily through computer assisted telephone interviewing. Our experience in applying other methods has made clear the advantages of telephone surveying:

• Telephone interviewing is timely. Responses from dissatisfied customers can be communicated swiftly for rapid service recovery. Employees can be "coached" on making improvements soon after a transaction.
  Telephone interviewing captures "the voice" of the customer.
• Interviewers can probe more deeply to achieve responses to open-ended questions (verbatims) for actionable content.
• Computer assisted telephone interviewing is highly accurate and cost-effective. The method eliminates many indirect costs associated with the paper survey process such as postage, data entry, paper costs and processing. Our costs are based on the volume of completed surveys.
• Telephone interviewing allows for random sampling techniques. This ensures reaching pre-arranged percentages of statistical validity. Managers and sales/service employees will attain a higher level of confidence that customer satisfaction scores are valid and will be more prone to affect positive change.
• Telephone interviewing is a positive experience for customers. It reflects a timely, proactive concern for the customer's experience with your company.

There are some significant differences between survey methodologies in terms of the utility of the information that can be recovered and the uses to which it can be put.

Paper/Mail Surveys:

Mailed out paper surveys suffer from the largest array of infirmities of any survey methods. They have the highest ratio of non-value costs of any survey methodology. The costs of forms, scanning, tabulating, postage in and out are a very significant portion of the total cost of the survey, but they add nothing to the quality of the results or the utility of the answers.

Paper surveys are also by their very nature slow. List processing and bulk or even first-class mail consume a great deal of time between the customer event and the survey presentation. Accuracy is lost. A great amount of time is also lost on the return and processing side. We can testify from experience, having inherited a large home delivery survey when it was conducted via a paper survey, that there is no way to make the survey results contemporary with customer events.

Mail surveys also endure another difficulty related to the statistical significance of the results: failure to get all the data needed to make the survey singularly actionable. If there are, for example, 100 key cells to measure (whether sales units or regions or districts), there is no quota control on a paper survey. The answers come through a self-selection process of the respondents. Users of the surveys are often left to make decision about the whole frame of the study without having data from the whole frame. This application of data is dangerous when you are trying to estimate customer alignment or understand attitudes.

Self-selection and non-response affect paper surveys in another way. The only people who respond are those that select themselves for some reason which is usually unknown to the surveyor. Often, surveyors think that a high response rate (30% to 40%) solves the problem created by self-selection bias, but it does not. Without further work, there is no possible way to know why the respondents decided to answer and why the non respondents decided not to answer. Not knowing anything about over half of the frame is a risky way to estimate customers' attitudes. With mail survey methodology some types of customers will be over-represented and some will be underrepresented. There is no way to tell between the two groups.

Computer Assisted Telephone Interviewing(CATI):

CATI is the methodology which permits quota control and statistical accuracy with results that are representative of the most recent activities of the measured units. CATI permits probing for verbatims on any question. CATI eliminates non-response bias and self-selection bias. The system picks the individuals to be surveyed based on their membership in the cells to be measured. This produces the least amount of surveys needed for the maximum decision making reliability.

CATI permits matching of customer event and reporting periods which allows field managers to use the results to drive their businesses in a tactical sense while headquarters managers can view trends and more strategic issues. With CATI, the surveyor can get to the customer very quickly to get their most accurate views and comments. The home delivery survey, for example, interviews customers no more than 48 hours after their delivery is accomplished. It also provides feedback to the unit manager in less than 72 hours. This is particularly important if the unit manager is using customer comments to manage associates.

Statistical precision and speed are of considerable importance where surveys are being used for measuring units or people. Hiring, firing and compensation decisions based on imperfect survey results can create a dangerous situation for any organization.

Moving from a Paper to a Telephone Survey

Summary

A large, national service organization moved from a paper customer satisfaction survey to a statistically sampled phone survey resulting in a total savings of more than $500,000 in three years (nearly 50% of its original survey budget) and improvement in response time from as much as 75 days of lag time to results within 48-72 hours.

Background Information

This company operates 60 locations across the United States (including Hawaii). Since the early 1980's, it has conducted a monthly survey of its customers to ascertain satisfaction levels. The results of the survey were used to pay bonuses to 75 third party service providers used by the organization.

In the paper format, a self-mailer survey was sent to approximately 1/2 of all customers for the month. Customers to be surveyed were selected randomly. The survey for each month was sent out in 2 waves. The first wave was mailed by the 3rd working day of the month and included customers from Day 16 through Day 30/31 of the previous month; the second wave was mailed by the 17th working day of the month and included customers from Day 1 through Day 15 of the current month. At the time of the conversion to a phone survey, approximately 250,000 surveys were mailed each month. Because of the expense associated with mailing these surveys, they were sent bulk mail.

Returned surveys were picked up from the post office weekly. Returns were sent to a post office box using a BRMS qualified first class mail permit. The results were tabulated and data entry accuracy verified using a double entry system. Customer comments were batched and forwarded to individual units on a weekly basis. No effort to capture these comments in a systematic way was made. The survey consistently achieved a 25 to 30% response rate overall; however, response rates among units ranged from less than 10% to nearly 50%.

There were several problems with this survey from the customer's perspective. The biggest problem for the customer was the lack of timeliness in the response. Often managers found customer complaints about individuals who had been terminated weeks and sometimes months ago. In addition, the ability to talk with individuals about service failures and have them remember a specific event was very difficult given the amount of time between the actual service date and the date the customer comments were received.

On a statistical basis, there were even more problems. The self-selection bias inherent in any volunteer survey results in fairly accurate knowledge of those that have responded, but no knowledge for the group of customers that did not respond. In addition, there were significant sampling deficiencies for some units.

Recommendation

In 1990, Marketing Solutions, Inc. (founding company of TeleSight, Inc.) was selected to conduct the paper survey for this client. The administrative portions of the survey (non-value added) for elements such as printing, postage and physical handling of the surveys continued to grow, resulting in price increases each year. When postal rates increased for the second time in the course of conducting this survey, MSI became determined to find some offsetting cost savings for the client. At the same time, we addressed other shortcomings of the survey, i.e., timeliness and sampling levels.

We knew that a telephone survey could solve the timeliness and sampling level problems. We were also confident that it would be cost-effective. We were pleased to discover that the phone methodology would not just hold down escalating survey costs, but actually save significant, hard dollars from day one.

In 1993 the survey was converted to a phone survey. The steps taken to accomplish this change included a slight modification of the survey design, communication with the field about the change, education of headquarters and field management about the sampling methods to be employed and establishment of electronic data exchange to communicate results.

Savings

The first year savings amounted to more than $300,000. They could have been even higher if strict sampling quotas had been used. In the first year, the sample sizes used for each unit were substantially higher than the levels calculated using statistical models developed for us by Northwestern University's Statistics Department. The reason for this "over sampling" was to give unit managers an "apples to apples" comparison of results from the paper survey to the phone survey. Our goal was to conduct a number of surveys that would approximate or exceed the number of paper surveys returned.

Within 9 months, we were able to reduce the cost per completed survey by 5% due to efficiencies achieved in call center processes to handle this survey. This 5% amounted to nearly $50,000.

In the following year, we introduced the sampling formulas to the unit managers, demonstrating to them that fewer surveys would generate the same results based on confidence levels of 95% +/- 5% (In fact, measurement of confidence levels based on actual survey results achieves 95% +/- 1% routinely.) This change led to another $150,000 savings.

In year three of the program, we worked with the client to analyze how the data gathered from the survey was being used. In the course of this analysis, we discovered that three of the questions were measuring aspects of service that were captured much more accurately with internal systems. Accordingly, those questions were dropped and one question added. The final, shorter survey resulted in reducing the cost per completed survey by nearly 20%.

Improved Measurements

In addition to being cost effective, the phone survey also provided extremely timely information to the unit managers. The speed of electronic communications allowed for surveys to be conducted within 48 hours of the service being performed. Results by unit are posted each night, allowing a unit manager to see his/her unit's progress through the month.

Even better, any negative responses are probed by the interviewers. These customer comments are transmitted to the appropriate unit manager each night. Positive comments that are volunteered by the customer are also captured and sent to the managers nightly. These verbatims provide unit managers with daily coaching tools as well as opportunities to give individuals public pats on the back for positive customer feedback.

Regional sessions with managers have also trained them in categorizing comments in order to get a better idea of the "big picture" the comments reflect. Managers categorize customer comments by individual, by day of service and by type of service. A look at how customer comments cluster over a 4 to 12 week period provides a good idea of what areas represent opportunities for improvement by the unit as a whole or for an individual.

Working with the client's Information Systems people, we were able to integrate the survey data with data collected in other parts of the client's organization to provide a more complete service scorecard that can be viewed electronically daily. While we emphasize that the survey sample is drawn for the entire month (and, therefore, can be skewed particularly at the beginning of a month when there are few surveys completed), managers find it useful to observe the pattern of service scores daily. They can quickly tell whether their scores are moving in the right direction. In addition, the daily verbatims tell them what is going wrong when there is a service failure. Each comment is accompanied with the customer's name, phone number, type of service and date of service allowing managers the ability to quickly move toward service recovery when appropriate.

Conclusion

A telephone survey can save significant dollars, allowing companies to eliminate non-value survey costs (postage, printing, clerical tasks) and obtain more actionable and more timely information. For management and employees alike, the immediacy of a telephone survey makes it more credible and therefore something that has greater connection to performance than "stale" paper survey results. For customers, a telephone survey provides an opportunity for them to provide feedback while the service event is fresh in their minds and allows for service recovery when appropriate

The availability of an inexpensive worldwide communications network has obvious implications for companies who gather and dispense data. For many companies, especially those with remote agents and business partners, the Web is an indispensable tool for making available the results of customer data gathering efforts.

TeleSight routinely develops custom web-based reporting applications for our customers so that results are more usable and timely. The availability of this on-line gauge lends excitement to the program.

We also develop web surveys. While in many cases there are issues with web surveying, just as with paper, in the area of statistical reliability, there are often programs where the target population is finite and addressable and can be "primed" ahead of time so that the results are usable. Examples include employee surveys, surveys of business partners and agents, and customer surveys wherein the customers are asked ahead of time, and sometimes incented, to complete the survey.