R.I.P for the CSI?

Is it time to write the obituary for the venerable customer service index survey?

Growing numbers of service managers, consultants and other industry observers think it's time to pull the plug on the dealership evaluation tool, which has been used to measure customers' satisfaction with dealerships' service since the 1980s.

Some believe it's an antiquated tool that no longer truly measures how well dealerships serve their customers. Others believe it strongly contributes to mental health issues for service advisers, whose compensation — and sometimes even their employment — is tied to CSI scores. That, in turn, contributes to the exodus of fixed ops employees from dealerships, they say.

"Turnover is a big problem for service departments, sometimes reaching 50 percent in the last year or two," says Eliza Johnson, a principal at Ducker Carlisle, a strategic consulting and advisory firm. "It seems like turnover is an unintended consequence of the CSI issue. Service advisers feel they're measured unfairly or can't reasonably estimate their compensation without knowing how the CSI scores will look."

Furthermore, many things can go wrong during a customer's visit that aren't the fault of service advisers. Yet they still take the hit for those that contributed to an unsatisfactory experience, Johnson notes.

Ducker Carlisle does annual service adviser, service manager and technician surveys on behalf of auto manufacturers to take the pulse of the industry. In the last survey of service managers conducted in February, about 10 percent of the 10,000 respondents included both solicited and unsolicited criticism of CSI scores, Johnson told Fixed Ops Journal.

"It's definitely not just a few people here and there making critical comments," she says. "It's absolutely a legit phenomenon. We've been hearing this for years."

Aaron Bratcher, service manager at Chevrolet of Columbus in Indiana, part of P4 Automotive Group, says the CSI contributes to advisers' poor mental health.

"There's absolutely a link there," he says. "I wrote service for eight years and had low scores from time to time, even though I did my job well.

"It caused a lot of anxiety," Bratcher says. "The dealership where I worked terminated you if you had bad CSI scores for three consecutive months — no discussion."

He says a typical adviser has a lot of worries outside of work — COVID-19, the economy, inflation, for example. Then that adviser goes to work and worries about getting "destroyed" by a customer on a survey because a technician puts 37 pounds of air in the tires instead of the suggested 33.

"What does that do to people?" Bratcher asks.

Bratcher says he knows service writers who mentally and emotionally have "given up."

"It basically takes all the wind out of their sails," he says. "If you use the CSI as a bulldozer, you're going to destroy a person's foundation of confidence. Some people can shake it all off, but others let it pile up."

Bratcher says he knows three service advisers who have left the industry because of the stress tied to CSI scores. "And two of them were very good service writers," he says.

Joe Chambers, fixed ops director at International Autos in Tinley Park, a Chicago suburb, agrees with Bratcher. International Autos owns seven dealerships in Illinois, Minnesota and Wisconsin.

"There's definitely a link between CSI scores and stress," says Chambers, who also co-hosts a show via Zoom called "Service Drive Live" on Sundays with colleague Erick Jauregui. "A lot of times I see guys get notifications of survey results and it legitimately ruins their day.

"Clearly, it's not the only contributing factor to stress," he adds. "But if they get a text while watching TV at home or playing with their kids, it rocks them to their core.

"They know if they don't make their numbers, they don't get the bonuses."

CSI elimination was a topic on "Service Drive Live" in June. The overall response from participants was that the CSI has outlived its usefulness and generally doesn't paint an accurate picture of how dealerships treat customers.

"It's too long and drawn out," Chambers says. "They lose their diagnostic ability. … We'd be better served with fewer questions."

Moreover, customers often weaponize CSI scores and use them as leverage for getting service discounts, he adds.

"Customers work the system," he says. "They know what CSI scores mean to a dealership, so they walk in and say, 'I have a survey in hand — what are you going to do for me?' "

Additionally, some dealers game the system by coaching customers about providing good CSI scores. That creates good scores that don't represent the dealership's actual level of service.

"That's reflected when we see that customer retention is much lower than customer satisfaction, often by quite a bit," Ducker Carlisle's Johnson points out. "It's totally counterintuitive … and OEMs often know there's a disconnect there — that the CSI scores don't give them the full picture."

In other instances, service advisers become the victims of customers with service expectations that can never be met. Bratcher recalls he once received a score of 75, which prompted a phone call to the customer to identify the reason for the score.

"The customer said that everything was perfect, but that he never gives out perfect scores because everyone has room for improvement," he says. "You get a comment like that and you kind of want to explode."

So if the CSI is antiquated and broken, what might be used instead?

Bratcher says a mechanism for obtaining customer feedback remains necessary.

"I'm afraid that if you take away the surveys, you also take away that voice of the customers," he says.

As such, Bratcher says he would favor a system that instead measures customer retention, which is a more accurate measure of how dealerships serve customers.

Chambers agrees. "If you keep getting repeat business over and over and over, you're doing something right," he says. "And that's very measurable."

Some industry observers endorse the Net Promoter Score, which centers on one primary question for consumers: On a scale of 1 to 10, how likely are they to recommend Business X to a family member or friend?

Asking just one question greatly increases response rates and the results are easily understood: a score of 9 or 10 means the customers are promoters; 7 to 8 makes them passive; and grades of 0 through 6 makes them detractors. Taking the percentage of promoters and subtracting the percentage of detractors produces a Net Promoter Score.

"I'm not a big fan," Chambers says. "I understand that it's short and easy to understand. But you shouldn't be penalized for receiving an 8 out of 10, which is a really good score — but not according to the NPS. It's just another number that can be manipulated."

Johnson agrees, noting that viable alternatives to gauging customer satisfaction exist. For example, many industries use a sales metric called share-of-wallet, which is an average of how much a consumer spends on a particular company's products or services compared with how much they spend on competitors' products or services, she says.

"It's a better metric because it's behavior-based, not sentiment-based," Johnson says.

She also suggests "mystery shopping" in which anonymous third-party shoppers pose as customers to determine dealerships' level of service. Reviews on social media platforms such as Twitter, Facebook, Yelp, Google and automaker websites also could be used.

"We don't advocate elimination of CSI scores," she says. "But we do think there are more viable and more accurate ways to measure the customer experience.

"The CSI also could be improved by altering the questions and removing any ties to pay plans, which creates bad behavior and frustrations," she says. "Even if OEMs start using other methods, I think the CSI survey will remain around for some time. It's not going to go away tomorrow."