Testing as a form of measurement

Joe Murphy – Compliance & Ethics Professional – August 2017

Practitioners today know that they need to measure and assess their compliance programs. Often, though, they seem to go inexorably to surveys as the tool for doing this. I like surveys and have used them, but they are just one tool. They have value but also serious weaknesses. For example, surveys typically depend on people telling the truth and reporting accurately. I am too skeptical to depend on this. Surveys can be superficial and mask issues that were not effectively covered in the survey questions. They can also be drafted and administered to manipulate results. Thus I believe measurement of compliance programs cannot be limited to surveys.

Here is one tool I do not generally see discussed in our field: testing, as in taking steps to see if systems work the way they are supposed to work. So, for example, one testing technique that is legally recognized in banking in the U.S. is checking for indications of lending discrimination using matched pairs, i.e., two similarly situated couples applying for loans, but one being a majority couple and one being in a protected class. (There is a special legal privilege protecting these tests.)

Of course, one should not go too far in this. Being sneaky and taking a “gotcha” approach to employees and agents can certainly breed ill will if done the wrong way. But there are many ways to test what is happening in a company that are reasonable and acceptable.

I knew of one company that wanted to test out its global helpline system that was provided by a vendor. The company had its internal audit organization place test calls from 10 different locations around the world. It then waited to see what happened. To the company’s consternation, half the calls never made it through the system to the Compliance Office. Thereafter the company changed providers for its helpline system.

Another example would be the use of “mystery shoppers.” While these anonymous shoppers might usually serve to measure levels of retail customer service, they could also test whether products on sale were properly labeled and look for other consumer protection issues.

In my practice as a lawyer, one field I cover is antitrust. So when it comes to simply accepting someone’s say-so that everything is OK, I am definitely an anti-trust lawyer. As compliance and ethics professionals, we should always be respectful of others, but we cannot simply accept things on faith and trust. We also need to check things out. When you are looking at how to determine whether your program is working and effective, consider ways that you can test things out to be sure they actually work and that people are doing what they say they are doing.

This article first appeared in Compliance & Ethics Professional.

Leave a Reply