“Compliance functions are still spending a disproportionate amount of time collecting data, versus time spent adding strategic value to the business through analyzing and trending the data collected.”
—In Focus: 2015 Compliance Trends Survey from Compliance Week and Deloitte
I’m continuously struck by the compliance industry’s challenges around program measurement and reporting. Most recently, it was the annual Compliance Trends Survey from Compliance Week and Deloitte that delivered the bad news: 35% of CCOs cite data reporting and analytics as one of the top three most challenging aspects of their job, while nearly a third of CCOs aren’t measuring the effectiveness of their program at all.
A few factors certainly contribute to that challenge, including:
- A lack of confidence in IT systems’ abilities to fulfill the CCO’s reporting responsibilities—which nearly 60% of surveyed CCOs cited
- A lack of technology to help meet reporting needs—OCEG’s research pegs 53% of organizations using spreadsheets, documents and email as their primary GRC tools
- A lack of confidence in metrics—42% of CCOs say they’re only “somewhat confident” or “not confident” in the metrics they use to give a true sense of effectiveness
Compliance teams can’t waste time with data that won’t ultimately help them make better decisions.
This presents obvious problems for a program’s defensibility: If you don’t know if or how well your compliance initiatives are working, you’ll be hard pressed to defend or improve your program. But the persistency of this trend is also disconcerting because it limits how far or quickly the compliance function overall can advance if it can’t provide the breadth or depth of business unit analysis as its peers (current or aspirational) in the executive suite. Every other function—from finance to sales and operations to HR—continually monitors, reports on and is held accountable for in-depth analysis of their performance. If compliance wants to consistently and meaningfully contribute to a company’s strategic conversations, while being able to demonstrate and defend its effectiveness to leadership and regulators, then comprehensive and thorough reporting will need to become a matter of course.
And while it’s critical to have data about your compliance program in order to analyze effectiveness and make ongoing improvements, there’s still a tendency to cling to “vanity” metrics—hollow metrics that don’t actually help you do much of either. With limited resources—and facing increasingly high stakes, expectations and scrutiny levels—compliance teams can’t and shouldn’t waste their time compiling and analyzing data that won’t ultimately help them make better compliance decisions.
Here are some overly broad and ambiguous metrics that many compliance teams still track—and some suggestions for how to make adjustments or add context to improve their value and utility.
Current metric: Incident reports, investigations and resolutions
Misconduct reporting and investigation trends are continuously cited by CCOs as one of their go-to metrics for program and risk management effectiveness. Without some additional context, though, they may be missing opportunities to address the cultural, behavioral or operational factors that enable or exacerbate the misconduct from occurring in the first place.
- Issue trends for key risk areas
- Issue trends by location, business unit, organizational title, employee demographics (tenure, salary, etc.)
- Trends between type of misconduct and the factors that contributed to the misconduct—including rationalization, lack of awareness, pressure, etc.
- Number and type of sanctions applied by incident type, location, business unit, organizational title, employee demographics, etc.
- Incident drivers to differentiate misconduct that was intentional, rationalized, unwittingly committed, driven by pressure/compensation and any correlations between drivers and risk areas, locations, business units, organizational titles, etc.
- Relationships between disclosures and incident trends
- Impact of training, policies, communication and incentives on number and types of incidents, contributing factors and resolutions reached
- Demographic or behavioral trends among involved parties (reporters, witnesses or subjects)
Current metric: Training initiatives and completion rates
As one of the most fundamental components of even the most nascent compliance programs, CCOs often report on the number and topics of the courses they’re distributing, completion rates and the results of any related comprehension tests or knowledge assessments.
- Training topics and trends for key risk areas
- Training trends (good and bad) by region, business unit, organizational title, etc.
- Number of disclosures submitted after conflicts of interest or GT&E training
- Impact of training rollouts and results on hotline trends
- Impact of incentives and communication initiatives on training engagement and understanding
Current metric: Policy initiatives and attestation rates
Distributing policies is nothing new for compliance teams, and as a result this has become a fairly mundane “check the box” activity.
- Policy distribution and attestation trends for key risk areas
- Attestation trends (good and bad) by region, business unit, organizational title, etc.
- Number of policy exceptions and disclosures submitted alongside rollouts of related policies like conflicts of interest, gifts and entertainment, etc.
- Relationships between policy attestations and training completion/certifications
- Impact of policy rollouts on misconduct reporting rates
- Impact of incentives and communication initiatives on policy engagement and understanding
Current metric: Risk assessment
Many organizations use the results of their risk assessment to prioritize their compliance program efforts—and with good reason, as regulatory guidance continually cites a risk-based approach to compliance as a hallmark of an effective program. But how do compliance teams move beyond using their risk registers as a punch list, to leveraging it for critical context in reporting their effectiveness?
- Risk disposition by location, business unit, organizational title, etc.
- Risk exposure increases due to incident, disclosure, training, culture assessment or policy trends
- Correlation or discrepancies—and analysis of reasons for the relationships—between risk assessment results and bellwethers of a company’s cultural environment like culture assessments, incident drivers and more
Yes, these metrics are driven in large part by the expectations set out by the Federal Sentencing Guidelines—but I’d argue that in their most basic form they follow the Guidelines in letter, not spirit. To truly optimize effectiveness and facilitate continuous improvement, context and deeper analytics of the data are needed. Only then can compliance move from a highly reactive function to one that’s cohesive, predictive, proactive and preventative in nature. After all, isn’t that the point?