Four Compliance Metrics Risk That Need to Die

Empty compliance metrics that aren’t worth tracking or reporting

“Compliance functions are still spending a disproportionate amount of time collecting data, versus time spent adding strategic value to the business through analyzing and trending the data collected.”
In Focus: 2015 Compliance Trends Survey from Compliance Week and Deloitte

I’m continuously struck by the compliance industry’s challenges around program measurement and reporting. Most recently, it was the annual Compliance Trends Survey from Compliance Week and Deloitte that delivered the bad news: 35% of CCOs cite data reporting and analytics as one of the top three most challenging aspects of their job, while nearly a third of CCOs aren’t measuring the effectiveness of their program through compliance risk metrics at all.

A few factors certainly contribute to that challenge, including:

  • A lack of confidence in IT systems’ abilities to fulfill the CCO’s reporting responsibilities—which nearly 60% of surveyed CCOs cited
  • A lack of technology to help meet reporting needs—OCEG’s research pegs 53% of organizations using spreadsheets, documents and email as their primary GRC tools
  • A lack of confidence in metrics—42% of CCOs say they’re only “somewhat confident” or “not confident” in the metrics they use to give a true sense of effectiveness

Compliance teams can’t waste time with data that won’t ultimately help them make better decisions.

This presents obvious problems for a program’s defensibility: If you don’t know if or how well your compliance initiatives are working, you’ll be hard pressed to defend or improve your program. But the persistency of this trend is also disconcerting because it limits how far or quickly the compliance function overall can advance if it can’t provide the breadth or depth of business unit analysis as its peers (current or aspirational) in the executive suite. Every other function—from finance to sales and operations to HR—continually monitors, reports on and is held accountable for in-depth analysis of their performance. If compliance wants to consistently and meaningfully contribute to a company’s strategic conversations, while being able to demonstrate and defend its effectiveness to leadership and regulators, then comprehensive and thorough reporting will need to become a matter of course.

And while it’s critical to have compliance risk metrics  about your program in order to analyze effectiveness and make ongoing improvements, there’s still a tendency to cling to “vanity” metrics—hollow metrics that don’t actually help you do much of either. With limited resources—and facing increasingly high stakes, expectations and scrutiny levels—compliance teams can’t and shouldn’t waste their time compiling and analyzing data that won’t ultimately help them make better compliance decisions.

Here are some overly broad and ambiguous metrics that many compliance teams still track—and some suggestions for how to make adjustments or add context to improve their value and utility.

Current compliance risk metric: Incident reports, investigations and resolutions

Misconduct reporting and investigation trends are continuously cited by CCOs as one of their go-to metrics for program and risk management effectiveness. Without some additional context, though, they may be missing opportunities to address the cultural, behavioral or operational factors that enable or exacerbate the misconduct from occurring in the first place.

Proposed compliance risk metrics:

  • Issue trends for key risk areas
  • Issue trends by location, business unit, organizational title, employee demographics (tenure, salary, etc.)
  • Trends between type of misconduct and the factors that contributed to the misconduct—including rationalization, lack of awareness, pressure, etc.
  • Number and type of sanctions applied by incident type, location, business unit, organizational title, employee demographics, etc.
  • Incident drivers to differentiate misconduct that was intentional, rationalized, unwittingly committed, driven by pressure/compensation and any correlations between drivers and risk areas, locations, business units, organizational titles, etc.
  • Relationships between disclosures and incident trends
  • Impact of training, policies, communication and incentives on number and types of incidents, contributing factors and resolutions reached
  • Demographic or behavioral trends among involved parties (reporters, witnesses or subjects)

Current metric: Training initiatives and completion rates

As one of the most fundamental components of even the most nascent compliance programs, CCOs often report on the number and topics of the courses they’re distributing, completion rates and the results of any related comprehension tests or knowledge assessments.

Proposed compliance risk metrics:

  • Training topics and trends for key risk areas
  • Training trends (good and bad) by region, business unit, organizational title, etc.
  • Number of disclosures submitted after conflicts of interest or GT&E training
  • Impact of training rollouts and results on hotline trends
  • Impact of incentives and communication initiatives on training engagement and understanding

Current metric: Policy initiatives and attestation rates

Distributing policies is nothing new for compliance teams, and as a result this has become a fairly mundane “check the box” activity.

Proposed compliance risk metrics:

  • Policy distribution and attestation trends for key risk areas
  • Attestation trends (good and bad) by region, business unit, organizational title, etc.
  • Number of policy exceptions and disclosures submitted alongside rollouts of related policies like conflicts of interest, gifts and entertainment, etc.
  • Relationships between policy attestations and training completion/certifications
  • Impact of policy rollouts on misconduct reporting rates
  • Impact of incentives and communication initiatives on policy engagement and understanding

Current metric: Risk assessment

Many organizations use the results of their risk assessment to prioritize their compliance program efforts—and with good reason, as regulatory guidance continually cites a risk-based approach to compliance as a hallmark of an effective program. But how do compliance teams move beyond using their risk registers as a punch list, to leveraging it for critical context in reporting their effectiveness?

Proposed compliance risk metrics:

  • Risk disposition by location, business unit, organizational title, etc.
  • Risk exposure increases due to incident, disclosure, training, culture assessment or policy trends
  • Correlation or discrepancies—and analysis of reasons for the relationships—between risk assessment results and bellwethers of a company’s cultural environment like culture assessments, incident drivers and more

Yes, these metrics are driven in large part by the expectations set out by the Federal Sentencing Guidelines—but I’d argue that in their most basic form they follow the Guidelines in letter, not spirit. To truly optimize effectiveness and facilitate continuous improvement, context and deeper analytics of the data are needed. Only then can compliance move from a highly reactive function to one that’s cohesive, predictive, proactive and preventative in nature. After all, isn’t that the point?