We recently sat down for a conversation with Donald Farmer, Principal at TreeHive Strategy, about insights vs. reporting and the value of visualizations.
Previously to TreeHive, Donald was at Qlik for six years, where he was both Vice President of Product Management and Vice President of Innovation.
Q: What is the difference between reporting and analytics, and how do we produce insights? Aren’t spreadsheets good enough?
DF: Spreadsheets are good for some cases. But when you think of reporting and analytics as two functions, the distinction between them is that with reporting, you have a very clear idea of what you want. You already have a hypothesis regarding what’s important. It’s been decided, either by the IT department, or the business users themselves or by the report designer. And now you see that report regularly, every day, every week, every month, every quarter, the annual report. It’s a summary of things that you know are important.
Analytics is a little different because it’s about investigation and discovery. When you’re an analyst, you might not know what you’re going to investigate on any particular day or week, but you know you’re going to investigate. It’s the difference between something with predefined criteria versus exploration.
Spreadsheets are not very good for predefined criteria. They are quite good for exploration, but they’re always limited to personal use, or at the very most, limited shared use. In some ways, they’re almost too freeform to be easily governed. It’s an important distinction. The spreadsheet has its role, but it certainly doesn’t have the ability to govern and the ability to create the repeated high-quality insight that you get with a good quality analytic tool.
A culture of analytics is emerging, especially in two areas. There’s always been, to a certain extent, analysis going on in the finance departments who are trying to analyze various financial procedures. And marketing has often taken great advantage of analytics.
But we’re starting to see it emerge in almost every aspect of business. People are using digital information created within the business in more complex and insightful ways because that’s where the advantage lies. Alan Webber, the guy who founded Fast Company magazine, once said that in the digital economy or the new economy, conversations are the most important form of work. Not meaning hanging around the water cooler all day, but that insights about the information you have are what really make the difference in your business. That recognition drives a lot of the analytic culture in business today.
Q: What are the business drivers you see as important?
DF: People are very driven by the need to visually explore. As I said, the investigative aspect of analytics is important. Self-service data preparation is also critical because, increasingly, as people want to get more insights from data, they want to bring it from their desktop sources and external sources: to bring data they may find on the internet, public data, demographic data, into the discussion. So data preparation becomes important. Both of those things are critical to success.
On the other hand, many people overlook the importance of publishing, sharing, and collaboration. They prepare the data to support a decision, but don’t discuss it with other people, or collaborate with them on the factors, the analytics, and the insight that went into that data. To my mind, publishing, sharing, and collaborating on data is absolutely critical to success.
Q: With regard to automating data preparation, how important are real-time analytics?
DF: Real-time analytics are increasingly sought after, but the definition of real-time varies from business to business. Very often my conversations with customers starts as, “Well, you want real-time. What is the rhythm of your business?” For some companies in manufacturing, for example, real-time may be if a machine fails, a reproduction line fails, then it needs to be immediately addressed within minutes. If you’re a brokerage and you’re working with live information from the stock exchange, real-time is maybe even sub-second response times. On the other hand, for businesses that have a slower turnover of goods and services, such as retail, the speed of response may be daily.
Real-time should be defined by the speed at which you can respond. If you can respond within hours or within minutes, then that’s the pace you should be tracking. Once you have that in perspective, it drives not only the definition of real-time within your business, but also how you automate those processes.
In many ways, an automated process is a human process that has been captured and made repeatable without human intervention. That means you must understand very well the logic and the impact of decisions that are going to occur automatically. It can require quite a lot of modeling, quite a lot of experience, and quite a lot of analysis before you can build those systems.
A lot of thought must go into this, and a good business understanding of what real-time actually means for your business. Reducing everything to sub-second-response time just because you technically can might get you very little advantage unless you understand that background.
Q: Regarding visualization tools such as the dashboard from Convercent Insights, how important is the role of visualizing data?
DF: Visualization is critical for three essential purposes. Human beings understand things visually. A good visualization has an impact that can cause you to win and makes you want to engage or stick with the information.
Another aspect of visualization, which I think is very important, especially for people who visualize issues with compliance, is how it facilitates the recognition of patterns, trends, and outliers that you would never pick up if you just saw the information in a tabular report or spreadsheet.
Visualization is also important for dealing with the investigative style, the exploratory style of analytics. Is it’s very easy to recognize change as you modify your selections. For instance, when you select a different month on the dashboard, or you select different views, or you select a different business unit, you can quickly see the differences between those units in a way that gives you what I call information “scent”. Not necessarily an immediate insight, but a good sense of where you should be looking next. You receive clues as to where you should be continuing your investigation.
These things are not strictly scientifically analytic. They’re related to the human psychology of what we’re investigating, and those aspects are very relevant to the needs of analytics and compliance. The ability to see outliers and patterns clearly. The ability to engage people because visualizations are interesting and attractive to human beings. And the additional scent and clues about where we might want to look next. These three aspects are critically important for the success of visualization.
Q: What role does visualization play in correlating information or understanding information across more than one data source?
DF: When it comes to looking at data across data sources, one of the great advantages of visualization is that it enables you to make comparisons very easily. With Convercent Insights, visual comparisons are pretty straightforward to make, even by people without a great deal of training or technical understanding of the data. Compare-and-contrast visualizations are particularly easy to create.
If you’re looking at integrating complex data sources across an organization, one of the nice things that visualization does is give you a shared analytical language. If you think of visualization as a language for explaining data to people, it’s a shared language across organizations. And that’s where we come down to issues such as collaboration, communication, and publishing of information. That shared language enables you to report and analyze many data sources across the enterprise with a common understanding of the results. And that’s always critical.
Q: What about predictive analytics? Is that just marketing jargon, or does it provide something meaningful?
DF: I won’t deny that there’s a lot of marketing jargon around predictive analytics, but it’s also a very real thing. Predictive analytics take the models that we know about our business, how our business has worked in the past, and projects it into the future, and that helps us.
When managing compliance issues, we might find hot spots, the dark spots. Perhaps the analysis and reporting system shows that new managers often have problems, especially if we look at into an existing team with a track record. The very simplest form of predictive analytics allows us to flag new managers. We know there have been problems in the past with new managers, therefore, let’s keep an eye on our new managers.
A more sophisticated model would take in more variables – such as the age and qualifications or job experience of the manager and team members – to make projections that are not easy to find from reporting and analytics. It’s about learning from the past and projecting into the future. It’s not hype. It is built on real technology and provides real value. In particular, predictive analytics enable you to watch for signs that something is going wrong and intervene before they go badly wrong. That’s one of the great values of predictive analytics—to detect not only existing patterns from the past, but to detect that those patterns are starting to repeat themselves and to raise alerts and warnings, enabling positive, proactive intervention. I think that’s the value of predictive analytics within compliance.
Q: Could you give a lesson learned or a best practice from your experience in deploying and using analytics.
DF: Focus on the exploratory use case to give business users access to data to create visualizations themselves. Encourage collaboration as much as possible. As Alan Webber says, “Conversation is the most important form of work.” Collaborate. Collaborate, discuss, publish, and debate. That’s where the value comes from.