The Art of the AI KPI, Part 1

Key Performance Indicators for Analytics.

Estrategia   |   David Sweenor   |   11 de octubre de 2022

“Not everything that can be counted counts and not everything that counts can be counted.”

-Albert Einstein

 

Some say data is the new oil. It’s not, as the World Economic Forum pointed out in 2018. Petroleum is a finite resource, and it takes eons to make more of it. Once the crude is extracted, refined, and consumed, that’s the end of it.

 

Contrast that with data. The amount of data being created is beyond our wildest dreams — it comes in all shapes and sizes and some firms estimate that it is growing at an exponential rate. In fact, the latest IDC Global DataSphere Forecast suggests that the amount will more than double in size by 2026. The fact of the matter is that the sheer amount of data available is incomprehensible to most – how big is a zettabyte, really? To put this into some perspective, it is estimated that “more than 5 billion customers interact with data every day – by 2025, that number will be 6 billion, or 75% of the world’s population. In 2025, each connected person will have at least one data interaction every 18 seconds.”

 

It is anticipated that there will be 175 ZB of data by 2025. “If you were able to store the entire Global Datasphere on DVDs, then you would have a stack of DVDs that could get you to the moon 23 times or circle the Earth 222 times.” [Source: IBID]

 

Now that’s a lot of data – but also speaks to the inefficiency of DVDs as a storage mechanism. In fact, a single gram of DNA can store about 215 petabytes (0.000215 zettabytes).

 

That’s why I prefer to think of data as a renewable resource. Of course, renewable energy involves the costs of installing solar farms, geothermal plants, biomass facilities, hydroelectric dams, and wind turbines. But once they’re in place, the energy released will be available for a long time. Even with ongoing maintenance, the up-front investment will provide continuing value.

 

That renewability ties into the concepts behind Bill Schmarzo’s book, The Economics of Data, Analytics, and Digital Transformation. Once data is transformed into higher value through analytics, your organization will receive that value in perpetuity.

 

There’s no limit on how many times and ways you can transform the data — its value will continue to increase. Here’s why.

 

Before Thinking About KPIs, Understand Your Game Plan

 

Before energy companies invest in developing a site, they generally gauge the fit with their corporate strategy. They conduct significant research to ensure that the infrastructure and facilities will have a positive, sustainable, significant return on investment. In the case of wind turbines, important factors include understanding the wind resource (minimum average annual wind speed), access to land and capital, proximity to transmission lines, and zoning regulations.

 

Similarly, before organizations embark on any analytic and AI initiative, business leaders need a clearly defined business strategy. Good business strategies often give rise to objectives and key results (OKRs). To achieve these OKRs, organizations will have strategic initiatives comprising one or more projects, each of which often requires AI and analytics.

 

AI should fuel business strategy.

 

For any given project, I always encourage business leaders to understand the business decision to be made, and then work backwards from there. After all, we don’t build wind farms for the sake of it. We construct them to provide value to society and returns to our company and municipality.

 

Failure to have a clear strategy can be disastrous. When your data isn’t as good as you think it is, you draw the wrong conclusions and waste effort. Or, you can spend so much time collecting data that you miss the window of opportunity. And some companies over invest in their data science team and underinvest in spreading data literacy among their workers.

 

The conversations we have with our customers start with the end goal. And our customers continually show us the value in conducting due diligence on business processes to see how data and analytics can be used to achieve their business goals. Everything starts with an objective.

 

Key Performance Indicators for Analytics

 

With the current economic uncertainty that surrounds us, Chief Data and Analytic Officers are under increasing pressure to track the business impact of data, analytics, and AI initiatives on business outcomes. In most organizations, that leads to talk of key performance indicators (KPIs).

 

But why should we care about KPIs at all? According to Michael Schrage, a visiting research fellow at the MIT Initiative on the Digital Economy, “When applied to digital transformation efforts, KPIs can drive business strategy and provide a distinct competitive advantage.” Furthermore, Schrage has identified seven fundamental characteristics of enterprises boosting their return on KPIs. They are:

  • KPIs help to lead and manage.
  • KPIs help to align the entire organization.
  • KPIs provide a holistic view of the customer.
  • KPI factors inform decision making.
  • KPI data is shared across business units.
  • KPIs don’t proliferate. Leaders should focus on 3 to 6 KPIs that will drive growth.
  • KPIs serve as data sets for machine learning (ML).

So, which are the best KPIs for your organization to track? Before you can answer that, your business leaders need to answer these questions:

  • What do you want to know?
  • How much do you need to know?
  • Why do you want to know this?
  • What is the value or impact between not knowing and knowing?

Once you know the answers to these questions, you can start establishing KPIs to track your success.
Independent of data, analytics and AI, we find that KPIs generally fall across the following dimensions:

  • Financial — Revenue growth, cash flow, burn rate, gross profit
  • Customer — Engagement rates, net promoter scores, acquisition costs, conversion rates
  • Support & Service — Turnaround time, mean time to resolve, SLA compliance, quality
  • Employee — Attrition and retention, satisfaction, engagement
  • Governance, Risk, & Compliance — Percent compliance to process, audit compliance, non-security incidents

In my next post, I’ll describe an approach to KPIs specifically for data, analytics and AI. Meanwhile, see where your organization sits on the analytic maturity curve and have a look at which use cases to start with.

Tags
  • Analytics Leader
  • Business Leader