One of the biggest challenges marketers and data teams face is cleaning data. More specifically, getting metrics from different sources into a format where they can be directly compared, and queried. One of the reasons this is more complex than it sounds is that many metrics don’t have a single definition that’s used by all platforms and teams.
The calculations for CTR (click-through rate) might seem straightforward and commonly understood, but when two platforms have different definitions for what constitutes a click, data analysis can start to get messy. This is where a metrics layer comes in to ensure that metrics are standardized, paving the way for accurate data analysis.
What is a metrics layer?
A metrics layer is basically a recipe for standardizing metrics. It’s a way to centralize how an organization calculates the KPIs that it uses in reporting in any domain.
As a data process, the metrics layer sits along the data retrieval pipeline and performs calculations on metrics from different sources based on standardized definitions. This ensures that metrics from different sources are directly comparable. To achieve this, the metrics layer contains metadata around metrics and how they’re calculated and formatted.
What does that mean?
When an organization has several different definitions of KPIs or metrics being used by different teams, this inevitably leads to confusion. The idea of the metrics layer is to standardize and centralize the definition of metrics so that every time you retrieve data from a system, you’re working with metrics that use the same calculation.
The metrics layer catalogs that calculation and makes it accessible to everyone, so they understand what these metrics mean, and that they come from one source of truth.
Historically analysts and data scientists would do these calculations in excel sheets and notebooks. However, automating metrics layers has become a necessity as the volume and granularity of data continues to grow.
The configuration of a metrics layer can differ greatly from one organization to another. Less data-mature teams might depend on a single person to manually clean and standardize data, and then copy it into a spreadsheet. However, teams with more advanced data ops might rely on code that automatically streamlines this standardization, and sends data to easy-to-understand dashboards instead of sprawling excel documents. The most data mature teams might even utilize machine learning as a part of this process to reduce human touch even further.
Why are metrics layers important?
1. Accurate aggregation and comparison between platforms
Each platform uses slightly different definitions for concepts like clicks or views. By standardizing these metrics across platforms, metrics layers ensure that data can be compared directly.
2. Accurate aggregation and comparison between teams
Metrics layers require proper definitions in place to define the KPIs your organization is working towards. This means there is less data disparity across different teams within the same organization.
3. Fast and flexible data analysis
Standardizing metrics before data can be fetched means much less manual, error-prone work on the part of the end user.
4. Avoiding bottlenecks to empower less data-fluent employees
Without a metrics layer, the less data-fluent employees rely on data engineers to translate and clean data for each use case. When a metrics layer is in place, the less data-fluent users can work faster, and more independently without this bottleneck.
So, metrics layers allow businesses to aggregate and compare metrics accurately across platforms and across their business. But what happens if you don’t have one?
The risks of not having a metrics layer include:
- Inaccurate comparisons of data between platforms and teams lead to poor visibility and bad decisions.
- Distrust in data leads to a culture where employees don’t feel confident making data-driven decisions that could give your organization a competitive advantage.
- Inefficient manual work to translate and clean data for each use case.
When should a metrics layer be introduced to the data pipeline?
Users have less flexibility once the metrics layer calculations have been implemented. They don’t have access to the raw data as it’s been transformed by the metrics layer calculations, and this can create limitations. So, deciding when to introduce a metrics layer to your data flow can take some consideration. Essentially there are two ends of this spectrum you need to weigh up:
- Implementing the metrics layer earlier on in the data pipeline and having less control over more data.
- Implementing the metrics layer later in the data pipeline and having more control over less data.
Option two might be better for larger companies with many end users who have lower levels of data fluency, as it gives them more freedom to query data without breaking tables or datasets. However, the limitations of this won’t work for everyone. For those who want more ways to manipulate their data, unifying data later in the pipeline comes at the cost of being able to make changes that require access to the data in a more raw form.
Challenges when implementing a metrics layer
Once a metrics layer is in place, teams can compare data from a single source of truth much more easily. However, there are some challenges you might face on the way to implementation. Here are just a few that you’ll need to be aware of.
1. Alignment
Oftentimes if your organization has two teams using different definitions, there might be specific reasons for this. For example, working with marketing cost might refer to cost coming from Meta, but it could also refer to the marketing cost coming from a marketing agency including their fee. In order to reach a decision, teams need to collectively figure out what their data priorities are and agree on the definition of a metric.
2. Anomalies
There will be certain cases where you can't create a standardized definition. This might be because the API doesn’t offer granular enough raw data to complete a specific calculation - or it might be because of an internal disagreement as in the point above. This means you might end up stuck with a particular metric definition from one platform that doesn’t compare directly with the rest of your data. In this case, you’ll need to format data fetches around this to explain and highlight where there are disparities.
3. Balancing efficiency with granularity
Many data storage and visualization platforms work off a pay-per-process model, meaning the more data you input, the more you pay — so data teams will want to optimize datasets for this. However, they need to find the balance between having granular data and flexibility so you’re getting what you need without overpaying. You can check out our blog on how to democratize data for more on this.
4. Getting people the right data skills on your team
Without efficient metric definitions and calculations in SQL, data teams often end up paying more than they need to for rows they’re not using efficiently. But finding the right people and skillsets to master this can be tricky, as the talent market is currently facing a data drought. At the moment, it’s hard to hire data analysts to make the metrics layer as efficient as possible.
