It’s the beginning of another quarter. Thomas, director of engineering, proudly presents his team’s performance in front of the board. The dashboards are shimmering green. “In the last quarter we closed 1,200 tickets, 15% more than in the previous quarter,” he references. “Our teams delivered a total of 500 ‘story points,’ and code coverage with unit tests rose to an impressive 85%!” Thomas finishes his presentation, expecting expressions of appreciation. Instead, after a moment of silence, the product director speaks up: “This all sounds impressive, Tomasz. But in the same quarter, the customer churn rate increased by 10%, and the adoption of all those new features you delivered with so much effort was less than 5%. So tell me, what have we really accomplished with all this work?” In that one moment, Thomas realized the brutal truth. His teams were running faster than ever, but they were running on a treadmill. They were generating an enormous amount of “activity,” but not necessarily “impact.” He was measuring the effort, not the result.

Read also: Automated vs. Manual Testing in 2026: Where Does AI Change t

This scene is a daily occurrence in many technology organizations. We’ve fallen into the trap of measuring what’s easy to count, rather than what actually matters. Lines of code, number of deployments, “velocity” - these are all metrics that give us an illusory sense of control and progress, but rarely say anything about the real value we deliver to customers and the business. Truly high-performing IT organizations are different from others not in that they work more, but in that they can accurately measure and optimize their impact on business performance. This article is a practical guide for leaders who want to stop managing activity and start managing results. We’ll explain the fundamental differences between two key tools - KPIs (Key Performance Indicators) and OKRs (Objectives and Key Results) - and show you how to build a consistent measurement system that connects the day-to-day work of development teams to the strategic goals of the entire company.

Why are traditional IT productivity metrics (story points, lines of code) misleading and harmful?

“The average defect removal efficiency of most software organizations is about 85%, meaning 15% of defects are delivered to customers.”

— Capers Jones, Applied Software Measurement | Source

For years, the IT industry has tried to find the “holy grail” - a single, simple metric to measure developer productivity. Thus, metrics such as the number of Lines of Code (LOC), the number of closed tickets or “velocity” (the number of “story points” completed in a sprint) were born and popularized. While each of these metrics can be useful in a very narrow, diagnostic context, using them as a primary measure of productivity, or worse, as a basis for evaluating and rewarding teams, is extremely damaging.

1. measure effort, not value: Writing 1,000 lines of code that is complicated, full of bugs and solves no problem is far worse than writing 10 lines of elegant code that solves a key customer problem. Similarly, a team that “burns through” 50 “story points” to build a feature that no one uses is far less productive than a team that implements 10 “story points” to create a small but extremely valuable change. These metrics say nothing about the business value of the work done.

2 They encourage inappropriate behavior: When people know they are being judged on a particular metric, they begin to optimize their behavior for that metric, often at the expense of common sense (known as Goodhart’s Law).

  • Are you measuring lines of code? Developers will start writing more long-winded, complex code to “crank out” a better score.

  • Do you measure the number of closed tickets? Teams will focus on closing the easy, trivial tasks quickly, avoiding the difficult and strategically important ones.

  • Do you measure “velocity”? Teams will start “pumping up” estimates (inflation of “story points”) to show that they are getting faster, and quality and technical debt will recede into the background.

3. are incomparable between teams: “Story points” are a subjective measure of complexity, unique to each team. Team A, which delivers 20 “story points” per sprint, is not necessarily less productive than Team B, which delivers 50. Comparing “velocity” between teams is one of the biggest and most common abuses of agile methods. It leads to unhealthy competition and destroys trust.

4 They ignore “invisible” work: Many key engineering activities do not translate directly into new features. Work on refactoring, improving the CI/CD pipeline, writing documentation or mentoring junior colleagues is absolutely critical to the long-term health of the system, but is not reflected in metrics such as “story points.” Focusing on these discourages investment in these important but “invisible” tasks.

Using these metrics to evaluate productivity is like judging a restaurant by the number of plates consumed, rather than by the taste of the food and guest satisfaction. To measure what really matters, we need a completely different set of tools.

What are vanity metrics and how do you recognize them in your team?

Vanity Metrics are metrics that look impressive at first glance, but in reality tell you nothing about the real health of your business and don’t help you make better decisions. They are often easy to measure and look beautiful on charts going up, which makes managers love to brag about them. The problem is that they often measure noise, not signal.

Eric Ries, creator of the Lean Startup methodology, defined a simple test to distinguish vanity metrics from actionable metrics: “Does this metric allow me to make a specific, useful decision? If not, it is probably a vanity metric.

Examples of vanity metrics in IT and business:

  • Number of registered users: The graph of the total number of registered users is almost always increasing. This looks great. But what if 95% of them registered, used the app once and never returned to it? A metric of value would be the number of active users in the last week or retention rate.

  • Number of mobile app downloads: One million downloads sounds like a success. But how many of those people actually launched the app? How many of them have gone through the onboarding process? How many of them use it regularly?

  • “Velocity” (story points per sprint): As we discussed earlier, this metric is very easy to manipulate and says nothing about the value delivered to the customer.

  • Number of deployments per day: By itself, this metric can be a vanity metric. So what if we deploy 20 times a day, if 10 of those deployments are hotfixes that fix bugs introduced by the previous 10 deployments? Only when combined with Change Failure Rate (the rate of failed deployments) does it become part of a valuable picture.

How to recognize vanity metrics?

  • It measures the “what?”, not the “why?”: It informs you of what happened, but does not help you understand why it happened and what to do about it.

  • It is impossible to conduct an experiment based on it: You caot formulate a hypothesis and verify it by looking at the change in this metric.

  • This is a summary metric, not a segmented one: The total number of users says nothing. But already the number of users segmented by acquisition channel or time cohort becomes a valuable metric.

  • It is easily manipulated: If a team can easily “improve” a metric without improving the real value, this is a warning signal.

The trap of vanity metrics is that they give a false sense of success and put people to sleep with vigilance. They lead to bad decisions based on incomplete or misleading data. The first step to building a healthy measurement system is to ruthlessly eliminate vanity metrics and replace them with metrics that reflect real business impact.

KPI vs OKR: what is the fundamental difference and when to use each tool?

In discussions about measuring success, two acronyms come up constantly: KPI and OKR. They are often confused or used interchangeably, which is a source of much confusion. In fact, KPIs and OKRs are two different but complementary tools that serve completely different functions. Understanding this difference is key to building a sustainable and effective measurement system.

KPIs (Key Performance Indicators):

  • What are they? KPIs are indicators of the health of a system or process. They are metrics that we monitor continuously to make sure everything is working as expected.

  • Metaphor: KPI is the dashboard of a car. It shows key parameters: speed, engine temperature, fuel level. As long as the indicators are in the “green zone,” you don’t need to take any special action - you just keep driving. However, if any indicator enters the “red zone” (for example, the oil light comes on), this is an alarm signal that requires immediate action.

  • Purpose: The purpose of KPIs is to **monitor and maintai ** the status quo or make incremental, evolutionary improvements.

  • Examples in IT: Service availability (uptime), API response time, Change Failure Rate, Customer Satisfaction (CSAT).

OKRs (Objectives and Key Results):

  • What are they? OKRs are a framework for setting and achieving ambitious goals. It is a tool for managing change and innovation.

  • Metaphor: OKR is a GPS navigation system. It doesn’t tell you your current speed or engine temperature. It tells you where you want to get to (Objective) and shows you what milestones you need to achieve to get there (Key Results). You use it when you want to actively change your position from point A to point B.

  • Goal: The goal of the OKR is to achieve a significant, breakthrough change within a specified, usually quarterly, time horizon.

  • Example in IT:

  • Goal: “Revolutionize the user experience during the onboarding process.”

  • Key Results: 1. Reduce the time required to complete onboarding from 10 minutes to 3 minutes. 2. Increase the rate of successful completion of onboarding from 60% to 90%.

How do KPIs and OKRs work together?

KPI and OKR are not in conflict - they are in symbiosis.

  • KPIs inform the need for an OKR: If one of your key KPIs is starting to decline (e.g., API response time is increasing), this could be a signal that you need to create an OKR focused on improving performance in the next quarter.

  • OKR can become a new KPI: Once an OKR is successfully completed and a new, higher level of performance is achieved (e.g., service availability increased from 99.5% to 99.95%), this new level can become a new standard, monitored by a KPI.

In short: KPIs are “business as usual” and OKRs are “business as unusual.” You need both. KPIs tell you whether your business is healthy. OKRs tell you in which direction it should grow.

What examples of good KPIs can be used to measure reliability and performance (DORA metrics)?

Defining good KPIs is an art. They have to be measurable, relevant and, most importantly, they have to reflect what is really important to the business and customers. In the world of modern IT, the absolute best and most proven starting point are the so-called DORA metrics, which, as we mentioned in the context of DevOps culture, are the result of years of research into what distinguishes high-performing technology organizations from low-performing ones.

DORA’s metrics are four key performance indicators (KPIs) that perfectly balance speed and stability.

Throughput metrics:

1 Deployment Frequency:

  • What does it measure? How often an organization successfully implements changes to the production environment.

  • Why is it important? This is an indicator of the agility and maturity of CI/CD processes. High-performance organizations deploy changes on demand (multiple times a day), while low-performance ones do so less frequently than once a month. High deployment frequency (in small batches) reduces risk and accelerates value delivery.

  • Maturity levels: Elite: on demand. High: once a day/week. Medium: once a week/month. Low: less frequently than once a month.

2 Lead Time for Changes:

  • What does it measure? How much time elapses between code approval (commit) and successful deployment to production.

  • Why is it important? It is a measure of the efficiency of the entire delivery pipeline. Short time means that the process is automated, efficient and free of bottlenecks (e.g., long waits for manual testing or approvals).

  • Maturity Levels: Elite: < 1 hour. High: < 1 day. Medium: 1 day - 1 week. Low: > 1 week.

Stability Metrics (Stability):

3 Change Failure Rate:

  • What does it measure? What percentage of production deployments cause a failure (e.g., require an immediate fix - hotfix, rollback - rollback, or cause an incident).

  • Why is it important? It is a key indicator of process quality and reliability. It shows whether speed is not achieved at the expense of stability. The goal is to keep this indicator as low as possible.

  • Maturity levels: Elite: 0-15%. High: 16-30%. Medium: 16-30%. Low: 46-60%.

4. Time to Restore Service (MTTR):

  • What does it measure? How quickly an organization can restore full operation of a service after an incident or outage in production.

  • Why is it important? It is a measure of the resilience (resilience) of the system and the effectiveness of incident response processes. In complex systems, failures are inevitable. The key is how quickly we can deal with them.

  • Maturity Levels: Elite: < 1 hour. High: < 1 day. Medium: 1-7 days. Low: > 1 week.

These four metrics create a balanced system. The first two (speed) push the organization forward, and the second two (stability) act as an emergency brake, ensuring that we are not going too fast. Regularly measuring and striving to improve these four KPIs is one of the most powerful ways to build a high-performance technology organization.

How do you write a good objective (goal) that is inspiring and points in a direction?

The Objective (Objective) in the OKR framework is more than just a line of text. It is the heart and soul of the entire quarterly effort. It is designed to answer the question “Where are we going?”. A well-formulated objective should be like a polar star - pointing in a clear direction, inspiring the team and ambitious enough to pull people out of their comfort zone.

Here are the key characteristics of a good target:

1 It is Qualitative and Inspirational: The goal should not contain any numbers. It is supposed to describe the desired future state in a way that is engaging and motivating. It should be something that people are proud to hang above their desk.

  • Wrong example (metrics as a goal): “Increase MAU by 15%.”

  • A good example (inspiring vision): “To create the most engaging experience for our new users.”

2 It’s Ambitious and a Little Uncomfortable: OKRs are not meant to describe day-to-day work (“business as usual”). They are used to achieve breakthroughs. A good goal should be ambitious, and its full realization should seem difficult at first. At Google, reaching 70% of a goal is considered a success. If you regularly reach 100%, it means your goals are not ambitious enough.

  • Bad example (safe, easy to achieve): “Keep the platform running.”

  • A good example (an ambitious, “stretch goal”): “To achieve the legendary reliability and performance of our platform.”

3 It is Action Oriented and Time Constrained: The goal should suggest action and be embedded in the timeframe of the OKR cycle (usually a quarter). It should be phrased in a way that gives a sense of urgency.

  • Bad example (passive, indefinite): “Improving customer satisfaction.”

  • A good example (active, embedded in time): “Deliver world-class technical support to our customers this quarter.”

4 It is Understandable and Unambiguous to the Team: The objective must be simple and clear. Each person on the team, reading it, should easily understand what the priority is for the coming quarter. Corporate jargon and complicated wording should be avoided.

  • Bad example (jargon, unclear): “Synergize our core competencies to maximize impact on the market paradigm.”

  • A good example (simple, clear): “Making our product the easiest and fastest solution on the market.”

Writing good goals is an art that takes practice. It is an exercise in strategic thinking and leadership. A well-set goal is half the battle - it gives meaning and energy to all the team’s work for the coming three months.

How do you define measurable key results (key outcomes) that actually measure progress, not tasks?

If the Objective (Objective) answers the question “Where are we going?”, then the Key Results (KRs) answer the question “How will we know we got there?”. It is at this stage that the mistake of confusing outcomes (outcomes) with actions (outputs) is most often made.

Key Result is not a task to be done. It’s a measurable outcome that is the result of completing multiple tasks.

Bad practice: KR as a task list (output-based)

  • Goal: Streamline the process of deploying new customers.

  • KR1: Launch the new onboarding wizard.

  • KR2: Prepare 5 new instructional videos.

  • KR3: Conduct training for the support department.

What is wrong with the above KRs? You can “tick off” all three, and yet the implementation process will not improve at all. We measured the performance of the work, not the result.

Best practice: KR as an outcome measure (outcome-based)

  • Goal: Create a seamless and lightning-fast implementation process for new customers.

  • KR1: Reduce the average time from registration to activation of a key function from 48 hours to 2 hours.

  • KR2: Increase the percentage of clients who complete the onboarding process on their own (without contacting support) from 50% to 85%.

  • KR3: Increase the satisfaction rate with the onboarding process (as measured by a survey) from 6/10 to 9/10.

Now we see a fundamental difference. These KRs explicitly define what “process improvement” means. They give the team autonomy to decide what tasks (launching a wizard, instructional videos, or something else entirely?) will best contribute to achieving these measurable results.

Features of a good Key Result:

  • It is Measurable and Verifiable: It must include a number. There must be an unambiguous way to tell at the end of the quarter whether it has been achieved or not. There is no room for subjective evaluations.

  • It measures Value, not Activity: It focuses on the impact on the user or business, not on the delivery of the feature. A good test is to ask the question: “So what?” If the answer to “We’ve delivered a new feature” is “So what?”, that means it’s not a good KR.

  • It’s Ambitious but Realistic: Like a goal, a CoR should be challenging. It should require effort and creativity from the team.

  • It is controllable by the team: the team must have a real impact on the metric it is measuring. Setting a KR of “Increase company revenue by 20%” for a single development team is a mistake, because it has no direct influence. But already a KR of “Increase conversions on a pricing page by 15%” is well within its reach.

Defining good, results-based key results is the hardest part of working with OKRs. It requires a shift in thinking, a deep understanding of the product and the discipline to separate what we do from what we want to achieve.

How do you combine long-term KPIs with quarterly, ambitious OKRs?

Creating a measurement system that harmoniously combines stability and monitoring (KPI) with ambition and change (OKR) is the sign of a mature, data-driven organization. These two tools should not live in separate worlds. They should form a cohesive, dynamic system in which one informs and influences the other.

“Health and Goals” model: You can think of it in terms of human health.

  • KPIs are our “vital signs.” Blood pressure, heart rate, temperature. We monitor them regularly. As long as they are normal, we don’t pay special attention to them. This is a “business as usual” state. Our goal is simply to “be healthy.”

  • OKR is our “training plan.” When we want to achieve a specific, ambitious goal - such as running a marathon - we create a plan. Our Goal becomes “Complete the marathon in under 4 hours.” Our Key Results are, for example, “Increase the weekly distance to 50 km,” “Improve the 10 km personal best by 5 minutes.” We work intensively on these goals for several months. During this time, we continue to monitor our “vital signs” (KPIs) to make sure that intense training does not negatively affect our health.

How does this work in practice in IT?

  • Define your KPIs: Each product or platform team should define and regularly monitor a small set (3-5) of KPIs that reflect the “health” of its area. For a product team, these might be user engagement and satisfaction metrics. For the platform team, these would be DORA metrics. These KPIs are visible on permanent dashboards.

  • Use KPIs to identify problems: Regular review of KPIs allows early detection of problems. If an application’s load time KPI starts to steadily increase (worsen), this is a signal that action needs to be taken in the next quarter.

  • Create an OKR to solve a problem or seize an opportunity: A deteriorating KPI becomes a natural candidate for a quarterly OKR. The team can define a Goal: “Make our application lightning fast,” with Key Results related to specific performance metrics. OKRs can also result from new strategic initiatives that are not tied to existing KPIs.

  • Monitor KPIs during OKRs: While the team is working hard to achieve its OKRs (e.g., implementing new, innovative features), it must simultaneously keep an eye on its “health” KPIs. Doesn’t the pursuit of novelty come at the expense of stability? Isn’t the rate of failed implementations growing dangerously? This keeps things in balance.

  • Once an OKR is achieved, it can become a new KPI: When a team successfully achieves an OKR and, for example, permanently reduces application loading times to a new and better level, that new level can become a new standard, monitored by a KPI.

This dynamic cycle, in which KPIs inform needs and OKRs drive change, creates a powerful, self-improving system that allows for simultaneous stability and the pursuit of innovation.

What are the most common mistakes and pitfalls when implementing OKRs in technology teams?

The OKR framework is deceptively simple in concept, but extremely difficult to implement correctly in practice. Technology teams, with their tendency to focus on solutions and tasks, are particularly prone to falling into the same repetitive traps.

1. confusing Key Results with Initiatives (Tasks): This is the number one mistake we discussed earlier. Teams create to-do lists and call them Key Results. This leads to a culture of “ticking off” tasks rather than achieving results. The cure: Always ask the question, “So what?” “We delivered feature X.” -> “So what?”. The answer to the second question is probably a good candidate for KR.

2 Too many OKRs: Teams, excited by the new system, try to set 5-7 ambitious goals and 5 OKRs for each of them. This is a simple path to loss of focus and paralysis. The cure: The “less is more” rule. The team should not have more than 1-3 Objectives per quarter, with 2-4 Key Results for each. OKRs are meant to answer the question “What is absolutely most important this quarter?”.

3. Cascading OKRs hierarchically: Traditional managerial thinking leads to an attempt at rigid, top-down cascading of OKRs, where a manager’s goals become his subordinate’s key outcomes. This kills autonomy and commitment. The cure: Goals should be consistent (aligned), not cascaded. Once company OKRs are a

ounced, teams should have the autonomy to define their own OKRs that they feel will best contribute to the overarching goals. The process should be more networked than hierarchical.

4. setting goals that are too unambitious (“sandbagging”): If OKRs are tied to the bonus system and employee evaluations, people will immediately start setting easy, safe goals that they are sure to meet 100%. This kills the whole idea of ambitious “stretch goals.” The cure: OKRs should be completely separated from salary evaluations. OKRs are a tool for navigation and learning, not for evaluation.

5 “Set and Forget” (Set and Forget): The team enthusiastically defines OKRs at the beginning of the quarter, and then returns to them only at the end to check the result. In the meantime, OKRs have no impact on daily work. The cure: OKRs need to live. Establish a regular weekly ritual (e.g., a 30-minute meeting on Friday) where the team reviews progress on OKRs, discusses problems and plans priorities for the following week in the context of quarterly goals.

OKR implementation is a learning process. The first quarter will almost certainly be unsuccessful. The key is patience, consistency and regular retrospectives to adjust and improve the process in subsequent cycles.

What does a measurement system look like for a modern IT organization that combines KPIs and OKRs?

The table below shows a sample, simplified model of how continuous “health” monitoring (KPIs) can be combined with quarterly, ambitious targets (OKRs) in different areas of responsibility of a technology organization.

CategorySample KPIs ("Health of the System").Sample Objective (OKR Objective)Sample Key Results (OKR Key Results).
**Reliability and Stability**Service availability (Uptime) > 99.9%. Rate of failed deployments (Change Failure Rate) < 15%. Achieve the legendary reliability of our flagship platform.Increase availability from 99.9% to 99.99%. Reduce the mean time to restore service (MTTR) from 4 hours to 30 minutes. Reduce the number of critical production alerts by 75%.
**Speed and Throughput**Lead time for change implementation (Lead Time) < 24h. Deployment frequency > 1 deployment/day. Drastically accelerate our value delivery cycle to enable faster experimentation.Reduce the average time from commit to deployment from 24 hours to 2 hours. Increase the frequency of deployments from 1/day to 10/day.
**Product Quality and UX**Customer satisfaction score (CSAT) > 8/10. Number of critical errors reported by customers < 5/month. Delight our mobile users with a seamless and intuitive experience.Increase the App Store rating from 3.8 to 4.7. Reduce the app crash rate from 1% to 0.1%. Increase the key workflow completion rate by 40%.
**Business Value**Retention rate > 90%. Customer acquisition cost (CAC). Customer lifetime value (LTV). Turn our free version of the product into an effective tool for generating paying customers.Increase the conversion rate from free to paid from 2% to 5%. Increase monthly revenue from new subscriptions by 50%.

Need testing support? Check our Quality Assurance services.

Let’s discuss your project

Have questions or need support? Contact us – our experts are happy to help.

How does ARDURA Consulting help organizations implement measurement systems that drive results?

At ARDURA Consulting, we understand that implementing an effective measurement system is one of the most difficult, yet most important, tasks of a leader. It’s a process that requires not only knowledge of the tools, but most importantly the ability to think strategically, to facilitate cultural change, and to have a deep understanding of how to connect technology to the business.

As a trusted advisor (Trusted Advisor), we support our clients at every stage of this journey:

  • Audit and diagnosis: We start with a review of your current metrics and processes. We help identify “vanity metrics” and understand where the biggest gaps in your measurement system lie.

  • System Design: Together with your leadership team, we design a cohesive system that combines the right KPIs to monitor health and an OKR framework to drive change. We help define initial, meaningful metrics and targets that align with your strategy and maturity level.

  • Facilitation and implementation: We help with the practical implementation of the OKR cycle. We conduct workshops for teams, teaching how to write good objectives and key results. We act as coaches and facilitators in the first, most difficult cycles of planning and evaluation, helping to build good habits.

  • Technology support: We advise on the selection and implementation of tools that support the measurement process - from monitoring and observability platforms to software for managing OKRs. Our engineering teams can help automate the collection and visualization of key metrics.

At ARDURA Consulting, we believe that companies that can effectively measure what matters win. Our goal is to give you a compass and navigation that will allow your organization to not only move quickly, but more importantly move in the right direction.

If you feel that your teams are busy but not necessarily effective, and you want to start making decisions based on hard data rather than intuition, consult your project with us. Together we can build a system that will turn effort into real results.