All Metrics

See all metrics Haystack supports

The following metrics are available with-in Haystack.

Issue

Metrics

• Issue cycle time: Total time spent in in-progress status category

• Issue lead time: Time from issue creation to issue moved to Done status category

• Issue time in status: Time issue spent on each status breakdown

• Issues created: Number of issues created

• Issues completed: Number of issues moved to Done status category

• Points completed: Number of story points moved to Done status category

  • If an issue does not have story points, it's counted as 0

• Issues injected: Number of issues injected

  • Injected issues are any issue that was added to the sprint after the sprint started.

• Points injected: Number of story points injected

  • Injected issues are any issue that was added to the sprint after the sprint started.

• Issues completed per member: Number of issues moved to Done status category per member

• Points completed per member: Number of story points moved to Done status category per member

  • If an issue does not have story points, it's counted as 0

• Issues created per member: Number of issues created per member

Filters Options

• Assignee

• Priority

• Issue Type

• Label

• Project

• SLA Status

• Team

• Investments

• Epic

• Custom JIRA Field

Group By Options

• Assignee

• Priority

• Issue Type

• Label

• Project

• SLA Status

• Team

• Story points (only for some)

• Custom JIRA Field

Pull Request

Metrics

• PR review time: Time from pull request open to merging

  • Time in draft states are excluded from review time calculation

• PR review time breakdown: Breakdown of review time includes 3 different metrics first response, rework, and idle completion time. Sum of these 3 will equal to review time metric.

• First response time: Time from pull request open to first comment

  • If no comments exist, first response time is 0

• Rework time: Time from first comment to last commit

  • If no comments exist, pull request open is considered as start time of rework time.

• Idle completion time: Time after rework is completed to merging a pull request

• PRs merged: Number of pull requests merged

• PRs unlinked: Number of pull requests unlinked to a Jira issue

• PRs merged per member: Number of pull requests merged per member.

Filters Options

• Author

• Reviewer

• Label

• Repository

• Team

Group By Options

• Author

• Reviewer

• Label

• Repository

• Team

Deployment

Metrics

• Deployments count: # of deployments made

Filters Options

• Service

• Environment

• Team

Group By Options

• Service

• Environment

• Repository

• Team

---

FAQ

Can I track DORA metrics?

Haystack gives building blocks for each DORA metric. You'll need to create your own dashboard.

• Change Lead Time (Issue Cycle Time)

• Deployment Frequency (Deployments Completed)

• Mean Time to Recovery (Issue Lead Time)

• Change Failure Rate

If you are in the search of what metrics to track, we highly recommend checking the following posts.

• First Principles of Engineering Metrics

• Engineering Metrics (Video Series)

Can I see Mean Time to Recovery (MTTR)?

Recommended Read: How to Improve Quality

Option 1: Track via Haystack

Haystack has a tight integration with Jira, but not with issue trackers. This means you'll need to utilize Jira to be able to track it inside Haystack.

This is also a best practice. All work that engineering team does should be tracked in their issue trackers.

1. Create Jira Issues

• Automatically create issues in Jira using OpsGenie

• Automatically create issues in Jira using Pagerduty

2. Create Graph In Haystack

Once you have created a consistent way to create tickets inside Jira whenever incident happens, next step is to track MTTR.

1. Go to Reports page

2. Click New Graph icon

3. Select Issue Lead Time as the metric

4. Select the filters that represents an incidents

   1. Typically teams use

      1. Issue Type: Bug & Priority: Highest

      2. Issue Type: Incident

5. Select the visualization type

   1. Use value to see the raw number

   2. Use line to see it over time

6. Click Save

Option 2: Track via Incident Management Systems

MTTR is a commodity metric meaning almost all Incident Management Systems support it built-in.

Below are the links for most common Incident Management Systems

• Pagerduty

• OpsGenie

Can I see Change Failure Rate (CFR)?

CFR is a metric that can be phrased as "Percentage of our deployments caused incidents".

As long as you have the definition of an incident, you will be able to track how CFR using either Issue completed or Deployments count filtered by an indicator that represents incidents.

Want more actionable metrics for quality? Checkout How to Improve Quality?

Can I see Development Time metric?

No.

As an alternative you can use Issue time in status metric which would give more reliable data.

Development time metric typically refers to time it takes to open a pull request. We have deprecated this metric as it was an unreliable metric. It's only available for older accounts.

There are 2 ways this metric can be calculated which both has reliability issues:

1. JIRA: Mixing timestamps of Version Control & Jira for the start and end of development time leads to large inconsistencies in the data; especially given unlinked pull requests, developers moving an issue to in-progress state chaoticly and other outliers that reduce data quality.

2. Version Control: Mixing timestamps of Version Control & Git for the start and end of development time leads to large inconsistencies in the data; especially given git squashing where it's easy to have 0 development time for some developers who uses this method.

How do I ensure if the metrics are correct?

You can always understand the raw data of any metric by clicking to the graph. A drill-in modal will appear with all the data points that was used to calculate that metric.

How are my team metrics calculated?

The following settings effect how the teams metric is calculated

1. Board Settings

   1. If Kanban: Shows all issues in your board based on your settings

   2. If Sprint: Shows all issues in your sprint based on your settings.

      1. Note: If an issue is in-progress but not in your sprint, Haystack will not show that issue.

2. Sprint Filter (only for sprint based boards)

3. Member Settings

4. Advanced Board Settings

You can always understand the raw data of any metric by clicking to the graph. A drill-in modal will appear with all the data points that was used to calculate that metric.

Do PR metrics count unlinked PRs?

Yes. All Pull Requests merged by team members will be included in calculations.

If you want to check specifically unlinked PRs, check PRs unlinked metric.

Are Tasks/Stories/Epics/Subtasks calculated in Issue based metrics?

Haystack does an implicit filter on metrics to ensure the data looks correct with an initial glance.

Subtask and Epic issue types are not calculated part of any metric. All other issue types would be included in the metrics.

You can validate this by using "Issue Type" filter.

Does Haystack exclude weekends and holidays?

Haystack does not exclude any particular date ranges like holidays, PTO, weekends, etc.

While initially this may feel like your metrics are artificially increased, we think about metrics from the perspective of the customer. In this case, the customer is still waiting during the weekends and the increased metrics continues to be a great signal on where bottlenecks might lie.