Automating workflow using Runbook conditions

FireHydrant offers powerful capability to customize automations of Runbook steps. With Runbook conditions, you can control when certain Runbooks as well as steps within them should attach/execute.

Runbook-Level Conditions

Runbook-level conditions will determine whether a Runbook will attach to an incident. This allows you to create Runbooks tailored to specific situations, such as only SEV1 incidents, or only incidents where my-specific-service is impacted.

FireHydrant is always scanning your incidents for matching Runbook conditions. This means Runbooks can attach to any of your incidents at any point in the incident lifecycle**.

**With some exceptions. See Condition Expiration below.

This also allows powerful "layering" capabilities. For example, an incident can be "escalated" from a SEV2 to a SEV1 and subsequently fire off only SEV1-specific automations.

Or, you can have shared and specific Runbooks that attach as the situations demand:

How to edit runbook conditions

1. Click Runbooks in the navigation and then click the Runbook you want to configure.

2. On the summary page, click Edit Runbook to configure changes for the Runbook.
3. In the edit page, you can configure Runbook conditions on the right panel under Execution rules.

Step-Level Conditions

In addition to Runbook-level conditions, individual steps can also conditioanlly execute. This allows flexible and powerful tailoring of each Runbook.

• Rules
  • All of the available conditions per-step are the same conditions available at the Runbook level, with a couple of exceptions.
• Scheduling
  • You can configure certain Runbook steps to repeat on intervals. See Repeating Steps.
  • You can also configure whether a step should immediately schedule and evaluate the condition, or if it should be initiated manually by a user after the Runbook has attached. This can be done via the UI or via /fh runbooks in Slack.

How to edit step conditions

1. Follow the same steps above to enter edit mode for the Runbook in question.
2. Find the specific step you'd like to add conditions to and on the right side, click the writing pad.

3. In the panel for that step, click Conditions & scheduling. This opens your options for applying rules that dictate when you want this specific step to run.

Repeating steps

Some Runbook steps allow for scheduling repeated intervals. This allows, for example, reminding the incident channel to post updates every 30 minutes. If a step allows this, you will see it as an extra option under Scheduling, pictured above.

Note: The shortest interval allowed today is 5 minutes.

Once a recurring step has been scheduled, you'll see it in the Command Center page on the Runbooks tab. You can also stop repeating Runbook steps from here as well.

Condition Expiration

Runbooks and Steps will not always execute or be on the lookout for forever. FireHydrant expires condition checks for Runbooks and Steps based on various conditions.

Runbook Expiration

FireHydrant will stop checking incidents for matching Runbook conditions when one of the following occurs:

• The incident reaches the Resolved milestone
  • After an incident is resolved, no new Runbooks will automatically attach, even if conditions match. You will need to attach them manually.
• The Runbook has already attached
  • We do not support unattaching and reattaching the same Runbooks to the same incident at this time. Other Runbooks not yet attached should still evaluate as normal.
• The incident is older than 30 days
  • All conditions expire after 30 days even if an incident has not resolved yet, so any changes to an incident after 30 days will not trigger a new Runbook attachment.

Step Expiration

FireHydrant will expire steps or certain aspects of steps when one of the following occurs:

• When the incident reaches the Resolved milestone
  • FireHydrant terminates any recurring steps when an incident is resolved
  • This does not apply to post-incident steps/conditions (e.g. When current milestone is Retrospective Completed). Those will still execute as normal post-resolution.
• When a step has been waiting for longer than 30 days
  • If a step has been waiting and checking for matching conditions for longer than 30 days, the step will expire.

Next Steps

• Set up some basic conditions in your Runbooks and test them out.
• Tailor the Runbooks for different situations (e.g. SEV1 vs. SEV2) or for different teams and impacted components.
• Add complexity to Runbook conditions by linking multiple sets of rules together with AND or OR statements.
• Browse the documentation on the left for the various different Runbook steps available, or check out our available integrations which will unlock more Runbook steps once configured.