Service Status Update: February 13, 2025

Summary

On Feb 13, 2025 at 16:11:51 UTC, we experienced an issue with the visual rendering of retrospective data where it appeared customer retrospective data was lost.

Slack, MS Teams, Signals, and web functionality were not impacted.

We’ve confirmed there was no data loss from this incident.

What Happened

While backfilling customer data from the old retrospective experience to the new experience, our frontend library parser misconfigured html elements from snake case to camel case. Our frontend editor was not able to visually render this incorrect format, despite the data being available. 

Out of an abundance of caution, we rolled back the new experience and thoroughly checked all logs to ensure that there was no data loss. 

Once we confirmed that there was no data loss, we backfilled all retrospectives to ensure that the data was in the correct format for the frontend to render. 

Customer impact

• Customers were unable to view list elements in their retrospectives 

No customer data was lost or compromised during this incident. 

Users were able to declare and manage incidents with no issue on Slack and MS Teams.

All Signals notifications were delivered as expected with no delays.

Resolution 

We quickly rolled back the new retrospective experience and confirmed that there was no data loss. 

All systems were returned to normal operations by 17:57:19 UTC.

Commitment to Reliability

We understand that you rely on our services for your critical operations. We are committed to maintaining the highest levels of service reliability and transparency, and encourage you to subscribe to our Status Page for timely communication. The measures we're implementing will help ensure we continue to meet the standards you expect from us.

Questions or Concerns?

If you experienced any issues during this incident that have not been resolved, please don't hesitate to contact our support team. We're here to help and will be happy to provide additional information or assistance.

Thank you for your understanding and continued trust in our service as we continue to rise to the standards we expect from ourselves.

Sincerely,

The FireHydrant Team