Service Status Update: December 18, 2024

Summary

On Dec 18, 2024 at 17:55:14 UTC, we experienced an outage that caused our web UI to fail to load for all users for 6 minutes. The incident has been fully resolved, and all systems are operating normally. We sincerely apologize for any disruption this may have caused your operations.

Slack and MS Teams functionality was not impacted.

Signals notification deliverability was not impacted. 

What Happened 

While debugging an issue we found in our API, we misconfigured a network certificate, which caused the certificate to fail to load. The old certificate was not available during the transition. As a result, users were unable to load the web UI and declare incidents. 

Customer impact

• Customers were unable to load the FireHydrant website and declare incidents for 6 minutes

No customer data was lost or compromised during this incident. 

Users were able to declare and manage incidents with no issue on Slack and MS Teams.

All Signals notifications were delivered as expected with no delays.

Resolution 

We quickly rolled back the misconfigured certificate and restored service. 

All systems were returned to normal operations by 18:03:08 UTC.

We have scheduled a planned maintenance window tonight at 05:00 UTC.

Commitment to Reliability

We understand that you rely on our services for your critical operations. We are committed to maintaining the highest levels of service reliability and transparency, and encourage you to subscribe to our Status Page for timely communication. The measures we're implementing will help ensure we continue to meet the standards you expect from us.

Questions or Concerns?

If you experienced any issues during this incident that have not been resolved, please don't hesitate to contact our support team. We're here to help and will be happy to provide additional information or assistance.

Thank you for your understanding and continued trust in our service as we continue to rise to the standards we expect from ourselves.

Sincerely,

The FireHydrant Team