Database Issue

Incident Report for Ontraport

Postmortem

At approximately 08:58 PST, one of our database servers experienced an unexpected restart event. This resulted in an isolated number of accounts losing some account functionality. The infrastructure team was alerted immediately to the loss and began standard recovery procedures, which resulted in services being restored fully by 09:28 PST.

The nature of the event, while understood, has a yet unknown cause. Issue tickets with the hardware vendor have been opened and we will continue to investigate the root cause. Logs, monitoring, and hardware events do not indicate the source of the restart; as such, this will be continued investigation until we are satisfied that we understand the full sequence of events.

Posted Aug 10, 2022 - 10:00 PDT

Resolved

The systems and services look like they are working normally, and no additional errors are being observed or reported.
Posted Aug 10, 2022 - 09:56 PDT

Monitoring

Services have been restored, we will continue to monitor and address any lingering issues as they appear or are identified.
Posted Aug 10, 2022 - 09:36 PDT

Identified

We have identified the issue and are verifying fix.
Posted Aug 10, 2022 - 09:19 PDT

Investigating

We are currently investigating a database issue that may result in account/service disruption for some accounts.
Posted Aug 10, 2022 - 09:08 PDT
This incident affected: Isolated (Isolated Accounts).