Skip to content
by Unity Nodes
Validator Incidents

Openshards Validator Outage Jan '23

1 posts252 views3 likesLast activity Jan 2023

  1. blazeOP 3

    The OpenShards pool experienced an outage on Jan 29th, 2023.

    Root cause: Network connectivity issue at its NYC data center.

    Fix applied: A failover was attempted on the backup node, but it had fallen behind with syncing. The primary node was then restarted and waited for headers to sync.

    Remediation: The backup node will be migrated to another provider, and Kuutamo will be set up to monitor sync and automated failover.

    We will lower our pool rate to 3% for one month to assist delegators with lost rewards.