NoVPS - Errors in cross-node communication in private clusters – Incident details

Errors in cross-node communication in private clusters

ResolvedPartial outage 30%7 hours 16 minutes

Affected

Public clusters

Operational from 6:30 PM to 8:13 PM, Partial outage from 8:13 PM to 9:56 PM, Operational from 9:56 PM to 1:46 AM

Private clusters

Operational from 6:30 PM to 8:13 PM, Major outage from 8:13 PM to 9:56 PM, Partial outage from 9:56 PM to 1:46 AM

  • ResolvedUTC
    This incident has been resolved.
  • UpdateUTC

    The public cluster has recovered, though we are still seeing an elevated number of errors in some public clusters.

  • MonitoringUTC

    We implemented a fix and are currently monitoring the result.

    A network configuration error on our provider's end caused a VLAN overlap.

    We have reconfigured the cluster subnets to prevent similar issues in the future.

  • IdentifiedUTC
    We are continuing to work on a fix for this incident.
  • InvestigatingUTC
    We are currently investigating this incident.