Following on from tonight’s planned maintenance work (https://status.giga.net.uk/incidents/ccqjtm4301ww
), as recommended by our Vendor TAC, our core LD8 router has failed upon the restart procedure. Currently our LD8 core router is down as a result, including most (but not all) services hosted here.
Customers with services routing via our Telehouse North site will be unaffected and those with managed backup/failover will be operating via these circuits.
Engaging with Vendor TAC has recommended a hardware replacement of our LD8 core router as all attempts so far to recover the router have failed.
Although we have a mission critical 4hr onsite SLA hardware replacement with this device, and we initiated the request for support shortly after 2am, our support partner are struggling to provide a replacement router with this 4hr contracted timeframe.
As such, we are mitigating the disruption as much as possible by reconfiguring and rerouting traffic including NNIs across our WDM ring to our THN POP. This is invoking our DR scenario.
The time it takes to invoke this DR scenario and restore services will be determined on the time it takes to reconfigure services across this WDM link to our THN router. We therefore expect customers whose services terminate in LD8 to potentially be experiencing an outage until late morning Thursday 1st April.
We are doing everything we can to bring this forward.
We have escalated the hardware replacement with our support vendor to provide a replacement router ASAP and will be making further such escalations in the coming hours.
Services are gradually being restored including DBX hosted voice customers.
Leased line NNIs will follow.
All broadband connections remain unaffected as these route via Telehouse North, our other core data centre.
The entire network is ‘at-risk’ due to this outage.
We will continue to post updates here as our DR plan progresses and Vendor TAC update us.
We apologise for any inconvenience this causes.