Emergency maintenance: LD8 core router replacement
Scheduled Maintenance Report for Giganet (and M12 Solutions) Status Page
Completed
Sadly tonight's maintenance to revert back to the LD8 router has failed.
Through troubleshooting with vendor TAC, it appears as though the replacement router chassis we've received is faulty, and the vendor has recommended a further replacement.

This has been arranged with the vendor and they are in the process of dispatching this to the data centre urgently.

As per the previous update, we have reverted the configuration to provide LD8 services via our THN site, and we can see affected services back online.
The network remains 'at-risk' due to this, however we believe all customers will be online.

We do not plan any more disruptive works from now until we commence further emergency maintenance, which shall be advertised ahead of time.

This is truly exceptional, and we thank you for your patience.

Further updates prior to the next emergency maintenance window shall be communicated on our master incident. https://status.giga.net.uk/incidents/vpcsj8jctqh5
Posted Apr 03, 2021 - 02:37 BST
Update
Further escalations within Vendor TAC have been made.

We have re-implemented the DR procedure to fail services across to THN whilst this issue is ongoing. Therefore circuits affected should now be back online.

Next update within an hour.
Posted Apr 03, 2021 - 01:25 BST
Update
Maintenance is ongoing to implement the replacement router however this is taking longer than anticipated.

Customer connections routing via LD8 will currently be down. The DR workaround is disabled whilst we’re migrating back.

Vendor TAC is engaged and we’re working through the issue.

Further updates within the next hour.
Posted Apr 03, 2021 - 00:21 BST
In progress
Scheduled maintenance is currently in progress. We will provide updates as necessary.
Posted Apr 02, 2021 - 22:00 BST
Scheduled
Following our LD8 core router failure, we received a replacement router this afternoon.
(https://status.giga.net.uk/incidents/vpcsj8jctqh5 & https://status.giga.net.uk/incidents/ccqjtm4301ww)

We are now therefore in a position to advertise emergency maintenance to implement the new router, and migrate all services back onto the LD8 core router which have been affected.

Preparation of the new router shall take place prior to 22.00hrs, and from 22.00hrs, we shall begin the disruptive process of moving services to the LD8 router.
This will cause momentary outages for all customer services routing via or within LD8.
(Broadband customers will remain unaffected as these route via the alternate core site - THN ).
We believe that we can minimise customer disruption to under 30mins, however we advertise a large window for contingency.
Should the process to migrate services back to the LD8 router fail, we shall reimplement the DR plan as is currently configured.

We apologise for any inconvenience this may cause, however it's our priority to re-establish the core router ASAP to provide full performance and resiliency for the network.

Customers who experience any problems following this maintenance work, please contact us via our usual support methods, and in particular, please raise a ticket via our Help Centre: https://support.giga.net.uk/hc/en-gb/requests/new.
We shall be monitoring our ticketing systems more closely over the bank holiday weekend for all customer requests.
Naturally, any business customers with valid 24x7x365 coverage, please contact you usual 24x7 Giganet support numbers if you experience any critical support issues.
Posted Apr 02, 2021 - 19:12 BST
This scheduled maintenance affected: M12 Giganet - Internet Services (Giganet Core - Routing, Giganet Core - Broadband Routing (LNS), Giganet Core - IP Transit, Giganet Core - Peering, Giganet Core - MPLS PWANs, Giganet Core - Hosted Firewalls, Carrier - ELITE/IGNITE (Leased line)), M12 - Voice Services (M12 SIP Trunks - LD8 SBC), M12 Giganet - Core Applications (DNS02 Recursive Server, DNS03 Recursive Server, RADIUS02 Server, SMTP02 Relay Server, NTP02 Server), M12 Giganet - Data Centres & Points of Presence (Giganet Core - Equinix LD8), and Contacting us/ Tools/ Portals (Giganet - Partner Portal).