London
Incident Report for Simwood
Resolved
We have continued to monitor the issue and have seen it remain stable for 2 hours. Further, we have received an update from the third party network that their issue was mitigated earlier. As such, we will be closing this incident.

The third party network advise they will be undertaking emergency engineering works later tonight which, whilst they should pose no risk to Simwood services, cause us to leave the existing voice re-routing in place. We will reroute traffic to the Volta site overnight (with no further update here) once external networks are confirmed stable.
Posted Jun 05, 2018 - 20:01 UTC
Monitoring
This has remained stable for an hour now but we are yet to receive the all clear from the third party network. We shall continue to monitor until we are happy this issue is resolved.

We are still seeing some calls being forced to the affected site by customers, despite our actions to re-route traffic away to the best of our ability by changing DNS. This will be due to misconfigured equipment using IP addresses directly or perhaps very long DNS caches.

Whilst this is beyond our control, we would again ask customers to follow our interop (viewable https://support.simwood.com/hc/en-us/articles/115008835528-Outbound-SIP) as fully as possible in order to minimise any disruption.

We're also aware that some customers may have experienced intermittent issues reaching anycasted services such as the portal or API where requests hit the London Volta site, these nodes were withdrawn from service during the period of peak instability but have now been restored.

Until we receive confirmation from the third party network that this incident is resolved, we will continue to leave voice traffic routed away from London as far as possible.
Posted Jun 05, 2018 - 18:40 UTC
Identified
This has been confirmed as a fault within a third party network, which is being investigated.

Traffic has been re-routed wherever possible, but we are continuing to see a small amount of calls on our London site. External monitoring to the published FQDNs, however, shows no issues therefore customers configured as per our published interop information should not be experiencing any issues.

We will continue to monitor the progress of this fault, and update this ticket as further information becomes available.
Posted Jun 05, 2018 - 16:24 UTC
Investigating
We are aware of issues affecting connectivity to our London (Volta) site. We believe this issue to be outwith the Simwood network, however are investigating this as a priority.

In the interim, voice traffic is being re-routed away from this site wherever possible. Customers configured per our published interop information should find outbound traffic already routing to our sites in Manchester and Slough.
Posted Jun 05, 2018 - 16:03 UTC
This incident affected: Availability Zones (London) and API and Portal.