This has been stable for over an hour now so we're marking this incident resolved. The issue was Apache2 hanging on a single container on a single host. As the API is anycasted by many containers on many hosts around the network, this only affected a small percentage of traffic from certain locations. Had Apache failed gracefully, others would have seamlessly taken the load and we'll continue investigating why it didn't offline.
Posted about 1 year ago. Sep 21, 2017 - 12:44 UTC
A fix has been implemented and we are monitoring the results.
Posted about 1 year ago. Sep 21, 2017 - 11:22 UTC
Our external monitoring is indicating intermittent API connectivity from certain locations. We're investigating.