(Fixed) Upstream API proxy service network connectivity – workaround

Our upstream provider has indicated there is a full scale outage in their Chicago data center. This affects our API and frontend proxy, but inbound mail is still working as normal. To work around the issue, the following endpoints can be used without HTTPS:

http://acai.mailsac.com

http://goji.mailsac.com

 

Status for our upstream provider can be monitored here (see Chicago):
https://www.vultr.com/status/

As always, the Mailsac status page will indicate when things return to normal:

https://status.mailsac.com

 

Postmortem

Beginning at 2018-07-17 10:39:17 Pacific, our upstream provider had a partial network outage in their Chicago data center. After determining that the API and UI proxy could not talk to the backend endpoints, we started the recovery plan of moving the proxy’s IP to a hot spare proxy. However due to maintenance blocks in our upstream provider’s system, we were unable to make changes to the IP allocations. After 11 minutes, network connectivity was restored.