Mail Flow Problem
Incident Report for Mailprotector

A code change occurred during normal operations that introduced a bug affecting the perimeter servers' routing tables, causing addresses for active users to be removed. The issue was identified by our internal monitoring systems immediately, and our operations team was notified and rolled back the change. A repair procedure was then run on the routing tables. Upon completion, all services returned to normal operation, and the incident was resolved.

All inbound and outbound emails through the perimeter were rejected during the initial part of the incident. No emails were queued; therefore, senders will need to resend any emails that were rejected. The incident lasted 27 minutes from origin to complete resolution, but not all users were affected the entire time.

Email flow is critical to most businesses, and we understand the immediate impact that can have on you and your clients. This incident was caused by human error, and the team is reviewing process to prevent a similar occurrence. We appreciate the quick feedback from all partners that contacted us. Thank you.

Posted Aug 22, 2018 - 12:24 EDT

Mail flow continues to operate as expected. Thank you.
Posted Aug 21, 2018 - 17:54 EDT
The routing table problem has been fixed. Email is flowing through the perimeter as expected.

We will continue to monitor the situation.
Posted Aug 21, 2018 - 17:04 EDT
A routing table problem has caused email flow to stop at the perimeter. The repair is underway and we expect mail flow to be back shortly.
Posted Aug 21, 2018 - 16:58 EDT
We are investigating a mail flow problem. An update on the situation will be posted as soon as possible.
Posted Aug 21, 2018 - 16:52 EDT
This incident affected: CloudFilter and SafeSend Email Security and Service Announcements.