Inbound queue lengths and mail flow appear to be healthy. The issue is resolved, and the implemented changes should prevent future surges from significantly delaying email.
Posted May 10, 2021 - 18:28 EDT
Monitoring
The team has been able to implement throughput improvements with an increase in mail processing speed. The queues are beginning to empty. The team will continue to monitor the queues through the end of the business day.
The new configuration will help reduce the chances of a future high traffic-related delay from occurring.
Posted May 10, 2021 - 16:25 EDT
Update
The team is working to process queues as quickly as possible. 100% more database capacity and 50% mail server capacity have been added. Timeouts have been adjusted to improve success rates.
Processing queues have improved over the last 30 minutes, but it will continue to take some effort to catch up during business hours.
We are continuing to work towards healthy queue metrics actively.
Posted May 10, 2021 - 15:41 EDT
Identified
Approximately 50% more capacity has been initiated to accelerate the draining of queues. The team is continuing to work on the issue actively.
Posted May 10, 2021 - 12:43 EDT
Investigating
Inbound queues are experiencing a surge. The team is investigating and increasing resources to mitigate the surge.
Posted May 10, 2021 - 11:23 EDT
This incident affected: CloudFilter and SafeSend Email Security.