Inbound email delivery delay
Incident Report for Mailprotector
Postmortem

An organization sent a poorly constructed email just before 8:00 AM ET on Tue, Oct 2, with over 2,000 email addresses in the CC field. The header of the email was too long to process as expected, creating a unique situation that contributed to a large amount of resource utilization.

Once the team identified the email message, custom handling for this poorly formed email was built. This allowed the message to be processed as expected without increasing resources. Future incidents with poorly constructed emails that exceed typical header data will be handled gracefully.

Posted Oct 02, 2024 - 11:16 EDT

Resolved
Mail flow has remained healthy, and email delivery performance is resolved.
Posted Oct 02, 2024 - 10:52 EDT
Monitoring
Mail flow has been restored to regular operation. The team is continuing to monitor the situation and ensure a complete resolution.
Posted Oct 01, 2024 - 14:51 EDT
Identified
The team has identified the issue, and mail flow performance has improved. Email delivery rate is just under one minute and is continuing to improve.
Posted Oct 01, 2024 - 13:59 EDT
Update
Some emails may be delayed over 30 minutes. The team is investigating the issue.
Posted Oct 01, 2024 - 10:14 EDT
Investigating
Email delivery delays have been observed on Shield, and the team is investigating the issue. At this time, the delays are over 15 minutes.
Posted Oct 01, 2024 - 09:45 EDT
This incident affected: Shield.