I-3 hosted email – Update

0810 PDT / 1610 BST – Our engineers have confirmed that the restoration of the disk subsystems will take longer than initially anticipated, to resynchronise all of the user data. The latest estimates for full restoration of services to I-3 hosted email systems is 1300 PDT / 2100 BST. This issue does not impact the MaxMail filtering systems. We will post further updates as they become available. Users of the hosted mail (i-3) system can use the message continuity feature within the MaxMail filtering systems, to access and send emails during this maintenance window.

I-3 service maintenance

0510 PDT / 1310 BST – Our engineers are currently troubleshooting a recurrence of the issue with the disk subsystem on the I-3 mail server, which is used for their email hosting by a small number of MAX Mail customers.

The http://mail.i-3.com service and control panel access will be unavailable from 0800 EDT/1300 BST on 23/05/2016 until 1200 EDT / 1700 BST on 23/05/2016 for essential maintenance.

During this time, I-3 mail customers will not be able to send or receive mail through the I-3 mail server directly, but no messages will be lost, and those customers can use the continuity feature within the MAX Mail control panel (http://mail.maxfocus.com) to both review any incoming messages in queue and to send new messages as necessary. We will update the status here with any updates as they become available.

Resolution of Networking Issue

10:30am Pacific / 6:30pm GMT – The networking issue within the Amsterdam datacenter has been largely resolved, and our engineers have updated the routing within that datacenter to circumvent any remaining issues.  Our monitoring shows that message processing has returned to normal within that facility; we will continue to monitor and will take any further action necessary.  Other datacenters were unaffected by the issue.

Networking Issue at Amsterdam Datacenter

10:00am Pacific / 6:00pm GMT – Our monitoring has detected a networking issue within the Amsterdam datacenter that may be impacting message processing within that facility.  That datacenter is used primarily for backup (lower MX priority) traffic, but some primary traffic is also routed through that datacenter.  We are in contact with the networking team at that datacenter, which is working to resolve the issue.  We will post further updates here.  Other datacenters are unaffected.

Resolution of Message Processing

6:30am Pacific / 2:15pm BST — An issue seen earlier this morning with a third-party partner has been resolved, and all datacenters are now processing mail normally.  We are working with the partner to determine the root cause, and we are also working on implementing better means of mitigating the risk of any future operational issues with this partner.

Update on Message Processing

1:00 PM Pacific / 9:00 PM BST —  General inbound and outbound message processing continues to be stable and operating at normal levels in all datacenters.   We are seeing some unusual behavior with journaling messages specifically from Office 365 to some of our systems for the archive; our engineers are currently investigating this.

Update on Message Processing

7:45pm Pacific /  3:45 AM BST —  Message processing is operating normally in both the Amsterdam and California datacenters, and has been stable for the last several hours.  Our engineers have identified a primary culprit and fix for the issues in Amsterdam, and are in the process of implementing additional systems in California to mitigate the risk of a recurrence.  We will continue to monitor and will post any significant updates here.