General slowness of applications and email delivery
We have completed a major overhaul of our mail storage subsystem, yielding extremely good results. This incident will be closed.
Last evening around 5:15pm our storage team was investigating and performed a failover operation that was supposed to be seamless. Some Mailstores lost access and we needed to recover them. There was about 20-25 minutes for recovery of all servers. All systems are operating normally at this time.
There was an issue around 5:20PM today which caused email services to be briefly unavailable. All services are fully functional at this time.
We are monitoring systems for slowness today. We will be engaging with our storage vendor to complete an emergency update of the storage firmware. The vendor believes that the firmware is directly related to this issue. Notification will be forthcoming through email.
Maintenance is complete for tonight. All services are back online. Storage firmware upgrade did not complete per manufacture's instruction. We have contacted the vendor and will continue to monitor system performance in the meantime.
Planned downtime is underway.
As of 12:30PM, performance has been restored to all aspects of BlueTie systems. We are reviewing options and may communicate an emergency maintenance window for this evening.
We are currently investigating an issue which is causing general slowness of the BlueTie Webmail, POP and IMAP services. Inbound email delivery is also impacted. We believe this to be storage subsystem related.
This incident affected: BlueTie Proprietary Solutions (Webmail, POP-IMAP-SMTP, Inbound Email Delivery).