Yesterday, we announced our plans to add some major upgrades to our server infrastructure for our LightCMS, WaterCMS, SkyCMS and Icebrrg products. Today, we initiated the upgrade process but, due to some complications, we decided to postpone the upgrades and revert to our prior hosting setup.
What happened?
We ran into some network routing complications that couldn't be adequately addressed within the confines of our maintenance window. As a result, we decided to postpone the upgrades. As soon as that decision was made, we reverted to our previous hosting setup as quickly as possible in order to minimize downtime.
How long was the system down?
Though we had scheduled a six hour maintenance window to allow for this contingency, the system was only down a total of one hour and fifteen minutes. This downtime was spread out over the period of 1:30am to 3:15am CDT. For approximately thirty minutes of what we are calling "downtime," the system was actually up and running with only a few pieces of functionality unresponsive.
What's the plan now?
For now, everything has been restored to full functionality exactly as it was before the upgrade attempt. It is still our plan to perform these upgrades, and we are tentatively rescheduling the upgrades for approximately one month from now, or sometime around mid-October. We will, of course, provide more specifics as the time approaches.
Why wait so long?
One reason is so that we can fully investigate the issues we encountered before making a new attempt. The other reason is to avoid interfering with some additional upgrades we have planned to roll out on the system in the next few weeks. There are some exciting things in the works that we'll be sharing more about hopefully next week.
Is your current hosting infrastructure sufficient?
Yes. We have more than enough server headroom to accommodate our current traffic loads and even multiples of them. While we are very committed to accomplishing these new server upgrades, we do not consider them essential to our service at this point. That said, we are constantly monitoring and working to improve our system and we will respond as needed to any issues that may arise.
Thank you for your support
Thanks to all of you for your support. We received several positive emails in reply to yesterday's announcement of our upgrades and we appreciate that greatly. Rest assured, these scaling plans are still in place. We look forward to continuing to serve as your online partner.
Posted on
Sat, September 19, 2009
by Tim Wall
filed under