Major Migration Outages for NaviSite
For the second time this year, a data center migration following an acquisition has led to major customer downtime. Many customers of Alabanza say their sites were offline for up to 72 hours as they were migrated to a data center at NaviSite (NAVI), which acquired Alabanza in August. NaviSite was moving customer accounts from Alabanza’s main data center in Baltimore to a NaviSite facility in Andover, Mass. See the end of this article for an update.
Alabanza is one of the largest hosting reseller specialists, and customers say a large chunk of the company’s 175,000 sites have been offline. The migration problems for NaviSite follow a similar outage in late July when Hostway moved ValueWeb servers from Miami to Tampa. Hostway said later that more than 500 servers suffered hardware failures when they were restarted in the new facility.
Accounts posted by customers at Christian Web Host (a major Alabanza reseller) and Web Hosting Talk indicate that the move was planned to take place via a data transfer to hardware at Andover, with no physical relocation of servers. But network latency problems created problems with rsync (a popular file transfer tool) and NaviSite decided that it needed to physically move servers from Baltimore to Andover, a trip of about 420 miles.
The data transfer effort began at 5 am Eastern time on Friday, according to posts by Lance Odeneal at Christian Web Host’s customer information site. Some of the server transfers were completed using rsync, but others encountered latency problems, leading NaviSite to load servers on trucks for the seven-hour drive to Andover. By Sunday evening, frustration was evident.
“I am honestly just at a loss at this point as I would understand each of you being very upset at us,” Odeneal writes in a post to customers at 7:50 pm Sunday. “We have been put in a position with several other hundred hosts around the world that has each of our hands tied in this.” He reported at 6 am this morning that servers were coming back online.
UPDATE: As of 7:30 pm Monday, many sites remain offline. “”We are really hoping this is something we’ll be able to address by a very reasonable amount of time, if not by tonight, then by early tomorrow morning,” chief marketing officer Rathin Sinha told the Register. An Alabanza status page says Navisite has “over 90% of the server operating systems up and running in our Andover, MA datacenter,” but also says it is “continuing to work on the name servers to enable the availability of the individual hosts.” The page offers contact customers who remain offline, but says that “we anticipate that our updates will continue throughout the evening.”
I am another host with Alabanza and confirm these issues. It has been a true nightmare and one that will hopefully spark changes within Navisite towards a more structured focused when these types of datacenter moves are planned.
I’ve created a blog to update my clients and appreciate a nice article like this that summarizes the situation.
Navisite is headed into its 70th hour of this massive outage.
Word from people who are trying to deal with them is that they are totally lost and don’t know what to do.
They have just totally screwed over 1,000s of people and are destroying thousands of businesses.
We are Another Alabanza/NaviSite host effected by the poor management of this company. We have recieved hundreds of customer calls and other 75 account cancellations since the outage which started three days a go.
At least point we are looking for another host, That’s if we have a big enough client base to need another host. Also looking at legal options regarding NaviSite and their total incompetance!
Francis KoerberPosted November 5th, 2007
I am grateful to Alabanza. I had FREE webhosting since the late 90′s. I was the webmaster for four years and a devoted employee. However, FREE is not enough to keep me there after this. I migrated all my sites to a new hosting company yesterday.
We are one of many Christian Web Host customers who were down due to this mess. And it was a mess no doubt. What I do appreciate is that before, during and after the situation CWH kept customers up to date on what would happen (to the best that they knew)and what was happening and why. I appreciate that they relayed facts not anger and finger pointing and at all times maintained concern for their customers and even asked people to pray for those who had caused the mess. Thankfully things are getting back to normal and hopefully we can all function again. Downtime or no downtime I would still recommend CWH.
I have THREE sites hosted with Christian Web Host and while I appreciate their keeping me updated on the issue, I have three separate entities questioning me now as to why we moved our site to Christian Web Host. I realize this is not their fault, however that doesn’t change the fact that 2 of my sites (and corresponding email) was down through yesterday afternoon. They are both back down again. My company and my Prebytery have had no email for 3 of the last 4 days. Unacceptable!
I have had 7 out of 9 days without email and a site… I’m a reseller, albeit very small, with only about 55 customers. But this has completely messed up my business…
Thank God, most of my clients are completely understanding about this, once I’ve explained the scenario.. The frustrating thing to me about this has not been the outage itself… I understand that sometimes massive failures take place in the IT Industry. but NaviSite’s complete lack of useful communication with their customers has been unreal. They posted a letter from their CEO on their site…
but their customer service agents are completely unhelpful and ignorant about any information that could be useful to me… just terrible.