NJ’s Data Centers Came Through Sandy Relatively Unscathed, NJTC Panelists Say

A panel of IT providers, data center end users, utility representatives and others discussed lessons learned from Superstorm Sandy at the New Jersey Technology Council (NJTC) Data Center Summit in Livingston on Dec. 13, 2012. This is Part 1 of that story.

Superstorm Sandy was on everyone’s mind at the data center summit.

Chatting with exhibitors, NJTechWeekly.com heard horror stories, some focused on customer locations and others about clients in negotiations to move their data to a facility, losing power or sustaining flooding that wiped out their on premisis equipment.

In general, however, the N.J. data centers themselves switched over to generators and didn’t flood.

Speaking to the audience earlier in the day, Eric Shepcaro, Telx (New York) CEO, said there was a big difference between the experiences of his company’s data centers in N.J. and those in N.Y.

“In New Jersey we were on generator power for a day; in New York, for six days,” he explained. “The customers didn’t have any downtime, however. However, a lot of our customers were affected, since they didn’t have a sufficient architecture plan. While our data centers were up and running, they all had remote users who couldn’t access the data center.”

Rashaad Bajwa, president and CEO of Domain Computer Services (Cranbury), moderated the panel discussion “Hurricane Sandy: A Discussion on Lessons Learned.” He said, “Our belief is that Sandy was a complete game changer for the data center industry … what makes this very different from [Hurricane] Irene, or 9/11, or the blackout of 2003, is that everyone [in New York and New Jersey] was impacted. There wasn’t anyone who didn’t … lose power, Internet or access to their space either at home or at work.”

Domain vets data center choices for customers. “We got a lot of what we did ‘before’ right, so we were able to be a hero for our customers,” Bajwa said, but there were a lot of lessons learned.

Later, during a discussion on voice communications, Bajwa said that while the data centers were up and running and customer data was protected, nearly 90 percent of his clients were affected by Sandy. Voice calls and email, then critical, were often down for a long time. “Internet and voice is their window to the outside world,” he said.

Atlantic Health System’s director of IT and operations, Pat Zinno, spoke about how the healthcare network’s 70 N.J. locations were affected by Sandy.

Atlantic’s Sandy story really started with Irene, he said: “Fortunately, from what we learned from Irene, our data centers and hospitals never lost connection to any of the systems. We had a lot of small practices that lost power — telephone lines down — but our mission-critical facilities stayed up and running through the entire storm.”

Zinno said that during Irene, when telephone service was brought to a halt, he had learned that all the lines terminated at Halsey Street in Newark, which was vulnerable. After Irene, “we sat down with Verizon and virtually eliminated Halsey Street from being part of the solution. … Our trunks today run through upstate New York and Philadelphia,” he said. Verizon and Atlantic participate in a disaster simulation — during which Verizon pulls the plug on some of the phone network’s capabilities to make sure it “fails over” properly — quarterly.

“The biggest takeaway for us was the business continuity aspect of our organization. It’s bigger than just disaster recovery for the IT department. It’s thinking about where people are going to go. Our corporate office didn’t open for five days, and there wasn’t a telephone pole left on the street,” Zinno noted. Atlantic had to move employees’ phones to different buildings, and it was all done on an ad hoc basis, he said.

Planning will be the key to overcoming this problem in the future, added Zinno: “We are sitting down with facilities people. We can move call centers around, but they’ve got to tell us where they are going to go,” and whether people will be working from home or another building.