NJ’s Data Centers Came Through Sandy Relatively Unscathed, NJTC Panelists Say, Part 2

A panel of IT providers, data center end users, utility representatives and others discussed lessons learned from Superstorm Sandy at the New Jersey Technology Council (NJTC) Data Center Summit in Livingston on Dec. 13, 2012. This is Part 2 of that story. Part one was published here.

At the summit, the CIO of global manufacturer Leviton Manufacturing Co. (Melville, N.Y.), Bob MacTaggert, shared his story about moving the company’s headquarters from Queens to Long Island. The location chosen was in a 170-mile-per-hour hurricane path, and “we thought it would be a good idea not to locate the data center in the corporate building,” he said.

After a search, Leviton found a colocation center in Cedar Knolls, N.J., at the intersection of the service areas of Jersey Central Power & Light, a subsidiary of First Energy (Akron, Ohio), and PSE&G (Newark). Explained MacTaggert, “We have two feeds coming in on both sides, plus adequate generation and a good location. For communications we have a redundant network globally.

“During Sandy, we didn’t receive any outages except to our Long Island location. Even though we had two totally diverse lines and we lost both of them, fortunately we had a third option and we were able to make that work. We put staff on-site and hunkered down.

“One of the problems we found was with the telephone service from Verizon, which ran out of capacity, and certain exchanges didn’t work on Long Island. Cellphone coverage problems occurred with both Verizon Wireless and AT&T Wireless as a result of problems making routing changes,” MacTaggert noted.

“Our plans for the future,” he said, “are to validate all the circuits between Long Island and our N.J. location, because sometimes service providers change [these paths], and you don’t really have the diversity you think you have.”

Roger Williams, director of national sales engineering at Comcast Business Class Services (Philadelphia), said Comcast had managed to keep most functions up and running for business clients.

“We support about 10 of the major healthcare organizations throughout the Philadelphia and N.J. areas. We had one site go down due to a fiber cut. A tree fell on an aerial strand. We were down for 24 hours and were back up.” He attributed this to Comcast’s diversity of cable locations — not having cable in predictable routes.

PSE&G’s case was made by Will Steffens, principal of Sugarloaf Associates (Hopatcong, N.J.), who had spent 34 years with PSE&G, interfacing with data centers and assisting in developing more than 8 million square feet of them in N.J.

Steffens said his former colleagues at PSE&G noted that “Irene was nothing compared to what Sandy did to the electrical infrastructure” in the state. PSE&G has a very good system overall, he said: “It’s very redundant and it’s very current. They spent a lot of money on it … but still it went down.”

Customers have little choice but to upgrade services to include even more redundancy, Steffens indicated. Quoting an industry rule of thumb, he said the larger the data center, the better its redundancy and viability.

Smaller data centers on the PSE&G 13 kV secondary service have only one line. “So you are hanging on one line, and that may be a very uncomfortable place to be,” noted Steffens.

Customers who go up one level in service have the option of buying a second line, but that is costly because “from the utility’s point of view, you are entitled to only one” line. Users have to think about the costs and benefits of each option, Steffens said.

He continued, “If your data center is sitting on a line that serves a sewer/water treatment plant or a hospital, you are sitting pretty,” because those are priority customers. But most customers won’t be able to tell where their center sits on the grid.

Rob Stephenson, chief marketing officer at Xand (Hawthorne, N.Y.), a private data center, managed-service, redundancy and cloud firm that operates in the Northeast, said, “Our learning process really started with Irene.” He added, “During Sandy … we had operations on full alert and everyone out in the trenches. Some of our data centers were closer to the path of the storm. We put on proactive [power] generation.”

“We lost commercial power in three of the six facilities, but we never lost a customer packet,” Stephenson said. Internet connectivity and power and cooling stayed up because Xand was on full alert and its facilities are located 30 to 100 miles outside major metropolitan areas like Philadelphia, Boston and N.Y., he stated.