Network Operations Centers (NOCs) are central locations from which an organization supports its computer network and telecom infrastructure, detects and resolves IT infrastructure incidents, and ultimately ensures data center availability. Sometimes they lie within the data center, sometimes externally. They are typically connected to a high-speed internet connection or directly to the internet backbone in order to provide as much bandwidth as possible.
But regardless of the arrangement, the aim is to maintain 24x7 network and data center availability. That goal can always be achieved—if money is no object. But problems can enter the picture when organizations attempt to achieve it in an affordable fashion.
This article reviews the functions of the NOC, as well as the various external and internal relationships that can exist between data centers and NOCs. Further, it offers data center managers various tips on how to better utilize internal and external IT resources in order to slash costs, improve staff efficiency, reduce resolution time and enhance the end-user experience.
NOC vs. the Data Center
If you wanted to converse with Voltaire, he demanded you define your terms. So let’s start by more clearly defining what a NOC is, how it relates to the data center and what the core functions are. Part of the challenge is that it depends on who you ask.
Avishai Shafir, director of product marketing at security policy orchestration provider Tufin, says data centers are there to host and manage the organization’s computing resources such as servers, Virtual Machines (VMs), storage and databases. They may also deal with some portion of security and network controls. Some are small and are part of the organization facility while others are dedicated centers to manage assets such as hardware, cooling, electricity and physical security in an efficient manner.
So how does that differ from a NOC? Shafir says a NOC is more of a management control center. The NOC manages infrastructure and procedural changes, events, customer calls, security, quality control and assurance, monitoring tools, ticketing systems, integration with customer tools, reporting and dashboards, and more. Just as important as the technology that lies within the NOC, however, is the operations and the support team required to deliver high-quality support.
“Many dedicated data centers have an internal NOC room to manage, monitor and fix IT assets and the network,” says Shafir.
The NOC size (room and number of engineers) is typically driven from the size of the data center and its business criticality. Not all data centers, he adds, have a NOC room with a 24/7 team. Smaller data centers often use automated monitoring and resolution software in place of a NOC in an attempt to deal with those functions without much human intervention.
Shafir believes managed service providers (MSPs) and telecommunication companies should manage a NOC within the data center, as the network, application, and data center availability are critical. Similarly, larger enterprises with sizable IT teams can probably afford to run their own NOC and keep costs at a manageable level. But those with smaller IT teams might be better served by outsourcing the NOC.
“The ultimate decision should be based on IT team size, the level of business criticality of the network, and the degree of sensitivity of the managed data or networks,” says Shafir. “Another important point is regulation—industries such as utilities are required to have their own internal NOC.”
Atchison Frazer, who at the time of the interview was chief marketing officer of performance data and analysis firm Xangati (he's now at Talari Networks), has a somewhat different take. He says data centers typically are intra-enterprise though they can be dispersed geographically based on user demand. NOCs, on the other hand, are usually an aggregation of multiple data centers, physical and virtual, that can at least be monitored and in some cases remotely managed from one uber data center.
“NOCs are widely used in network security, defense/spy/homeland security agencies, unified communications/digital video (physical security) and managed/hosted/cloud service providers,” says Frazer.
INOC, a company that operates a NOC which provides monitoring and management services to data centers and colocation facilities, considers data centers to be buildings that host servers, storage, network and telecommunications equipment; they also are, typically, the interconnection points between various carrier networks. The facilities are designed with extensive power, cooling, security and network considerations.
Peter Prosen, director of network operations at INOC, differentiates this definition of the data center from that of the NOC. He says that NOCs are control centers that are operated by 24x7 technical staff to continuously monitor and handle incidents related to the health and performance of infrastructure belonging to data centers, enterprises, service providers and other entities.
“A data center almost always has a 24x7 support operation of some sort, at a minimum to provide physical access to clients and to be the on-demand smart hands support for cabling and equipment troubleshooting,” says Prosen.
That leaves data center managers with the choice of operating an internal NOC or outsourcing those functions to a NOC service provider. The first option would require the data center to extend the internal functionality of its existing 24x7 operation so that it includes infrastructure monitoring.
Prosen says that a data center might elect to go internal if it has a relatively simple technology environment consisting of a limited number of equipment vendors as well as relatively repetitive configurations and tasks. Additionally, this would be the right choice if there is enough work to keep the staff busy. Prosen cautions data centers managers that maintaining their own NOC almost always requires at least two people to be present at all times. This, of course, is costly in terms of personnel alone as you have to have shifts running round the clock.
Data center managers have to take into account that it may not be worth it to run a NOC unless there is enough work to keep personnel busy. If that is the case, it might be better to outsource the NOC. Additional barriers on adding a NOC internally can sometimes be the hefty price tag often associated with infrastructure management tools. If the data center doesn’t already have them, it may be cheaper to obtain these management functions externally. Similarly, a scarcity of internal talent can change the cost equation – the hiring of experienced NOC staff won’t be cheap.
“Outsourcing a NOC takes away the need to own and maintain software tools, hire, train and manage additional staff, and develop workflows,” says Prosen. “Choose to outsource if you don’t have enough work to keep staff busy round the clock, if the technologies and vendors being managed are complex and numerous, if hiring and maintaining a staff is difficult in your particular area, or management does not have experience running a NOC.”
Obviously, the smart data center manager will review total cost of ownership (TCO) for the internal and external NOC scenarios before finalizing any decisions. The results tend to be clear cut in many cases. Outsourcing of the NOC is the way to go if the cost of implementing or expanding one internally would be far more than passing the functions onto others. NOC providers, says Prosen, can usually take advantage of economies of scale by using the same equipment and staff to serve multiple organizations. Similarly, they may be able to provide a big reduction in telecom costs due to the greater availability of connectivity options and carriers at their disposal.
Containing Internal NOC Costs
But how about those who decide to maintain all of their NOC functions internally? Prosen says a good way for data centers to save on internal NOC costs is by reducing the number of alarms or incidents that occur on the infrastructure. This means having a well-designed redundant and resilient infrastructure. A key element of that is infrastructure management.
“You can reduce costs by having a good set of infrastructure management tools that can correlate events, automate certain tasks and provide visibility into network health,” says Prosen.
Frazer of Xangati recommended integrating high-performance analytics tools within the internal data center NOC so that service assurance is measurable and degradation or bottlenecks can be avoided. Done right, this can have a positive impact on costs. In addition, such tools contain capacity utilization metrics which can help the data center manager in rightsizing of assets and forward planning. This is an important element in the avoidance of the common practice of hardware over-purchase.
“End-user experience is what it’s all about, but the more complicated the data center (hybrid-cloud architecture, for example) the more difficult it is to gain visibility into the end-user’s computing environment,” says Frazer.
Xangati provides real-time data peering into end-user desktops to monitor, capture, record and analyze every interaction between applications, VM’s and every virtual and physical component of the data center. Over 300 enterprises, government agencies, healthcare organizations, educational systems and cloud providers use Xangati including the likes of Comcast, British Gas, Univita Health, Harvard University and the U.S. Army.
Shafir of Tufin also talks about curtailing NOC expenditures by utilizing the right tools. There are many monitoring tools, for example, as well as problem resolution software that can be used to ensure smooth operation. This reduces costs as there is less need for people, and the time to understand the issues and solves them is faster.
Take the case of monitoring and management of security controls such as firewalls. Some data centers have hundreds of firewalls, each managed by a set of rules, and these firewalls require updating on a daily basis. Any change in one of the thousands of rules in each firewall can lead to lack connectivity or application downtime – and that drives up overall costs.
“The NOC team needs the right set of tools to monitor and resolve firewall issues, as understanding that the root cause of a problem is due to a security change,” says Shafir. “In addition the NOC team needs to ensure that fixes to firewall settings doesn’t open security holes that might enable vulnerabilities into the enterprise network.”
Take Your Time
Clearly a solid case can be made in favor of outsourcing the NOC. Equally, there can be much in favor of retaining those functions internally. It is up the data center manager to take everything into consideration before any final decision is made.
The most important element is the bottom line. Will it ultimately be cheaper to outsource or not? To keep things on an even playing field, dig into the details of the cost equation to see just how much each option will cost.
But regardless of which path is chosen, whether internal or external, it costs money to put a NOC there and provide high-quality services. Therefore, be in no hurry to rush into things. Equally, consider the financial implications of not having adequate NOC services available. Without them, data center availability rates can plummet and data center staff resources may be consumed in putting out the inevitable fires that will result.
“Look beyond the up-front or recurring expenses to see what value or enablement you are getting for what you spend,” says Greg Schulz, an analyst with Server and StorageIO Group.
He challenges data center managers to ask themselves if they are shifting or deferring costs, hiding or masking or moving a problem, or are they unlocking some value and enabling resiliency. Instead of looking at it only as a cost exercise, seek to know how it may compliment and enable the environment.
“Start thinking of NOCs from a strategic business asset perspective as opposed to seeing them as a cost-overhead,” says Schulz.
Drew Robb is a freelance writer based in Florida.