Cisco OTV: Virtually Spanning Data Centers


In the beginning, there were text files. As the Internet evolved, the files moving across the network grew in size – first photos, then videos, then widgets. And now, with virtualization transforming many IT departments, virtual machines loom as the next payload networks must confront.

On Monday Cisco Systems (CSCO) will officially introduce a new technology that will make it easier to seamlessly move groups of virtual machines between data centers.

Cisco says its Overlay Transport Virtualization (OTV) solves many of the challenges that have made it difficult to shift large workloads between facilities, potentially opening new frontiers in disaster recovery, data center consolidation, and even energy management.

Tunneling Over IP
OTV is a new feature of the Nexus OS operating system that encapsulates Layer 2 Ethernet traffic within IP packets, allowing Ethernet traffic from a local area network (LAN) to be tunneled over an IP network to create a “logical data center” spanning several data centers in different locations. OTV technology will be supported in Cisco’s Nexus 7000 in April 2010, and existing Nexus customers can deploy OTV through a software upgrade.

Cisco says its overlay approach makes OTV easier to implement than using a dark fiber route or MultiProtocol Label Switching (MPLS) over IP to move workloads between facilities.

“This way the server team can move virtual servers and the network team doesn’t have to reinvent the wheel,” said Craig Huitenga, director of marketing for Cisco’s data center solutions.

Here are some details that Cisco is expected to emphasize in its announcement:

  • Operational Simplicity: Since OTV is an overlay technology, it does not require a network redesign to deploy. “With just four commands per site required, OTV can be enabled in a matter of minutes over existing networks,” Cisco says. “Adding a new data center to the OTV domain is simple, configuration is only required at the new location, OTV automatically synchronizes with all other sites.”
  • Transport-independent: OTV is a MAC routing scheme. Ethernet frames are encapsulated in IP packets and transported over any network that supports IP, therefore, OTV can be deployed over the Internet, private IP network or MPLS.
  • Increased Resiliency: Cisco says it has built resiliency features into OTV, which are automatically enabled when OTV is configured. These include multi-pathing, multi-homing and loop prevention. OTV also automatically suppresses flooding of unknown Layer 2 traffic. It says these features ensure that failures (such as broadcast storms or spanning-tree loops) in one data center are contained and do not propagate to other data centers.

“Moving workloads between data centers has typically involved complex and time-consuming network design and configurations,” said Ben Matheson, senior director, global partner marketing, VMware. “VMware VMotion can now leverage Cisco OTV to easily and cost-effectively move data center workloads across long distances, providing customers with resource flexibility and workload portability that span across geographically dispersed data centers.

“This represents a significant advancement for virtualized environments by simplifying and accelerating long-distance workload migrations,” Matheson added.

Terremark An Early Adopter
Among the early adopters of OTV is managed hosting and colocation specialist Terremark Worldwide, a key partner for both Cisco and VMware. Michael Duckett, Terremark’s General Manager of Network Services, says OTV has “already proven to be beneficial for the multi-site deployments of our Enterprise Cloud and the delivery of our cloud-based disaster recovery services.”

“After extensive testing, Cisco OTV appears to be a groundbreaking technology for data center interconnect, and could be a very significant contribution to the industry,” said  Duckett. “With OTV, we are able to more easily manage virtualization and cluster domains beyond a single data center, enable workload mobility between data centers, optimize compute resources across data centers, and help ensure business continuity by distributing applications and resources.”

Telenor Consolidates Data Centers
Another OTV customer is Norwegian telecom provider Telenor Group, which is planning to use the new technology to consolidate 22 data centers in Norway down to four large data centers, which will operate as one large logical data center.

OTV’s potential benefits extend beyond disaster recovery and server consoldiation scenarios. As we’ve previously noted, the ability to seamlessly shift workloads between data centers creates energy management possibilities, including a “follow the moon” strategy which takes advantage of lower costs for power and cooling during overnight hours. In this scenario, virtualized workloads are shifted across data centers in different time zones to capture savings from off-peak utility rates.

One likely point of debate: OTV is a Cisco technology that extends an existing standard, which is likely to be noted by rivals in the highly-competitive Battle for the Data Center. Cisco says it expects to eventually take OTV to standards bodies.

Get Daily Email News from DCK!
Subscribe now and get our special report, "The World's Most Unique Data Centers."

Enter your email to receive messages about offerings by Penton, its brands, affiliates and/or third-party partners, consistent with Penton's Privacy Policy.

About the Author

Rich Miller is the founder and editor at large of Data Center Knowledge, and has been reporting on the data center sector since 2000. He has tracked the growing impact of high-density computing on the power and cooling of data centers, and the resulting push for improved energy efficiency in these facilities.

Add Your Comments

  • (will not be published)


  1. This has some legs... I just blogged about it as well. From an operational perspective, does this mean if one DC is up, then all are up?

  2. A new technology? Try proprietary technology that isolates the company that endorses this method of tunneling. MPLS Pseudowires have provided this functionality for 10 years. If there is an IP core in the middle, MPLS can be transported on GRE to provide this today on a cross vendor network. However, it should be noted that creating layer 2 extensions between geographic disparate sites has it's own set of issues (ie. traffic engineering, a false sense of host to host latencies, etc).

  3. Mark: So, with OTV we are giving you L2 adjacency between your data centers--in terms of overall survivability, this is one piece of the puzzle--it certainly helps alot but you still need to make sure your storage architecture, app architecture, etc can help you take advantage of it. Truman: So, OTV is not tunneling, it is encapsulation. This is a useful distinction because it is one of the reasons OTV can reduce complexity in DCI scenarios. While there are a number of options for DCI out there, including ones offered by Cisco, our customers tell us they have shortcomings which OTV strives to address. No protocol is perfect and every customer is different, so OTV provides a another option, for some it will be a good fit and for others it will not, but at the end of the day, giving customers options is, I think you will agree, a good thing. To that end, we have published a doc to help customers understand the differences between OTV and VPLS: Your last observation is spot on--any kind of LAN extension still requires the data center operations team to do their due diligence. Even with OTV, as with any technology, customers still need to go through a traffic engineering exercise to ensure the DCI is offering the operational characteristics required. The nice thing about OTV is that it does not entail having to deal with network performance degradation, increased processing overhead, or markedly greater management and design complexity. Regards, Omar Cisco Systems

  4. Victor Moreno

    The newness in the technology is in allowing layer 2 extensibility that preserves the multi-homing, multi-pathing, failure isolation and flood handling characteristics of an L3 transport. Basically, layer 2 connectivity with the attributes of an L3 connection. This is NOT achieved by VPLS/EoMPLS pseudo wires today. In addition, OTV provides a much simplified configuration model and is transparent to the existing network which makes it non-disruptive and very easy to deploy. Again, MPLS PWs don't provide a simple deployment model. What you get is a true overlay non-disruptive solution with OTV, vs. a network redesign project with traditional pseudo wires. Many of the elements in OTV are standards based already and work is in progress to standardize the overall solution. In the mean time, the overlay nature of the solution keeps it from being isolational. OTV traffic can leverage existing transports, MPLS or other.

  5. Thanks Omar, while I agree that in implementation the procedure may be simple, the mechanics behind OTV remain somewhat of an abstraction of a rather complicated nature; a nature that has for many years burned networks. For example, the fact that OTV has a difference in signaling properties on unicast vs. multicast-enabled networks. This functionality seems rather superfluous in that if we *simply* extended a layer 2 network (by MPLS/VPLS/PWE3/dark fiber/etc) we would be able to handle any type of ethernet frames regardless of unicast vs. multicast traffic types. This also begs the question on how MLD/IPv6 multicast is handled by OTV? Secondly, I would cringe at the concept of using proxy arp between sites as a feature, and yet this is the recommended method in connecting DCs. Verifying true reachability (at a layer 2 basis) is abstracted by networking methods that have caused worlds of pain for the past 15 years. While MPLS technologies have their faults, when you encapsulate ethernet frames on ingress, you get an unadulterated ethernet frame on the other side. All the basic networking functions that hosts do (ie. arp, broadcast, multicast, etc) work exactly as if the ethernet cable were across the room. The same can not be said for OTV. Victor, EoMPLS/PWE3/VPLS technologies can be tunneled over IP networks today. Many people are doing this right now because it has worked for 10 years, it's part of IETF standards that are adopted by the top 5 vendors, and it's the same constructs that are used in normal networks. Lastly, OTV, as far as I understand is a NX-OS feature. This means that in order to deploy this I need a NEXUS switch. What about all the 65xx or other devices with MPLS that I already have? What if the other disaster recovery site doesn't have a nexus 7k, I am out of luck. While OTV does some sexy things, I just don't think we are looking at anything that is ground breaking. Been there, done that ...

  6. FP

    I agree...this seems contrary to the way the industry is moving. It also seems to be a statement from Cisco that the 7000 won't have MPLS feature set that is currently the de-facto standard of doing these things. The best quote from the document has got to be this one; “This way the server team can move virtual servers and the network team doesn’t have to reinvent the wheel,” But they are re-inventing the wheel.....

  7. Victor Moreno

    Thanks Truman, My reference to L3 is not with respect to using an L3 transport to carry the encapsulated traffic, as you state this has been done for years. What OTV does is treat the L2 traffic that is going over the overlay in a way that allows flood/failure control as well as multi-pathing comparable to what is available for Layer 3 traffic. Although OTV handles Ethernet traffic in a different way, Ethernet data streams remain unaltered while large diameter Ethernet limitations are addressed. OTV is available on Nexus 7000 today, and will be available across the broader Cisco product line in the near future. FP, Cisco is fully committed to supporting MPLS across the entire product line, including the Nexus 7000. We are also committed to customer success and therefore provide innovative functionality designed to simplify network operations and optimize network services. The "wheel" that is not to be re-invented is the network design, the point is that the functionality can be added seamlessly to the network without requiring a re-design of affected Layer 2 domains or the network backbone.

  8. james

    Truman... is it true you work for Juniper? Have you considered disclosing this and being transparent like the guys at Cisco on this blog have?

  9. So Victor, what exactly do you do for Cisco? FP, Cisco is fully committed to supporting MPLS across the entire product line, including the Nexus 7000. We are also committed to customer success and therefore provide innovative functionality designed to simplify network operations and optimize network services.

  10. James Ewing

    No more Truman? Too bad, I liked the debate. James Ewing Cisco Systems

  11. shah

    There is a flavor of this work in IETF L2VPN working group, called IPLS. Cisco engineers are co-authors of this work.

  12. Is the technology available today for deploying? Whom have the expertise with that in Unite Arab Emirates?

  13. I have worked with Cisco Nexus for the last 3yrs. And its the most eastest Deployed Design to come along in a long time.. Keep Up The Good Work Cisco ''''

  14. CiscoFreak

    Where are you Truman? It was a wonderful debate. learned lot nuances about OTV and other L2VPN technologies. please come back and have more discussions here

  15. Max Fleurival

    I have a question . what are the steps require to encrypted all the extended Vlans and Xlvan ?

  16. I have deployed the OTV feature on couple of Data-centers.. As of now its good technology to connect multiple geographical location in single logical domain. Only the the exception its Cisco proprietary and you must have Nexus 7K or ASR in the edge. HP has invented similar feature called "EVI" - Ethernet Virtual Interconnect.