Michael Patterson is product manager for Scrutinizer NetFlow Analyzer, and CEO, co-founder of Plixer International, Inc.
Network traffic monitoring for many companies has morphed into a demand for advanced NetFlow reporting. For over 10 years, NetFlow has been exporting the data that the majority of companies need to perform traffic analysis and billing. Using the elements exported in NetFlow v5, vendors have developed products which help companies with three primary functions:
- Report on the top applications, hosts, protocols, interfaces, subnets, etc. on their networks
- Detect viruses, intrusions and other unwanted behaviors
- Invoice customers by providing accurate IP traffic counts
Traditional NetFlow Reporting
Dozens of companies today deliver NetFlow reporting, analysis and billing systems. The technology has gone through several revisions, but the use of a seven field tuple key for aggregation seems to have been so well thought out that a decade later, the traditional Cisco 7-tuple key is still very much in use for:
1. Source IP address
2. Destination IP address
3. Source port for UDP or TCP, 0 for other protocols
4. Destination port for UDP or TCP, type and code for ICMP, or 0 for other protocols
5. IP protocol
6. Ingress interface (SNMP ifIndex)
7. IP Type of Service
Packets that share all of the above seven key fields are aggregated into a single flow – and then bytes and packets are totaled. The shortcomings of the above result in clear limitations, however, the valuable insight gained through this architecture has resulted in an industry worth several hundred million dollars. Now, a decade later, this market is also starting to mature.
Advanced NetFlow Reporting
With the flow customer base educated on the limitations of NetFlow, there is now an expanded demand for more details. This, combined with the desire from hardware vendors to export additional packet details, has led to an emerging next generation NetFlow. To meet the demands, Cisco’s NetFlow v9 was developed to be extendable for new packet details (i.e. elements) such as VoIP Jitter, Packet loss, URLs, etc. With NetFlow v9, not only are new elements possible, the fields used for the aggregation of packets (i.e. the tuple) can easily be defined to meet the needs of the end user.
Many of the existing Cisco routers sold over the last several years can be upgraded to support these new technologies. When enabled, the CPU impact can be increased to as high as 25 percent. For enterprises with limited network monitoring budget, investing in refurbished Cisco hardware is an option. This would require the network administrator to get familiarized with setting up Flexible NetFlow.
Obviously less details are available for reporting when a shorter tuple is used, however, the flexible nature of NetFlow v9 allows it to cater to the individual needs of each business. Hence this has given birth to a new name “Flexible NetFlow” which is really an extension of NetFlow v9 and not a new version.
Although Cisco NetFlow interoperates with other vendors, the limitations of NetFlow v9 left it somewhat closed to outside vendors in need of exporting new elements. This led Cisco to recommend NetFlow v9 to be primarily used for the proposed standard — Internet Protocol Flow Information Export (IPFIX). With this, when a router, switch or other hardware claims support for SNMP, many consumers expect support for most of MIB2.
MIB2 details include sysName, sysUpTime and SysContact which pretty much all vendors supporting SNMP support. Details on the configuration of the hardware are different per vendor and are not kept in MIB2 rather they are stored in the enterprise MIB. The same holds true when a vendor supports NetFlow or IPFIX. Similar to SNMP, IPFIX allows common elements like MIB2 to be exported in the same format by all vendors. It also allows for extensions (i.e. Enterprise MIBs) empowering vendors like SonicWALL to export additional elements such as URLs, performance metrics, security threat messages, packet captures, Caller ID and the like.
The need for additional ‘new’ information elements exported in flows has been driven by a number of different factors including:
- The change in the nature of traffic that network administrators want to observe (e.g. smart phones, VPNs, WAN compression, multicasts)
- Today’s applications sharing the same ports (e.g. TCP port 80) and driving hardware to perform deep packet inspection (e.g. Cisco NBAR) to identify actual applications such as BitTorrent, Webex, Skype, etc.
- Difficulty with troubleshooting performance issues when limited to NetFlow v5 fields which don’t include details on jitter, TCP latency or packet loss
Next generation flow exports change the competitive field for both flow exporters and flow reporting tools because it is no longer a game of who supports NetFlow or IPFIX. Today’s leading flow vendors can be differentiated by exporting something new that makes traffic and security management easier and more insightful for emerging technologies such as smart phones, VPNs and VoIP.
Industry Perspectives is a content channel at Data Center Knowledge highlighting thought leadership in the data center arena. See our guidelines and submission process for information on participating. View previously published Industry Perspectives in our Knowledge Library.