All packets are not created equal: why DPI and policy vendors look at video encoding

As we are still contemplating the impact of last week's US ruling on net neutrality, I thought I would attempt today to settle a question I often get in my workshops. Why is DPI insufficient when it comes to video policy enforcement?

Deep packet inspection platforms have evolved from a static rules-based filtering engine to a sophisticated enforcement point allowing packet and protocol classification, prioritization and shaping. Ubiquitous in enterprises and telco networks, they are the jack-of-all-trade of traffic management, allowing such a diverse set of use cases as policy enforcement, adult content filtering, lawful interception, QoS management, peer-to-peer throttling or interdiction, etc...
DPIs rely first on a robust classification engine. It snoops through data traffic and classifies each packet based on port, protocol, interface, origin, destination, etc... The more sophisticated engines go beyond layer 3 and are able to recognize classes of traffic using headers. This classification engine is sufficient for most traffic type inspection, from web browsing to email, from VoIP to video conferencing or peer-to-peer sharing.
The premise, here is that if you can recognize, classify, tag traffic accurately, then you can apply rules governing the delivery of this traffic, ranging from interdiction to authorization, with many variants of shaping in between.

DPI falls short in many cases when it comes to video streaming. Until 2008 or so, most video streaming was relying on specialized protocols such as RTSP. The classification was easy, as the videos were all encapsulated in a specific protocol, allowing instantiation and enforcement of rules in pretty straightforward manner. The emergence and predominance of HTTP based streaming video (progressive download, adaptive streaming and variants) has complicated the task for DPIs. The transport protocol remains the same as general web traffic, but the behaviour is quite different. As we have seen many times in this blog, video traffic must be measured in different manner from generic data traffic, if policy enforcement is to be implemented. All packets are not created equal.

• The first challenge is to recognise that a packet is video. DPIs generally infer the nature of the HTTP packet based on its origin/destination. For instance, they can see that the traffic's origin is YouTube, they can therefore assume that it is video. This is insufficient, not all YouTube traffic is video streaming (when you browse between pages, when you read or post comments, when you upload a video, when you like or dislike...). Applying video rules to browsing traffic or vice versa can have adverse consequences on the user experience.
• The second challenge is policy enforcement. The main tool in DPI arsenal for traffic shaping is setting the delivery bit rate for a specific class of traffic. As we have seen, videos come in many definition (4k, HD, SD, QCIF...), many containers and many formats, resulting in a variety of different encoding bit rate. If you want to shape your video traffic, it is crucial that you know all these elements and the encoding bit rate, because if traffic is throttled below the encoding, rate, then the video stalls and buffers or times out. It is not reasonable to have a one-size-fits-all policy for video (unless it is to forbid usage). In order to extract the video-specific attributes of a session, you need to decode it, which requires in-line transcoding capabilities, even if you do not intend to modify that video.

Herein lies the difficulty. To implement intelligent, sophisticated traffic management rules today, you need to be able handle video. To handle video, you need to recognize it (not infer or assume), and measure it. To recognize and measure it, you need to decode it. This is one of the reasons why Allot bought Ortiva Wireless in 2012, Procera partnered with Skyfire and ByteMobile upgraded their video inspection to full fledged DPI more recently. We will see more generic traffic management vendors (PCRF, PCEF, DPI...) partner and acquire video transcoding companies.

Net neutrality denied for US broadband

Tuesday this week, The appeals court of the DC district ruled that the FCC (US regulator) had no authority to impose "Open Internet Order" (net neutrality) rules to broadband carriers. The rationale is that broadband carriers, such as the plaintiff -  Verizon  - are to be considered on the same level as Google, Apple and Netflix and should not be subjected to net neutrality provisions.

Those among you who read this blog know my stance on net neutrality in mobile networks and as it pertains to video.  This ruling is the first that recognizes that carriers fixed and mobile, essentially are in the same market as internet content and service providers and that imposing net neutrality rules on the former would benefit the later in an anti-competitive manner.

Net neutrality does not mean do nothing and let the traffic sort itself out. It is inefficient and contrary to public interest. Because of video elasticity, not managing actively traffic causes overall congestion, which impacts user experience and raises costs.

Unfortunately, many regulators worldwide, in a short-sighted attempt to appear open and supportive of the "free internet" enact net neutrality edicts that cripple their economy and reduce consumer's choice and quality of experience.

Net neutrality is a concept that can be applied to a network with large to infinite capacity with little to no congestion. It is a simplistic Keynesian view of network dynamics. The actors in the internet content delivery chain are all trying to produce the best user experience for their customers. When it comes to video, they invariably equate quality to speed. As a result, content providers, video players, web browsers, phone manufacturers are all trying to extract and control the most speed for their applications, devices sites to guarantee a superior user experience.

The result is not a self-adjusting network that distributes resources efficiently based on supply and demand but an inefficient flow of traffic that is subject to race conditions and snowball effects when Netflix, Google / Youtube, Apple and others compete for network capacity for their devices / browsers /apps / web sites / content.

As the CES show wrapped up last week, ripe with 4K content and device announcements, one cannot help but think that traffic management, prioritization, sponsored data plans, are going to become the rule going forward. That is if the regulators accept the current ruling and adapt the concept of net neutrality to a marketplace where access providers are no longer in control and content and devices dictate usage and traffic.