All packets are not created equal: why DPI and policy vendors look at video encoding

As we are still contemplating the impact of last week's US ruling on net neutrality, I thought I would attempt today to settle a question I often get in my workshops. Why is DPI insufficient when it comes to video policy enforcement?

Deep packet inspection platforms have evolved from a static rules-based filtering engine to a sophisticated enforcement point allowing packet and protocol classification, prioritization and shaping. Ubiquitous in enterprises and telco networks, they are the jack-of-all-trade of traffic management, allowing such a diverse set of use cases as policy enforcement, adult content filtering, lawful interception, QoS management, peer-to-peer throttling or interdiction, etc...
DPIs rely first on a robust classification engine. It snoops through data traffic and classifies each packet based on port, protocol, interface, origin, destination, etc... The more sophisticated engines go beyond layer 3 and are able to recognize classes of traffic using headers. This classification engine is sufficient for most traffic type inspection, from web browsing to email, from VoIP to video conferencing or peer-to-peer sharing.
The premise, here is that if you can recognize, classify, tag traffic accurately, then you can apply rules governing the delivery of this traffic, ranging from interdiction to authorization, with many variants of shaping in between.

DPI falls short in many cases when it comes to video streaming. Until 2008 or so, most video streaming was relying on specialized protocols such as RTSP. The classification was easy, as the videos were all encapsulated in a specific protocol, allowing instantiation and enforcement of rules in pretty straightforward manner. The emergence and predominance of HTTP based streaming video (progressive download, adaptive streaming and variants) has complicated the task for DPIs. The transport protocol remains the same as general web traffic, but the behaviour is quite different. As we have seen many times in this blog, video traffic must be measured in different manner from generic data traffic, if policy enforcement is to be implemented. All packets are not created equal.

• The first challenge is to recognise that a packet is video. DPIs generally infer the nature of the HTTP packet based on its origin/destination. For instance, they can see that the traffic's origin is YouTube, they can therefore assume that it is video. This is insufficient, not all YouTube traffic is video streaming (when you browse between pages, when you read or post comments, when you upload a video, when you like or dislike...). Applying video rules to browsing traffic or vice versa can have adverse consequences on the user experience.
• The second challenge is policy enforcement. The main tool in DPI arsenal for traffic shaping is setting the delivery bit rate for a specific class of traffic. As we have seen, videos come in many definition (4k, HD, SD, QCIF...), many containers and many formats, resulting in a variety of different encoding bit rate. If you want to shape your video traffic, it is crucial that you know all these elements and the encoding bit rate, because if traffic is throttled below the encoding, rate, then the video stalls and buffers or times out. It is not reasonable to have a one-size-fits-all policy for video (unless it is to forbid usage). In order to extract the video-specific attributes of a session, you need to decode it, which requires in-line transcoding capabilities, even if you do not intend to modify that video.

Herein lies the difficulty. To implement intelligent, sophisticated traffic management rules today, you need to be able handle video. To handle video, you need to recognize it (not infer or assume), and measure it. To recognize and measure it, you need to decode it. This is one of the reasons why Allot bought Ortiva Wireless in 2012, Procera partnered with Skyfire and ByteMobile upgraded their video inspection to full fledged DPI more recently. We will see more generic traffic management vendors (PCRF, PCEF, DPI...) partner and acquire video transcoding companies.

Net neutrality denied for US broadband

Tuesday this week, The appeals court of the DC district ruled that the FCC (US regulator) had no authority to impose "Open Internet Order" (net neutrality) rules to broadband carriers. The rationale is that broadband carriers, such as the plaintiff -  Verizon  - are to be considered on the same level as Google, Apple and Netflix and should not be subjected to net neutrality provisions.

Those among you who read this blog know my stance on net neutrality in mobile networks and as it pertains to video.  This ruling is the first that recognizes that carriers fixed and mobile, essentially are in the same market as internet content and service providers and that imposing net neutrality rules on the former would benefit the later in an anti-competitive manner.

Net neutrality does not mean do nothing and let the traffic sort itself out. It is inefficient and contrary to public interest. Because of video elasticity, not managing actively traffic causes overall congestion, which impacts user experience and raises costs.

Unfortunately, many regulators worldwide, in a short-sighted attempt to appear open and supportive of the "free internet" enact net neutrality edicts that cripple their economy and reduce consumer's choice and quality of experience.

Net neutrality is a concept that can be applied to a network with large to infinite capacity with little to no congestion. It is a simplistic Keynesian view of network dynamics. The actors in the internet content delivery chain are all trying to produce the best user experience for their customers. When it comes to video, they invariably equate quality to speed. As a result, content providers, video players, web browsers, phone manufacturers are all trying to extract and control the most speed for their applications, devices sites to guarantee a superior user experience.

The result is not a self-adjusting network that distributes resources efficiently based on supply and demand but an inefficient flow of traffic that is subject to race conditions and snowball effects when Netflix, Google / Youtube, Apple and others compete for network capacity for their devices / browsers /apps / web sites / content.

As the CES show wrapped up last week, ripe with 4K content and device announcements, one cannot help but think that traffic management, prioritization, sponsored data plans, are going to become the rule going forward. That is if the regulators accept the current ruling and adapt the concept of net neutrality to a marketplace where access providers are no longer in control and content and devices dictate usage and traffic.

Thoughts on Flash Networks' acquisition of Mobixell

This is it. The news hit the press officially today and officiously yesterday through Azi Ronen's blog. Flash Networks has acquired Mobixell Networks. The newly formed company will command a leading market share in deployment in the video optimization segment.

This acquisition is the latest in a long series (see Marlin/Openwave, Opera/Skyfire, Citrix/ByteMobile and Allot/Ortiva...) and there are certainly more to come. If you are a frequent reader of this blog, it will not come as a surprise to you to see further concentration in that space.

Historically speaking, the companies in this segment have been under-capitalized to go after the market opportunity. Start-ups and reboots have been the rule rather than the exception and only recently did medium to large size companies such as Citrix, Allot, NSN, Huawei and Cisco entered the market. Inevitably, an increase in competition, together with a quasi full penetration of tier 1 has led to a price attrition.

Mobixell has been one of the proponent of the price war and while the strategy to acquire market share at any cost has served its purpose, since it has put them in the second place in term of installed base, It has been punishing on their bottom line. The company might have experienced some "investor fatigue" that has led to the historical CEO and CTO, both co-founders leaving the company earlier this year.
As disclosed to my clients, the change of direction would coincide with a strategy change, with an emphasis on profitability, but the company was already committed to a growing customer base that would require more capital to serve efficiently.

The new entity will have  a critical mass of customers in this space and a dominant market share in term of deployments, but not in revenue, as Citrix/ByteMobile still dominates most of the high-margin tier 1 mature operator groups.
No doubt this is not the endgame for Flash Networks and that more consolidations are to be expected in the near future.

Flash Networks' success will now require a large product management undertaking, to digest Mobixell , make the necessary choices between a product base almost entirely redundant and cajole both companies' customers with a roadmap that will be worth waiting for while the products align.

As mentioned previously, between policy management, optimization, charging, signalling management and DPI, there are too many vendors and too many functions with large overlap. Video is no doubt an important element of the equation as it now dominates data traffic but it is a relatively misunderstood technology that requires specialized and costly R&D investment. With so many under capitalized start ups, it is easier to acquire the technology than to develop it in-house. Particularly if you consider that it takes 40-55m$ and 7 years to bring a product to market. Many companies have under-estimated the skill set necessary to operate in video and an acquisition is also the best vehicle to acquire experienced engineers and patents. Full in-depth analysis of the market and the vendors' strategy can be found in the mobile video optimization report and workshops.

What's in store for 2014: wholesaler or value provider?

First, happy new year to all my clients and readers out there in the blogosphere. Thanks for your continuous support, comments and readership.

For those of you who have been reading the news lately, we have seen a couple of events over the last months that, in my mind will amount to a watershed moment in the OTT video market space.

Consider this:

1. Comcast allows HBO to become fully OTT and to offer HBO Go to Comcast broadband subscriber WITHOUT a cable subscription.
2. Verizon buys Edgecast, rumors follow of AT&T in discussions with Akamai.
3. Netflix pilots new price models.
4. Comcast and Nielsen work together to monetize VOD with advertising insertion and audience measurement.

These are just but a few of the headlines that grabbed my attention. The market is changing fast and the quest for viable business models is accelerating.

Content providers, creators, aggregators:

OTT video content providers are reaching a stage of maturity where content creation / acquisition was the key in the first phase, followed by subscriber acquisition. As they reach critical mass, the game will change and they will need to simultaneously maximize monetization options by segmenting their user base into new price plans and find a way to unlock value in the mobile market.

Network operators:

I was recently conducting a workshop with a large tier 1 group of network operators and the problem was acutely perceived. On one hand, current video quality in mobile networks (even LTE) is not sufficient for video monetization, furthermore, the various regulators are enforcing unfair constraints under the cover of net neutrality, fundamentally misunderstanding the balance of forces between network operators, content providers and device vendors. On the other hand, video behaves differently from the rest of the data traffic and must be managed in order to provide a good user experience for most users.
Network operators have failed so far to provide an enticing OTT experience (video or voice or messaging) and are likely to fall into two categories in 2014.

Wholesaler or Value provider?

Our industry, with  its stiff regulations is ill-equipped to protect the value and the jobs provided by network operators worldwide. Companies coming from the web and the broadcast industry are more likely to succeed in mobile nowadays. They have the advantage of a more flexible range of business models, little to no barrier to entry, as the infrastructure is guaranteed and provided by network operators and have the control of the value and delivery chain as long as they control the content.

As a wholesaler, network operators can still offer a variety of services and value to content providers / aggregators (see my cartoon video here). But they have to accept and internalize their role as a network specialist, to provide value by categorizing traffic and consumers and opening up APIs to allow more control over the experience to content providers.Obviously, the game, here is to run the network as efficiently as possible, as close to capacity as possible and to generate margin on volume, rather than services.

For those network operators who want to have a more active role in the value chain, there is a path, but it requires some profound changes in its organization and mindset that I am not sure many can muster. It requires first forging relationship with content providers and aggregators. The mistrust between the entities is not lost on Verizon and it is a large reason why they are buying a CDN. Content providers have no issue letting CDNs package, encode, encrypt, ad insert their content but have huge objections letting network operators do so. Network operators have a fantastic opportunity, though, when it comes to tailoring content and services for their subscribers, according to their subscription, location, device, etc... The second step revolves around advertising. There is much that can be said here and I will expand on these themes in 2014.