Data traffic optimization feature set

Data traffic optimization in wireless networks has reached a mature stage as a technology . The innovations that have marked the years 2008 – 2012 are now slowing down and most core vendors exhibit a fairly homogeneous feature set. 

The difference comes in the implementation of these features and can yield vastly different results, depending on whether vendors are using open source or purpose-built caching or transcoding engines and whether congestion detection is based on observed or deduced parameters.

Vendors tend nowadays to differentiate on QoE measurement / management, monetization strategies including content injection, recommendation and advertising.

Here is a list of commonly implemented optimization techniques in wireless networks.

•  TCP optimization
• Buffer bloat management
• Round trip time management
• Web optimization
• GZIP
•  JPEG / PNG… transcoding
• Server-side JavaScript
• White space / comments… removal
• Lossless optimization
• Throttling / pacing
• Caching
• Adaptive bit rate manipulation
• Manifest mediation
• Rate capping
• Lossy optimization
• Frame rate reduction
• Transcoding
• Online
• Offline
• Transrating
• Contextual optimization
• Dynamic bit rate adaptation
• Device targeted optimization
• Content targeted optimization
• Rule base optimization
• Policy driven optimization
• Surgical optimization / Congestion avoidance
• Congestion detection
• TCP parameters based
• RAN explicit indication
• Probe based
• Heuristics combination based
• Encrypted traffic management
• Encrypted traffic analytics
• Throttling / pacing
• Transparent proxy
• Explicit proxy
• QoE measurement
• Web
• page size
• page load time (total)
• page load time (first rendering)
• Video
• Temporal measurements
• Time to start
• Duration loading
• Duration and number of buffering interruptions
• Changes in adaptive bit rates
• Quantization
• Delivery MOS
• Spatial measurements
• Packet loss
• Blockiness
• Blurriness
• PSNR / SSIM
• Presentation MOS

An explanation of each technology and its feature set can be obtained as part of the mobile video monetization report series or individually as a feature report or in a workshop.

Allot continues its spending spree

After the acquisition of Ortiva Wireless, announced in April for $15-$17m, Allot announces today the acquisition of Oversi networks for $16m in cash with a conditional, performance related extra $5m.

Oversi Networks is a provider of transparent caching solutions for OTT and P2P traffic. Specifically, Oversi has been developing a purpose-built video cache, one of the first of its kind.

Many vendors in the space have caches that have been built on open source general-purpose web caches, originally to manage offline video optimization scenarios (for those not able to transcode mp4, flv/f4v containers in real time). As the long tail of video content unfolds, social media and virality create snowballing effects on some video content and a generic web cache shows limitations when it comes to efficiently cache video.

The benefits of a hierarchical, video specific cache then becomes clear. Since video nowadays come in many formats, containers, across many protocols and since content providers repost the same video with different attributes, titles, URLs, duration...etc, it is quite inefficient to cache video only based on metadata recognition. Some level of media inspection is necessary to ascertain what the video is and whether it really corresponds to the metadata.

All in all, another smart acquisition by Allot. On the paper, it certainly strengthens the company position, with technologies compatible and complementary with their legacy portfolio and the recent Ortiva's acquisition. It will be interesting to see how Allot's product portfolio evolves over time and how the different product lines start to synergize.

Edge based optimization part I: RAN traffic optimization

As video traffic grows in mobile and fixed networks alike, innovative companies are looking at optimizing traffic closer to the user. These companies perform loss-less and lossy optimization at the edge of the networks, be it directly in the CDN's PoP or at the RNC in mobile radio networks. We will look today the cellular RAN based optimization and look at Edge optimization in fixed networks in a following post.


As I have indicated in previous posts (here), I believe implementing video lossy optimization in the core network or the backhaul to be very inefficient without a good grasp of what is happening on the user's device or at least in the radio networks. Core network based mobile video optimization vendors infer the state of network congestion by reading and extrapolating the state of the TCP connection. Looking at parameters such as round trip time, packet loss ratio, TCP window, etc... they deduce whether the state of the connection improves or worsens and increase or decrease the rate of optimization. This technique is called Dynamic Bit Rate Adaptation and is one of the most advanced for some of the vendors out there. Others will read the state of the connection at the establishment and will feed and set the encoding rate based on that parameter. 

The problem, with these techniques is that they deal with the symptoms of congestion and not the causes. This leads vendors to taking steps in increasing or reducing the encoded bit rate of the video without understanding what the user is actually experiencing in the field. As you well know, there can be a range of issues affecting the state of a TCP connection, ranging from the device's CPU, its antenna reception, the RAN's sector occupancy from a signalling standpoint, whether the user is moving, etc... that are not actually related to a network payload TCP congestion. Core vendors have no way to diagnose these situations and therefore are treating any degradation of signal as a payload congestion, in some cases creating race conditions and snowball effect where the optimization engine actually contributes to the user experience's degradation rather than improve it.


RAN based optimization vendors are deployed in the RAN, at the RNC or even the base station level and perform a real-time analysis of the traffic. Looking at both payload and signalling per sector, cell, aggregation site, RNC, they can offer a great understanding of what the user is experiencing in real time and whether a degradation in TCP connection is the result of payload congestion, signalling issues or cell handover for instance. This precious data is then analysed, presented and made available for corrective action. Some vendors will provide the congestion indications as a diameter integration, with the information travelling from the RAN to the Core to allow resolution and optimization by the PCRF and the video optimization engine. Some vendors will even provide loss-less and lossy techniques at the RAN level to complement the core capabilities. These can range from payload and DNS deep caching, to TCP tuning, pacing, and content shaping...


This is in my mind a great improvement to mobile networks, allowing to break the barrier between RAN and Core and perform holistic optimization along the delivery chain, where it matters most, with the right information to understand the network's condition.
The next step is having the actual capability to have the device report to the network its reading of the network condition, together with the device state and the video experience to provide feedback loop to the network. The vendors that will resolve the equation device state + RAN condition + Policy management + video optimization = better user experience will strike gold and enable operators to truly monetize and improve mobile video delivery.

Mobile video optimization 2012: executive summary

As I publish my first report (description here), have an exclusive glance with the below summary.

Executive Summary

V

ideo is a global phenomenon in mobile networks. In only 3 years, it has exploded, from a marginal position (less than 10%) to dominating mobile traffic in 2012 with over 50%.

Mobile networks until now, have been designed and deployed predominantly for transactional data. Messaging, email, browsing is fairly low impact and lightweight in term of payload and only necessitated speed compatible with UMTS. Video brings a new element to the equation. Users rarely complained if their text or email arrived late, in fact, they rarely noticed. Video provides an immediate feedback. Consumers demand quality and are increasingly assimilating the network’s quality to the video quality.

With the wide implementation of HSPA (+) and the first LTE deployments, together with availability of new attractive smartphones, tablets and ultra book, it has become clear that today’s networks and price structure are ill-prepared for this new era.

Handset and device vendors have gained much power in the balance and many consumers chose first a device before a provider.

In parallel, the suppliers of content and services are boldly pushing their consumer relationship to bypass traditional delivery media. These Over-The-Top (OTT) players extract more value from consumers than the access and network providers. This trend accelerates and threatens the fabric itself of the business model for delivery of mobile services.

This is the backdrop of the state of mobile video optimization in 2012. Mobile network operators find themselves in a situation where their core network is composed of many complex elements (GGSN, EPC, browsing gateways, proxies, DPI, PCRF…) that are extremely specialized but have been designed with transactional data in mind. The price plans devised to make sure the network is fully utilized are backfiring and many carriers are discontinuing all-you-can-eat data plans and subsidizing adoption of limited, capped, metered models. Radio access is a scarce resource, with many operators battling with their regulators to obtain more spectrum. The current model to purchase capacity, based on purchasing more base stations, densifying the network is finding its limits. Costs for network build up are even expected to exceed data revenues in the coming years.

On the technical front, many operators are hitting the Shannon’s law, the theoretical limit for spectrum efficiency. Diminishing returns are the rule rather than the exception as RAN become denser for the same available spectrum. Noise and interferences increase.

On the financial front, should an operator follow the demand, it would have to double its mobile data capacity on a yearly basis. The projected revenue increase for data services shows only a CAGR of 20% through 2015. How can operators keep running their business profitably? 

Operationally, doubling capacity every year seems impossible for most networks who look at 3 to 5 years roll out plans.

 Solutions exist and start to emerge. Upgrade to HSPA +, LTE, use femto cells or pico cells, change drastically the pricing structure of the video and social services, offload part of the traffic to wifi, implement adaptive bit rate, optimize the radio link, cache, use CDNs, imagine new business models with content providers, device manufacturers and operators… All these solutions and other are examined in this report.

Video optimization has emerged as one of the technologies deployed to solve some of the issues highlighted above. Deployed in over 80 networks globally, it is a market segment that has generated $102m in 2011 and is projected to generate over $260m in 2012. While it is not the unique solution to this issue, {Core Analysis} believe that most network operators will have to deploy video optimization as a weapon in the arsenal to combat the video invasion in their network. 2009 to 2011 saw the first video optimization commercial deployments, mostly as a defensive move, to shore up embattled networks. 2012 sees video optimization as a means to complement and implement monetization strategies, based on usage metering and control, quality of experience measurement and video class of service delivery.

Flash in the cloud

Flash Networks announced today that it is making its Harmony Mobile Internet Services Gateway optimization and monetization solution available in the cloud. The solution that was traditionally deployed in mobile core networks will soon be deployed in private and public clouds.

"Harmony Mobile Internet Services Gateway integrates web and video optimization, analytics, traffic management, web monetization, content control, cell-based congestion awareness, centralized caching, service orchestration, and an intelligent policy engine in a single gateway. "

I spoke today with Merav Bahat, VP Marketing and Business Development at Flash Networks and she adds: "We wanted to introduce the capability for our customers to use cloud services and cloud computing with our platform. Harmony will continue to be deployed in the core networks and in conjunction, can be deployed in private and public clouds. We have been able to duplicate several functions from our platform such as caching, storage and CPU-intensive transcoding and put them in the cloud to offer great additional savings , higher hit rates and enhanced quality of experience".`

As seen here and here, Flash Networks is the third company in the video optimization space who has announced plans to offer a cloud-based solution. Caching, transcoding, content recommendation are some of the services that Flash Networks will perform in the cloud, to benefit carriers with multi-sites or multi-networks footprint.

Cloud-based video optimization is gaining traction, as more and more mobile network operators  see the necessity to deploy video optimization (over 80 have selected vendors to date) but balk at the CAPEX and footprint necessary to enable a good quality of experience.

Cloud deployments and cloud computing were, until recently, seen as an improbable technology to deploy real time video encoding services, but a few tier one operators have tested and are deploying the technology as we speak. It seems that the technology is reaching market validation stage and is getting a much larger acceptance from the carriers' community. It is a good move from Flash Networks to capitalize on this market trend and expand their offering in that space.

Starhub selects Mobixell

Mobixell Networks announced today that it has been selected by Singapore's Starhub. Mobixell will deploy its Seamless Access gateway to perform intelligent traffic management, advertising insertion and video optimization.


Liong Hang Chew, Assistant Vice President of Mobile Network Engineering at StarHub said, “We chose Mobixell Seamless Access to enable a new era of mobile data traffic handling, increasing efficiency and improving customer experience. At the same time, implementing Seamless Access will enable future services such as content security and other possible revenue-generating features."


The deal was won almost a year ago, in the summer of 2010.

Allot to acquire Flash Networks for $110 /$120 M?

This is the latest rumor from Globe. Allot, who has raised almost $80M a month ago and was rumored to be acquired by F5, then to discuss acquisition of Mobixell or PeerApp last year, has a $500M market cap. Flash Networks has raised over $61M.

The resulting company could be booking about $120M in sales and be profitable.

Allot, in a briefing with Jonathon Gordon, Director of Marketing, two weeks ago was noting: " Our policies focus more and more on revenue generation. With over 100 charging plans surveyed in our latest report, we see more and more demand for bundle plans for social networks and video. We can already discriminate traffic that is embedded, for instance, we can see that a user is watching a video within a facebook browsing session, but we cannot recognize and analyse the video in term of format, bit rate, etc...Premium video specific policies raise a lot of interest these days."

No doubt, the acquisition of an optimization vendor like Flash Networks can solve that problem, by creating a harmonious policy and charging function that actually manages video, which accounts for over half of 2011 mobile traffic globally.

As discussed here and here, video optimization becomes an attractive target for telco vendors who want to extend beyond DPI and policy. Since video is such a specialized skill, it is likely that growth in this area will not be organic. It is likely that the browsing gateway / DPI / PCRF / Optimization segments will collapse over the next 2 years, as they are atomized markets, with small, technology-driven under-capitalized companies and medium -to-large mature companies looking to increase market share or grow the top line.

Mobixell update and EVO launch

Mobixell was founded in December of 2000 to focus on mobile multimedia adaptation. Their first product, launched in 2002, was for MMS (Multimedia Messaging) adaptation and was sold through OEMs such as Huawei, Ericsson, NSN and others. It launched a mobile TV platform in 2008, and a mobile video optimization product in 2010. Along the way, Mobixell acquires Adamind in 2007, and 724 Solutions in 2010.


Mobixell has 16% market share of the deployed base of video optimization engines. Nearly 18 months after the launch of the video optimization module in their Seamless Access product suite, Mobixell launches EVO (for Evolved Optimization).


As a follow-up from the 360 degrees review of the video optimization market and in anticipation of the release of my market report, I had a recent chat with Yehuda Elmaliach, CTO and co-founder at Mobixell about their recent announcement, introducing Mobixell EVO.


"We wanted to address the issue of scalability and large deployments in video optimization in a new manner. As traffic grows for Gbps to 10's and 100's of Gbps, we see optimization and particularly;y real-time transcoding as a very CPU intensive activity, which can require a lot of CAPEX. The traditional scaling model, of adding new blades, chassis, sites does not make sense economically if traffic grows according to projections."
Additionally, Yehuda adds "We wanted to move away from pure volume reduction, as a percentage saving of traffic across the line to a more granular approach, focusing on congestion areas and peak hours."


Mobixell EVO is an evolution of Seamless Access video optimization that complements Mobixell capabilities with cloud-based services and benefits. The current Seamless Access product sits on the Gi Interface, after the GGSN and performs traffic management, shaping and video optimization. The video optimization features at that level are real-time transcoding, dynamic bit rate adaptation, offline transcoding and caching. Mobixell EVO proposes to complement or replace this arrangement with a cloud-based implementation that will provide additional computational power and storage in an elastic and cost effective manner for real time transcoding and for a hierarchical caching system.


Yehuda adds: "We have launched this product based on customer feedback and demand. We do not see customers moving their infrastructure to the cloud only for the purpose of optimization, but for those who already have a cloud strategy, it fits nicely. EVO is built on the principles of virtualization, geometric and automatic scalability and self replication to take advantage of the cloud architecture. "


An interesting development for Mobixell. EVO has no commercial deployment yet and is planned to be generally available in Q2 2012 after current ongoing trials and proof of concepts. Mobixell sees this platform being deployed first with carriers private clouds, then maybe using mixed private and public clouds. The idea is a waterfall implementation, where routine optimization is performed at the Gi level, then moves to private cloud or public ones as peak and surges appear on the network. The idea has a certain elegance, particularly for operators that experience congestion in a very peaky, localized manner. In that case a minimum investment can be made on Gi and complemented with cloud services as peaks reach certain thresholds. It will be interesting to see if Mobixell can live up to the promises of EVO, as security, bandwidth, latency and scalability can reduce the benefits of a mixed core / cloud implementation if not correctly addressed.
Mobixell is the second vendor to launch cloud based optimization after Skyfire.

Bytemobile T 3000 series & Unison update

Bytemobile released this week a new platform (T3000) and a new product (T3100).
With more than 40 operator deployments, Bytemobile is the leader in the video optimization market. The new platform is launched to allow Bytemobile to address the intelligent traffic management market .

Mikko Disini, in charge of the new T 3OOO series and Unison platforms discussed with me the rationale behind the introduction of the new product and how it complements Unison.

T3000 series has been created in an effort to provide more monetization options for mobile broadband operators. For those familiar with Unison, which is essentially a web and video proxy and optimization gateway, T3100 expands beyond browsing to proxy and manipulate all traffic, including UDP based applications, P2P, video chat, RTSP, etc..
While Unison remains a software based solution, on off-the-shelf IBM blade center, T3000  series is a purpose built IBM hardware based appliance. T3100 combines load balancing, DPI, PCEF and traffic rules in one package. Bytemobile is planning to introduce new products on the T3000 platform in the future.

Mikko commented that the rationale behind the hardware based approach is to be more channel-friendly. " It is easier to deploy, package, explain, it is an easier sale".

My opinion is that Bytemobile makes a smart move to expand their product portfolio with new verticals. While there is a large level of overlap between Unison and T3100 today, Bytemobile can upsell their installed base with purpose-built solutions. While in the past, Unison was a Swiss Army knife, for a market who was looking for a quick solution, that had a bit of everything, the growth of the traffic is forcing many vendors to separate applications to have more granular scalability.


With T3000, Bytemobile moves more decidedly into the DPI, load balancing, PCEF space than with Unison. Additionally, moving to a hardware appliance model is going to enable them further to resist price erosion, reusing the Unison tactics of bundling several applications and features together with different market prices and models.
What remains to be seen is how effective the strategy is going to be in acquiring new channels, beyond IBM, NSN and TechMahindra now that T3000 is sure to encroach on some bigger players such as F5 and Cisco... or maybe, this is the strategy?

Cloudlet, CDN and content acceleration

As indicated in a previous post, there is much that could be gained from examining in more details how mobile networks could benefit from performing sophisticated content manipulation in the cloud, rather than in core network or in the device.


Yesterday, Citrix Systems and Juniper Networks agreed and invested in Cotendo, who has announced a $17M round of financing. AT&T is already a partner and the company focuses on the enterprise and media segments. Cotendo's massive network allows for global presence, while its technology is focusing on accelerating the mobile web experience.

Additionally, its cloud based implementation provides an alternative to massive CDN strategy which requires point of presence increase, with traffic and geographic expansion, a model that is scalable but economically difficult.

Most of the company positioning is about web generally. Video seems to be missing for the moment.
It will be interesting to see how Cotendo evolves into the mobile realm, disrupting carrier and CDN strategies in the future.

Mobile video 102: lossless and lossy compression

Mobile video as a technology and market segment can at times be a little complicated.

Here is simple syllabus, in no particular order of what you need to know to be conversant in mobile video. It is not intended to be exhaustive or to be very detailed, but rather to provide a knowledge base for those interested in understanding more the market dynamics I address in other posts.

Compression (lossless) and optimization (lossy)

• Compression is the action of reducing the size of the representation of a media object without loosing data. It is lossless when, after decompression, a compressed media is absolutely equal to the original. Compression methods are based on statistical analysis, to represent recurrent data items within a file. PNG, GIF, Zip, gzip and deflate are lossless compression formats. Throttling, just-in-time delivery and caching are lossless delivery methods.
• Optimization is a form of compression called lossy in the sense that it discards data elements to achieve reduced size. The optimized version is not identical to the original. Transcoding, transrating are lossy methods.

Lossy optimization methods:

• Frame per second (fps): A video is composed of  a number of still frames (pictures). The illusion of movement is achieved after 15 frame per second. TV is 24 to 30 fps (depending on the standard and whether it is progressive or interlaced). Many lossy optimization method will reduce the frame per second ratio in order to reduce the size of a file.
• key frame: Not all frames contain the same amount of data in video optimization. The main way to reduce the quantity of information in a video is to use statistical analysis to predict motion. In other words, analyse differences between one frame to the following. Most optimization method will model only the difference between a frame and the next one, therefore not coding all the information. Key frames or Intra frames are the frames used as reference. When lossy optimization is performed using fps reduction, one has to be careful not to remove the key frames or the user experience will be garbled with many artifacts.
• Bit rates: Bit rate is the rate at which a video is encoded (quality) or transmitted. 
• Encoding bit rate:The encoding bit rate represent the amount of information that is captured in each frame. It is measured in kbps or (kilobit per second) Mbps (Megabit per second). HD video is encoded at 20Mbps, SD at 10 Mbps, internet video usually around 1Mbps and video transmitted on wireless between 200 and 700 kbps.
• Variable bit rate (VBR) or transrating is a lossy optimization method that will vary the encoding bit rate throughout the video to take into account lossy network conditions.
• Constant bit rate is used for broadcast, fixed line connection and generally lossless transmissions.
• Delivery bit rate: When a video is transmitted over a wireless network, the connection capacity dictates the user experience. The bit rate of delivery should always exceed the bit rate of encoding of the video for smooth viewing. If the delivery bit rate goes below the encoding bit rate, buffering, stop and go is experienced. Lossy optimization techniques such as VBR allow to reduce the encoding bit rate in real time, as the delivery bit rate varies.
• Transcoding is the action of decoding a video file and recoding it under a different format. This lossy method is effective to reduce a video file from a definition, format that are not suitable for mobile transmission (HD, 3D...). Additionally, a lot of size saving can be operated by changing the aspect ratio (4:3 or 16:9 from TV) or changing the picture size (HD 1080 is 1920 x 1080 pixels, while most smartphones WVGA are 800 x 480 pixels). You can reduce drastically a video file size by changing the picture size.
• Transprotocol is the action to change the protocol used for video transmission. For instance, many legacy phones that do not support progressive download cannot access internet video unless they are transprotocoled to RTSP.