Video is eating the internet: clouds, codecs and alliances

A couple of news should have caught your attention this week if you are interested the video streaming business.

Amazon Web Services confirmed yesterday the acquisition of Elemental. This is the outcome of a trend that I was highlighting in my SDN / NFV report and workshops for the last year with the creation of specialized clouds. Elemental's products are software based and the company was the first in professional video to offer cloud-based encoding on Amazon EC2 with a PaaS offering. Elemental has been building virtual private clouds on commercial clouds for their clients and was the first to coin the term "Software Defined Video". As Elemental joins AWS, Amazon will be one of the first commercial clouds to offer a global, turnkey, video encoding, workflow, packaging infrastructure in the cloud. Video processing requires specific profiles in a cloud environment and it is not surprising that companies who have cloud assets look at creating cloud slices or segregated virtual environment to manage the processing heavy, latency sensitive service.

The codec war has been on for a long time, and I had previously commented on it. In other news, we have seen Amazon again join Cisco, Google, Intel, Microsoft Mozilla and Netflix in the Alliance for Open Media. This organization's goal is to counter unreasonable claims made by H.265 / HEVC patent holders called HEVC advance who are trying to create a very vague and very expensive licensing agreement for the use of their patents. The group, composed of Dolby, GE, Mitsubishi Electric and Technicolor is trying to enforce a 0.5% fee on any revenue associated with the codec's use. The license fee would apply indiscriminately to all companies who encode, decode, transmit, display HEVC content. If H.265 was to be as successful as H.264, it would account in the future for over 90% of all video streaming traffic and that 0.5% tax would be presumably levied on any content provider, aggregator, APP, web site... HEVC advance could become the most profitable patent pool ever, with 0.5% of the revenues of Google, Facebook or Apple's video business. The group does not stop there and proposes a license fee on devices as well, from smartphones, to tablets, to TVs or anything that has a screen and a video player able to play H.265 videos... Back to the Alliance for Open Media who has decided to counter attack and vows to create a royalty-free next generation video codec. Between Cisco's Thor, Google's VPx and Mozilla Daala, this is a credible effort to counter HEVC advance.

The Streaming Video Alliance, created in 2014 to provide a forum for TV, cable, content owners and service providers to improve the internet video streaming experience welcomes Sky and Time Warner Cable to the group already composed of Alcatel-Lucent, Beamr, CableLabs, Cedexis, Charter Communications, Cisco, Comcast, Conviva, EPIX, Ericsson, FOX Networks, Intel, Irdeto, Korea Telecom, Level 3 Communications, Liberty Global, Limelight Networks, MLB Advanced Media, NeuLion, Nominum, PeerApp, Qwilt, Telecom Italia, Telstra, Ustream, Verizon, Wowza Media Systems and Yahoo!. What is remarkable, here is the variety of the group, where MSOs, vendors, service providers are looking at transparent caching architectures and video metadata handling outside of the standards, to counter specialized video delivery networks such as Apple's, Google's and Netflix'

All in all, video is poised to eat the internet and IBC, starting next week will no doubt bring a lot more exciting announcements. The common denominator, here is that all these companies have identified that encoding, managing, packaging, delivering video well will be a crucial differentiating factor in tomorrow's networks. Domination of only one element of the value chain (codec, network, device...) will guarantee great power in the ecosystem. Will the vertically integrated ecosystems such as Google and Apple yield as operators, distributor and content owners organize themselves? This and much more in my report on video monetization in 2015.

Data traffic optimization feature set

Data traffic optimization in wireless networks has reached a mature stage as a technology . The innovations that have marked the years 2008 – 2012 are now slowing down and most core vendors exhibit a fairly homogeneous feature set. 

The difference comes in the implementation of these features and can yield vastly different results, depending on whether vendors are using open source or purpose-built caching or transcoding engines and whether congestion detection is based on observed or deduced parameters.

Vendors tend nowadays to differentiate on QoE measurement / management, monetization strategies including content injection, recommendation and advertising.

Here is a list of commonly implemented optimization techniques in wireless networks.

•  TCP optimization
• Buffer bloat management
• Round trip time management
• Web optimization
• GZIP
•  JPEG / PNG… transcoding
• Server-side JavaScript
• White space / comments… removal
• Lossless optimization
• Throttling / pacing
• Caching
• Adaptive bit rate manipulation
• Manifest mediation
• Rate capping
• Lossy optimization
• Frame rate reduction
• Transcoding
• Online
• Offline
• Transrating
• Contextual optimization
• Dynamic bit rate adaptation
• Device targeted optimization
• Content targeted optimization
• Rule base optimization
• Policy driven optimization
• Surgical optimization / Congestion avoidance
• Congestion detection
• TCP parameters based
• RAN explicit indication
• Probe based
• Heuristics combination based
• Encrypted traffic management
• Encrypted traffic analytics
• Throttling / pacing
• Transparent proxy
• Explicit proxy
• QoE measurement
• Web
• page size
• page load time (total)
• page load time (first rendering)
• Video
• Temporal measurements
• Time to start
• Duration loading
• Duration and number of buffering interruptions
• Changes in adaptive bit rates
• Quantization
• Delivery MOS
• Spatial measurements
• Packet loss
• Blockiness
• Blurriness
• PSNR / SSIM
• Presentation MOS

An explanation of each technology and its feature set can be obtained as part of the mobile video monetization report series or individually as a feature report or in a workshop.

Allot acquires Ortiva Wireless

You probably by now all know to whom I was referring to in my last post, when I was mentioning rumors of video optimization vendors getting closer to policy vendors. Allot announced this morning the acquisition of Ortiva Wireless for an undisclosed amount. 


This is the 4th consolidation in this space in 24months, after Ripcode was acquired by RGBNetworks, Dilithium's assets were sold to OnMobile in 2010 and Openwave products division was acquired by Marlin Equity partners earlier this year. Additionally, in related spaces, Avaya acquired Radvision and RealNetworks licensed its codec to Intel in 2012.

I had first heard that Ortiva was in advanced discussions with Allot on March 31st. At that point, Ortiva having allegedly lost future business with Sprint to Bytemobile was in a dire situation, as far as future revenue prospects where considered. Furthermore, one of its main investors, Intel does not appear on the last two financing bridges filed with the SEC. Allot, who had been rumored to have looked at many vendors in the space over the last 18 months, was the number one contender for a possible acquisition. Neither company wanted to offer comments at that stage, even when last week, the rumor became public in Israel and was commented on Azi Ronen's blog here. 


Beyond the purely opportunistic approach of this acquisition, it makes a lot of sense for Allot to have tried and integrate video optimization functions in its portfolio. Bytemobile has strong announced ties with Openet and last week, at the Policy control and real time charging conference 2012, the core of many discussions revolved around how to monetize the tide of OTT video traffic.


I was appalled to hear that, when asked about the best way to price for video, a panel composed of Reliance India, Vodafone Spain and Telefonica Czech, was mostly concerned about congestion and looking at pricing based on time of day. This is a defensive, cost-containment strategy that is sure to backfire. Many vendors who have been selling cost reduction as the main rationale for video optimization have backpedaled in the last few months. As it happens, many operators found out that in peak periods, managing aggressively the size of the feeds to reduce costs is not working. They see that a reduction in 20 to 30% of the size of the individual feeds does not mean less cost, but 20 to 30% more users accessing the same capacity at the same time. Which leads in many cases to no additional  revenue since they have not found a way to monetize OTT traffic and no cost reduction, since the network is still not able to meet the demand.


It is of course, one of many possibilities, but what strikes me, is that the industry has not yet agreed on what is the best way to measure video. Capacity (Megabytes), definition (HD or standard), duration, recentness, rights value or speed (Megabit per second) are some of the metrics that can be used for video charging, but in absence of a single accepted metric throughout the industry, many operators are hitting a wall. How is the industry supposed to monetize a traffic that it is not able to measure properly ? How can prices be shared and accepted by all the actors of the value chain if they measure the value of a video differently?
Costs for content owners and aggregators are measured in rights, geographies, storage, version control... Costs for CDNs are measured in geographies, point of presence, capacity... Costs for mobile carriers are measured in capacity, speed, duration, time of day, geography...


This is a  conundrum this industry will need to solve. If the mobile network operators want to "monetize" OTT video traffic, they first need to understand what measures can be used across other mobile networks horizontally and vertically with the other players of the value chain. Only then, an intelligent discussion on value and price can be derived. In the meantime, OTT vendors will continue selling (and in most cases giving) video content on mobile networks, increasing costs with no means for a viable business model.

Intel gets Real: Intel Buys $120m Codec Patents From RealNetworks

"RealNetworks, Inc. (Nasdaq: RNWK) today announced that it has signed an agreement to sell a significant number of its patents and its next generation video codecs software to Intel Corporation for a purchase price of $120 millions. Under terms of the sale, RealNetworks retains certain rights to continue to use the patents in current and future products.

"Selling these patents to Intel unlocks some of the substantial and unrealized value of RealNetworks assets," said Thomas Nielsen, RealNetworks President and CEO. "It represents an extraordinary opportunity for us to generate additional capital to boost investments in new businesses and markets while still protecting our existing business.

"RealNetworks is pleased Intel has agreed to acquire our next generation video codec software and team," said Nielsen. "Intel has a strong reputation as a technology innovator, and we believe they are well positioned to build on the development work and investment we've made in this area."

"As the technology industry evolves towards an experience-centric model, users are demanding more media and graphics capabilities in their computing devices.  The acquisition of these foundational media patents, additional patents and video codec software expands Intel's diverse and extensive portfolio of intellectual property," said Renée James, Intel senior vice president and general manager of the Software and Services Group.  "We believe this agreement enhances our ability to continue to offer richer experiences and innovative solutions to end users across a wide spectrum of devices, including through Ultrabook devices, smartphones and digital media."

In addition to the sale of the patents and next-generation video codec software, RealNetworks and Intel signed a memorandum of understanding to collaborate on future support and development of the next-generation video codec software and related products.

"We look forward to working with Intel to support the development of the next-generation video codec software and to expanding our relationship into new products and markets," said Nielsen.

RealNetworks does not anticipate that the sale of the approximately 190 patents and 170 patent applications and next generation video codec software will have any material impact on its businesses. RealNetworks businesses include a wide variety of SaaS products and services provided to global carriers, RealPlayer, the Helix streaming media platform, GameHouse online and social games, SuperPass and other media products and services sold both directly to consumers and through partners."

Another strong message and movement in the video encoding space. Video intellectual property as we have seen here is becoming increasingly strategic

For or against Adaptive Bit Rate? part III: Why isn't ABR more successful?

So why isn't ABR more successful? As we have seen here and here, there are many pros for the technology. It is a simple, efficient means to reduce the load on networks, while optimizing the quality of experience and reducing costs.

Lets review the problems experienced by ABR that hinder its penetration in the market.

1. Interoperability
Ostensibly, having three giants such as Apple, Adobe and Microsoft each pushing their version of the implementation leads to obvious issues. First, the implementations by the three vendors are not interoperable. That's one of the reason why your iPad wont play flash videos.Not only the encoding of the file is different (fMP4 vs. multiplexed), but the protocol (MPEG2TS vs. HTTP progressive download) and even the manifest are proprietary.This leads to a market fragmentation that forces content providers to choose their camp or implement all technologies, which drives up the cost of maintenance and operation proportionally.MPEG DASH, a new initiative aimed at rationalizing ABR use across the different platforms was just approved last month. The idea is that all HTTP based ABR technologies will converge towards a single format, protocol and manifest.

2. Economics
Apple, Adobe and Microsoft seek to control the content owner and production by enforcing their own formats and encoding. I don't see them converge for the sake of coopetition in the short term. A good example is Google's foray into WebM and its ambitions for YouTube.

4. Content owners' knowledge of mobile networks
Adaptive bit rate puts the onus on content owners to decide which flavour of the technology they want to implement, together with the range of quality they want to enable. In last week's example, we have seen how 1 file can translate into 18 versions and thousand of fragments to manage.Obviously, not every content provider is going to go the costly route of transcoding and managing 18 versions of the same content, particularly if this content is user-generated or free to air. This leaves the content provider with the difficult situation to select how many versions of the content and how many quality levels to be supported.
As we have seen over the last year, the market changes at a very rapid pace in term of which vendors are dominant in smartphone and tablets. It is a headache for a content provider to foresee which devices will access their content. This is compounded by the fact that most content providers have no idea of what the effective delivery bit rates can be for EDGE, UMTS, HSPA, HSPA +, LTE In this situation, the available encoding rate can be inappropriate for the delivery capacity.

In the example above, although the content is delivered through ABR, the content playback will be impacted as the delivery bit rate crosses the threshold of the lowest available encoding bit rate. This results in a bad user experience, ranging from buffering to interruption of the video playback.

5. Tablet and smartphone manufacturers knowledge of mobile networks
Obviously, delegating the selection of the quality of the content to the device is a smart move. Since the content is played on the device, this is where there is the clearest understanding of instantaneous network capacity or congestion. Unfortunately, certain handset vendors, particularly those coming from the consumer electronics world do not have enough experience in wireless IP for efficient video delivery. Some devices for instance will go and grab the highest capacity available on the network, irrespective of the encoding of the video requested. So, for instance if the capacity at connection is 1Mbps and the video is encoded at 500kbps, it will be downloaded at twice its rate. That is not a problem when the network is available, but as congestion creeps in, this behaviour snowballs and compounds congestion in embattled networks.

As we can see, there are  still many obstacles to overcome for ABR to be a successful mass market implementation. My next post will show what alternatives exist to ABR in mobile networks for efficient video delivery.

For or against Adaptive Bit Rate? part I: what is ABR?

Adaptive Bit Rate streaming (ABR) was invented to enable content providers to provide video streaming services in environment in which bandwidth would fluctuate. The benefit is clear, as a connection capacity changes over time, the video carried over that connection can vary its bit rate, and therefore its size to adapt to the network conditions.The player or client and the server exchange discrete information on the control plane throughout the transmission, whereby the server exposes the available bit rates for the video being streamed and the client selects the appropriate version, based on its reading of the current connection condition.

The technology is fundamental to help accommodate the growth of online video delivery over unmanaged (OTT) and wireless networks.
The implementation is as follow: a video file is encoded into different streams, at different bit rates. The player can "jump" from one stream to the other, as the condition of the transmission degrades or improves. A manifest document is exchanged between the server and the player at the establishment of the connection for the player to understand the list of versions and bit rates available for delivery.

Unfortunately, the main content delivery technology vendors then started to diverge from the standard implementation to differentiate and control better the user experience and the content provider community. We have reviewed some of these vendor strategies here. Below are the main implementations:

• Apple HTTP Adaptive (Live) streaming (HLS) for iPhone and iPad: This version is implemented over HTTP and MPEG2 TS. It uses a proprietary manifest called m3u8. Apple creates different versions of the same streams (2 to 6, usually) and  breaks down the stream into little “chunks” to facilitate the client jumping from one stream to the other. This results in thousands of chunks for each stream, identified through timecode.Unfortunately, the content provider has to deal with the pain of managing thousands of fragments for each video stream. A costly implementation.
• Microsoft IIS Smooth Streaming (Silverlight Windows phone 7): Microsoft has implemented fragmented MP4 (fMP4), to enable a stream to be separated in discrete fragments, again, to allow the player to jump from one fragment to the other as conditions change.  Microsoft uses AAC for audio and AVC/H264 for video compression. The implementation allows to group each video and audio stream, with all its fragments in a single file,  providing a more cost effective solution than Apple's.
• Adobe HTTP Dynamic Streaming (HDS) for Flash: Adobe uses a proprietary format called F4F to allow delivery of flash videos over RTMP and HTTP. The Flash Media Server creates multiple streams, at different bit rate but also different quality levels.  Streams are full lengths (duration of video).

None of the implementations above are inter-operable, from a manifest or from a file perspective, which means that a content provider with one 1080p HD video could see himself creating one version for each player, multiplied by the number of streams to accommodate the bandwidth variation, multiplied by the number of segments, chunks or file for each version... As illustrated above, a simple video can result in 18 versions and thousand of fragments to manage. This is the reason why only 4 to 6% of current videos are transmitted using ABR. The rest of the traffic uses good old progressive download, with no capacity to adapt to changes in bandwidth, which explains in turn why wireless network operators (over 60 of them) have elected to implement video optimization systems in their networks. We will look, in my next posts, at the pros and cons of ABR and the complementary and competing technologies to achieve the same goals.

Find part II of this post here.

Google vs. rest of the world: WebM vs H264

After the acquisition of ON2 technologies, as anticipated, Google has open-sourced its video codec WebM (video format VP8 and audio format Ogg Vorbis encapsulated in Matroska container) as an attempt to counter-act H.264.

The issue:

In  the large-scale war between giants Apple, Adobe, Microsoft and Google, video has become the latest battlefield. As PCs and mobile devices consume more and more video, the four companies battle to capture content owners and device manufacturers mind share, in order to ensure user experience market dominance.

Since video formats and protocols are fairly well standardized, the main area for differentiation remains codecs.

Codecs are left to anyone's implementation choice. The issue can be thorny, though, as most codecs used to decode / encode specific formats require payments of license to intellectual property owners.

For instance, the H.264 format is managed by MPEG LA, who has assembled a pool of patents associated with the format, from diverse third parties and is licensing its usage, collecting and redistributing royalties on behalf of the patent owners. H.264 is a format used for transmission of videos in variable bandwidth environment and has been adopted by most handset manufacturers, Microsoft, Apple and Adobe as the de-facto format.

If you are Google, though, the idea of paying license to third parties, that are in most case direct competitors for something as fundamental as video is a problem.

The move:

As a result, Google has announced that they are converting all of Youtube most watched videos to WebM and that the format becomes the preferred one for all Google properties (Youtube, Chrome...).

The purpose here, is for Google to avoid paying royalties to MPEG LA, while controlling user experience by trying to integrate vertically the content owners, browser and device manufacturers codec usage.

It does not mean that Google will stop supporting other formats (flash, H.264...) but the writing is on the wall, if they can garner enough support.

The result:

It is arguable whether WebM can actually circumvent MPEG LA H.264 royalty claims. There are already investigations ongoing as to whether VP8 is not infringing any of H.264 intellectual property. Conversely, the U.S. Department of Justice is investigating whether MPEG LA practices are stifling competition.

In the meantime, content owners, device manufacturers, browser vendors have to contend with one new format and codec, increasing the fragmentation in this space and reducing interoperability.

Mobile video 101: protocols, containers, formats & codecs

Mobile video as a technology and market segment can at times be a little complicated.

Here is simple syllabus, in no particular order of what you need to know to be conversant in mobile video. It is not intended to be exhaustive or very detailed, but rather to provide a knowledge base for those interested in understanding more the market dynamics I address in other posts.


Protocols:

There are many protocols used in wireless networks to deliver and control video. You have to differentiate between routing protocols (IP), transmission protocols (TCP & UDP), session control (RTP), application control (RTSP) and content control protocols (RTCP). I will focus here on application and content control.

These protocols are used to setup, transmit and control video over mobile networks

Here are the main ones:

• RTSP (Real Time Streaming Protocol) is an industry protocol that has been created specifically for the purposes of media streaming. It is used to establish and control (play, stop, resume) a streaming session. It is used in many unicast on-deck mobile TV and VOD services.
• RTCP (Real Time transport Control Protocol) is the content control protocol associated with RTP. It provides the statistics (packet loss, bit transmission, jitter...) necessary to allow a server to perform real-time media quality control on an RTSP stream.
• HTTP download and progressive download (PD). HTTP is a generic protocol, used for the transport of many content formats, including video. Download and progressive download differentiate from each other in that the former needs the whole content to be delivered and saved to the device to be played asynchronously, while the later provides at the beginning of the session a set of metadata associated with the content which allow it to be played before its complete download.
• Microsoft silverlight, Adobe RTMP and Apple progressive streaming. These three variants of progressive download are proprietary. They offer additional capabilities beyond the vanilla HTTP PD (pre-encoding and multiple streams delivery, client side stream selection, chunk delivery...) and are the subject of an intense war between the three companies to occupy the mindset of content developers and owners. This is the reason why you cannot browse a flash site or view a flash video in your iPhone.

Containers:

A container in video is a file that is composed of the payload (video, audio, subtitles, programming guide...) and the metadata (codecs, encoding rate, key frames, bit-rate...). The metadata is a set of descriptive files that indicate the nature of the media, its duration in the payload. The most popular are:

• 3GPP (.3GP) 3GP is the format used in most mobile devices, as the recommended container for video by 3GPP standards.
• MPEG-4 part 14 (.MP4) one of the most popular container for internet video.
• Flash video (FLV, F4V). Adobe-created container, very popular as the preferred format for BBC, Google Video, Hulu, metacafe, Reuters, Yahoo video, YouTube... It requires a flash player.
• MPEG-2 TS: MPEG Transport Stream is used for broadcast of audio and video. It is used in on-deck broadcast TV services in mobile and cable/ satellite video delivery.

Formats
Formats are a set of standards that describe how a video file should be played.

• H.263 old codec used in legacy devices and applications. It is mandated by ETSI and 3GPP for IMS and MMS but is being replaced by H.264
• H.264, MPEG4 part 10, AVC is a family of standards composed of several profiles for different use, device types, screen sizes... It is the most popular format in mobile video.
• MPEG2 is a standard for lossy audio and video compression used in DVD, broadcast (digital TV, over the air, cable, satellite). MPEG2 describes two container types: MPEG2-TS for broadcast, MPEG-2 PS for files.
• MPEG4 is an evolution of MPEG2, adding new functionalities such as DRM, 3D and error resilience for transmission over lossy channels (wireless for instance).  There are many features in MPEG 4, that are left to the developer to decide whether to implement or not. The features are grouped by profiles and levels. There are 28 profiles or part in MPEG 4. A codec usually describe which MPEG-4 parts are supported. It is the most popular format on the internet.

Codecs

Codec stands for encoding and decoding a media stream. It is a program that has the ability to decode a video stream and re encode it. Codecs are used for compression (lossless), optimization (lossy) and encryption of videos. A "raw" video file is usually stored in YCbCr (YUV) format which provides the full description of every pixel in a video. This format is descriptive, which requires a lot of space for storage and a lot of processing power for decoding / encoding. This is why a video is usually encoded in a different codec, to allow for a better size or variable transmission quality. It is important to understand that while a container obeys strict rules and semantics, codecs are not regulated and each vendor decides how to decode and encode a media format.

• DivX Proprietary MPEg-4 implementation by DivX
• WMV (Windows Media Video) - Microsoft proprietary
• x264 a licenseable H.264 encoding spoftware
• VP6, VP7, VP8... proprietary codecs developed by On2 technologies, acquired by Google and released as open source