Tuesday, March 20, 2012

Mobile video QOE part II: Objective measurement

Objective measurement of video is performed using mathematical models and algorithm measuring the introduction of noise and the structural similarity of video objects.

There are several mathematical models such as PSNR (Peak Signal to Noise Ratio) and SSIM (Structural Similarity) for instance, that are traditionally used for these calculations. The complexity resides in the fact that a mathematical difference from one pixel to another, from one frame to another does not necessarily translate equally in the human eye.
PSNR is a measure that has a medium to low accuracy but is quite economic in computation. It represent possibly up to 10% of the CPU effort necessary to perform a transcoding operation. This means that although it provides a result that is not fully accurate, the model can be used to compute calculations as the file is being optimized. A vendor can use PSNR as a basis to provide a Mean Opinion Score (MOS) on the quality of a video file.
Video quality of experience measurement can be performed with full reference (FR), reduced reference (RR) or no reference (NR).

Full Reference
Full reference video measurement means that every pixel of a distorted video is compared to the original video. It implies that both original and optimized video have the same number of frames, are encoded in the same format, with the same aspect ratio, etc… It is utterly impractical in most cases and requires enormous CPU capacity to process, in many cases more than what is necessary for the actual transcoding / optimization.

Here is an example of a full reference video quality measurement method under evaluation and being submitted to ITU-T.
As a full reference approach, the model compares the input or high-quality reference video and the associated degraded video sequence under test. 

Score estimation is based on the following steps:

1)          First, the video sequences are preprocessed. In particular, noise is removed by filtering the frames and the frames are subsampled.
2)          A temporal frame alignment between reference and processed video sequence is performed.
3)          A spatial frame alignment between processed video frame and the corresponding reference video frame is performed.
4)          Local spatial quality features are computed: a local similarity and a local difference measure, inspired by visual perception.
5)          An analysis of the distribution of the local similarity and difference feature is performed.
6)          A global spatial degradation is measured using a Blockiness feature.
7)          A global temporal degradation is measured using a Jerkiness feature. The jerkiness measure is computed by evaluating local and global motion intensity and frame display time.
8)         The quality score is estimated based on a non-linear aggregation of the above features.
9)          To avoid misprediction in case of relatively large spatial misalignment between reference and processed video sequence, the above steps are computed for three different horizontal and vertical spatial alignments of the video sequence, and the maximum predicted score among all spatial positions is the final estimated quality score.

Reduced reference 
Reduced reference video measurement is performing the same evaluation as in the full reference model but only on a subset of the media. It is not widely used as frames need to be synchronized and recognized before evaluation.

No reference 
No reference video measurement is the most popular method in video optimization and is used usually when the encoding method is known. The method relies on the tracking of artefacts in the video, such as blockiness, jerkiness, blurring, ringing…etc. to derive a score.

Most vendors will create a MOS score from proprietary no reference video measurement derived from mathematical models. The good vendors constantly update the mathematical model with comparative subjective measurement to ensure that the objective MOS score sticks as much as possible to the subjective testing. You can find out who is performing which type of measurement and their method in my report, here.

No comments: