Use Deep Video Analysis for Optimization

Development of a video encoder is an iterative process. Over time, incremental improvements yield better and better performance. Metrics relying on mathematical measures of rate and distortion are commonly used to quantify these improvements. Visual inspection of compressed sequences is necessary as mathematical measures of distortion do not fully capture all aspects of perceived visual quality. Bitstream analyzers also play a crucial role in the development of video encoders as they enable to visualize individual syntax elements of a bitstream as well as intermediate steps in the video reconstruction process. At NGCodec we use Macs and the Intel® Video Pro Analyzer is a natural fit since it supports HEVC and runs natively on Mac OS X.

Understanding precisely where bits are being spent is important to locate regions of an image where encoding algorithms could be improved. The heat map view allows quick visualization of the bit distribution within a picture. In addition to commonly used color coding the Intel Video Pro Analyzer further displays an actual bit count for each Coding Unit (CU), which helps put things into perspective. Once “hot” areas are localized additional views detailing mode usage, prediction and residuals are used to understand the underlying cause of the high bit count. With this understanding, alternate encoding strategies are developed, tested and further analyzed.

Figure 1 Heat map with bits/CU overlay

Figure 1 Heat map with bits/CU overlay

Sometimes unexpected artifacts may appear in a compressed video. It may be difficult to explain how they come about by simply looking at decoded pictures. Using the Intel Video Pro Analyzer it becomes easy to determine the source of artifacts. On one occasion we faced unexplained artifacts that were present in a picture consisting of only intra blocks. The detailed Prediction Unit (PU) view, including 3 versions of the reference samples used for prediction (raw, padded, and filtered) was instrumental in understanding the phenomenon. Once the phenomenon was understood it was possible to design encoding strategies that avoided the issue.

Figure 2 Detailed intra prediction view with reference samples in 3 different stage

Figure 2 Detailed intra prediction view with reference samples in 3 different stage

So in summary we evaluated several HEVC bitstream analyzers and ultimately purchased multiple seats of the Intel Video Pro Analyzer because:

  • It allows us to deliver superior video encoding algorithms  
  • It runs natively on various OS platforms, including Mac OS X
  • It has extensive support for HEVC profiles, including 10-bit and 4:4:4 profiles
  • It efficiently processes large resolution files, including UHD/4K
  • It has a rich feature set that enables detailed low-level analysis of HEVC bitstreams

You can learn more about the Intel Pro Analyzer including downloading a free 30-day trial from here.

Comment