Next Generation of Video compression: The Video Format War…

By Eng. (Dr.) Maheshi B Dissanayake

According to [youtube.com/yt/press/statistics.html] 1,300,000,000 users use Youtube while 300 hours of video are uploaded to the channel every minute. In the modern lifestyle, online video sharing, real time content upload as well as on demand video watching has become a way of life. This lifestyle change has put enormous pressure on the data service. In the last three decades compression standards are coming up at frequent intervals, as a measure to address this demand.

This article outlines the current state of the art in video compression standards and will walk you through the bottlenecks faced in adopting modern compression standards as well as what are the emerging video compression standards..

What is compression?

Raw footage of video, contains huge amounts of correlations both in spatial domain( within a single image) as well as in temporal domain (between neighboring images). Recording these correlations again and again within the same video creates a considerably large amount of redundancy, which requires extra bits. In video compression, we exploit these redundancies and rearrange the data to reduce the redundancy while maintaining desirable final quality level. Some compression algorithms are 100% reversible (lossless compression) while others are not (lossy compression).

In practice many compression standards adopt lossy compression algorithms, by removing some contents in the video which are not sensitive to the human visual system, i.e without damaging the output quality much. By doing so the compression system achieves a significantly high compression ratio ( = size of the raw footage / size of the compressed output)

MPEG and video compression market

The most popular video compression standard in today’s market is H.264. H. 264 is a video compression standard, or a codec, jointly developed by the International Telecommunications Union and International Organization for Standardization/International Electrotechnical Commission Moving Picture Experts Group. It is very efficient and provides high-quality images with minimal amount of bandwidth.

With the advancements in the video recording technologies, videos with extreme high quality such as 4KHD was introduced to the market. To meet the increasing resolution and output quality the Moving Picture Experts Group (MPEG) proposed HEVC/H.265 as the successor of H.264, which offers 50% better data compression at the same level of video quality and has the potential to support resolutions up to 8K (8,192x4,320) pixels. On January 25, 2013, the technical content of HEVC was finalized, and on April 13, 2013, it was formally declared as a standard.

To reach this extremely high compression, many novel algorithms were developed by the HEVC developers which eventually ended up in patent pools. These patent pools resulted in a high licence cost in HEVC embedded hardware and software, which worked negatively in the 2020 market as it faces a worthy competitor: VP9, which is a loyalty free video codec.

AOM rival

Back in 2010, Google acquired a video compression company that owned Video Processing 8 (VP8), a comparable alternative to H.264. Youtube which is owned by Google later open-sourced VP8 and developed it successor VP9, which is an open and loyalty free video compression codec. VP9 too was able to achieve 50% of bit rate reduction compared to H.264 and it supported 8K resolution. Moreover, Google partnered with companies such as Intel, ARM, Broadcom, Marvell, Nvidia, Qualcom, RealTek, MediaTek, and Sigma for manufacturing VP9 compatible hardware, chips and components.

In 2015, Amazon, Cisco, Google, Intel, Microsoft, Mozilla, and Netflix created the Alliance for Open Media (AOM) which designed another contender for the video compression standard market, named AV1 which use VP9 as the base. With time, AV1 was backed by giants in the industry such as AMD, ARM, NVIDIA , Facebook, Hulu, and Apple.

In spite of AV1’s significant backing, HEVC remains a strong contender largely because of its significant head-start. Yet, both of these codec depends highly on customized hardware at play back. As hardware support upgrades take time, the penetrate of both standards to the markets is still under rated.

Future of Video Compression: With loyalty or without loyalty

Digital media technology is developing at an aggressive rate. Hence, it is essential to continue with the cycle of investment in R&D to meet the demand in the technology.

The next generation of video compression standard HEVC /H.265 is the H.266 or VVC. At IBC 2018, a preliminary implementation based on VVC was demonstrated and the final standard was approved on 6 July 2020. Will H.266 ultimately replace the H.265? It should be noted that H.265 never fully replaced H.264. Hence there is a possibility for a much powerful compression standard to emerge before H.266 is fully adopted in the video compression market.

Further the AOM is looking ahead for AV2 as AV1 is picking up momentum. AV2 might be the one perhaps destined to oust HEVC from the broadcasting domain. High-level content adaptive techniques tested with AV2, has claimed 20% to 40% savings in bandwidth with combination of HEVC and content aware encoding, compared with HEVC alone.

Hence with the improvement in the compression ratio with both developments is highly favourable. Yet, what about the cost of the codec? Every tiny improvement is associated with manpower and critical designs. It takes millions if not billions of dollars of investments. Who will shoulder these expenses? Which will drive the contributors to exploit new innovations in the video compression sector and at what cost? As with any standards, industry adoption will depend on technical merit, ease of implementation and business cases.

All in all, the right question to ask would be who will win the race? The loyalty free codec or high loyalty codec? Only time will tell us…

 

Eng. (Dr.) Maheshi B Dissanayake.

PhD(Surrey, UK), BScEng(Peradeniya, SL)
Senior Lecturer
Department of Electrical and Electronic Engineering,
Faculty of Engineering,
University of Peradeniya, Sri Lanka.