From mightychroma Blogger.
Great post by MUX.com
NGCodec Announces AV1 Support and a 2X Performance Improvement in Broadcast Quality Live Video Encoding
NGCodec Demonstrates the Highest Performance, Highest Quality Video Encoders for Live Streaming and a future of mobile Virtual and Augmented Reality at CES 2019. Cloud Encoded Video Unlocks Mobile, Social Streams, AR/VR, and IOT on 5G.
The Intersection of 5G and Emerging Video Standards Makes it Possible to Deliver Live Streams at Broadcast Quality to Any Screen
NGCoded has the highest performance, lowest latency, best visual quality video encoders available and is demonstrating the future of live video streams and mixed reality experiences at CES 2019. If you would like to meet with us, use our online scheduling tool at www.ngcodec.com/ces online or visit our booth at Sands Expo Center,Level 1, Hall G, Booth 52786 at CES 2019.
Video streams are a massive component - 70% - of internet traffic and growing fast. At CES 2019 NGCodec will for the first time have our own booth and will be demonstrating the future of live, interactive internet video. We are announcing our upcoming AV1 support and double density encoding - a 2X performance improvement - coming later this year. We are also demonstrating VR encoding for real time, interactive games at the full resolution and spec of the HTC Vive featuring our latest latency improvements, delivering an ultra low latency video stream with performance and visual quality indistinguishable from the native stream at less than 20Mbits/second.
Today’s Fastest, Highest Visual Quality Video Encoders
Today, NGCodec has the highest performance video encoder available and the only encoders which can provide broadcast quality live streams on H.265/HEVC and VP9. It produces encoded video at the highest quality and lowest bitrate employing the latest video standards. At CES 2019 you can see our new NGCodec developer platform, a small dedicated video encoder for prototyping cloud-scale encoding.
Emerging codecs including H.265/HEVC and VP9 offer higher compression rates with the same or better visual quality and encoding costs. Implementations vary – the compression rate and visual quality of a given encoder can be dramatically different. NGCodec has developed state-of-the-art encoders, using Xilinx FPGAs, providing over 30% better compression with H.265/HEVC and VP9 compared to existing H.264/AVC encoders targeting live workloads. For video streaming, better compression means reduction in bandwidth.
Google, Twitch, and NGCodec recently published a VP9 implementation white paper on hardware based encoding for live VP9 encoding and Twitch has created a pubic test stream as a technology demonstration. You need VP9 decoder support in your browser which is available with Chrome and can be viewed at https://www.twitch.tv/vp9dev online.
“FPGA-based real-time VP9 encoding can deliver at least 25% bitrate savings compared to the highest-quality H.264 encoders deployed in Twitch’s production today.” -- The Twitch Blog, December 2018
A 5G Future - Streaming Interactive Applications to VR/AR Screens
In the future, improving performance in video encoding makes it possible to deliver broadcast quality interactive experiences at modest bitrates to all consumers on all devices. Emerging services will stream experiences from the cloud and cloud edge - virtual reality, augmented reality, and interactive video - with low latency indistinguishable from a local PC.
On emerging 5G networks, users can experience the performance of a high performance computer on lightweight VR and AR glasses with applications processed and encoded to video with low latency in the cloud. NGCodec is working with Ericsson and other industry leaders to demonstrate VR over 5G. In 2018 we provide the first demo of interactive VR on a live 5G network at the Ericsson Silicon Valley Experience Center, encoding video to H.265/HEVC at less than 20Mbits/second using the HTC Vive at full resolution and framerate spec with no perceived latency when compared to use on a desktop PC.
For more information feel free to reach out to firstname.lastname@example.org or feel free to book some time with our online tool www.ngcodec.com/ces at CES 2019.
RTMP was originally developed by Macromedia as method of streaming video, audio and data between their Flash player and a server. Macromedia was acquired by Adobe and though the original Flash product has very much waned in popularity, the RTMP protocol for live streaming is still very popular.
But relatively speaking it is now an old technology and there are newer and more innovative ways of streaming video, one of which is SRT (Secure Reliable Transmission). This is an open source protocol that was originally developed by Haivision and Wowza which now consists of an alliance of over 150 companies, including NGCodec, that are committed to utilising and integrating the SRT protocol. One of the most recent newest members being Microsoft.
The main aim of SRT is to stream with very low latency even over fluctuating networks and protect against packet loss. With an additional benefit of security in terms AES encryption. This is especially important for live streaming over a public network.
But to understand the benefits of SRT and compare it against other emerging video streaming technologies, it is necessary to go back to the underlying IP network protocols underpinning the internet, particularly of TCP/IP versus UDP.
TCP and UDP are protocols used for sending bits of data — known as packets — over the Internet, but they work in different ways.
TCP (Transmission Control Protocol) is most used by everyday internet applications to guarantee delivery of packets through a handshaking mechanism between the sender and the receiver. If packets are not received, then they are resent. Though guaranteeing an authentic delivery of packets, it is very slow and does not optimise on fluctuating networks. RTMP and other HTTP streaming-based protocols, which includes MPEG-DASH and HLS, rely on TCP/IP for handshaking and replacing packets that are lost in transmission. This means there is a potential issue of latency which is not efficient for high performing video streams.
UDP on the other hand does not have a handshaking mechanism. It basically sends the packets and hopes for the best. But in terms of latency it is much reduced and actually becomes a more ideal solution for video streaming. It is possible to use other error correction mechanism with UDP and so this is where SRT starts to look attractive compared to other streaming mechanisms.
SRT uses the UDP protocol and is designed to be utilised over lossy networks to ensure reliability. It does this with the use of a “high-performance” sender and receiver module – one that does not choke the network with handshaking acknowledgements. This allows it to scale and maximize the available bandwidth. SRT guarantees that the packet cadence (compressed video signal) that is sent is identical to the one that is received at the decoder.
SRT adds in other features specifically designed for efficient and secure video streaming: -
· 128/256 AES encryption providing security on a link level over public networks
· Is content agnostic and brings together multiple video, audio, and data (metadata) streams within a single SRT stream, allowing it to easily support highly complex workflows
· Supports both sender and receive modes (in contrast to both RTMP and HTTP that only support a single mode) that is useful for passing firewalls
· Within the sender / receiver module, it is possible to detect network performance and use a form of adaptive bitrate and correction with regards to latency, packet loss and jitter
· Can support current and next generation compression techniques ie H264 (AVC), H265 (HEVC), AV1, VP9
· SRT is open source and royalty free and available on the Github platform
RTMP is not dead by any stretch of the imagination and is well adopted by many CDN and other video management streaming services and will be utilized for many years to come. But given its restrictions in terms of latency and encryption, it is clear with the quality of the alliance it has behind it, SRT will emerge as the next generation video streaming protocol. It is already being deployed to replace RTSP, especially in those high-end next generation live video contribution workflows.
NGCodec is dedicated to providing low latency FPGA video encoding solutions for live encoding and is a member of the SRT alliance. Our new EvalKit will naively support SRT.
Twitch will deploy NGCodec’s VP9 Encoding Solution Starting in Early 2019 Reducing Bandwidth Requirements for High Quality Live Streams by 25%
November 12, 2018. NGCodec, a pioneer in cloud video processing, and Twitch, the leading service and community for multiplayer entertainment, will deploy VP9 encoding to deliver high quality live streams at 25% of the bandwidth of H.264/AVC. Twitch has licensed NGCodec’s encoding solution to produce a high compression-efficiency video on the royalty-free VP9 codec. With this solution, Twitch users stand to receive broadcast quality streams with significantly lower bandwidth requirements.
“VP9 decoding is widely supported by our viewers and the royalty free codec offers high quality streams at much lower bandwidth than the 15 year old H.264/AVC standard,” said Martin Hess, Sr. Director Video Platform at Twitch. “Deploying NGCodec and FPGA-based servers on our private cloud will substantially reduce bandwidth costs while delivering high quality live streams.”
NGCodec has pioneered low latency live video encoding on XilinxⓇ programmable FPGA processors which is the only real-time, broadcast quality live VP9 encoder solution in the market. The NGCodec VP9 RealityCodecTM simultaneously encodes up to 32 multi-resolution streams on a single FPGA compute unit. This dramatic reduction in infrastructure requirements eases the migration to the high-quality, high efficiency VP9 codec.
“The emergence of VP9 is helping to pave the way toward ubiquitous high quality streaming,” said Matt Frost, Director, Product Management, Google Chrome. “Increasing video quality without increasing bandwidth is a win win for users and streaming service providers.”
“We are thrilled by the opportunity to power millions of live, social streams using our latest VP9 encoding solution,” said Oliver Gunasekara, CEO of NGCodec. “Migrating to VP9 has clear technical advantages which our customers can now enjoy with lower infrastructure and bandwidth costs.”
About NGCodec Inc.
NGCodec® has been in passionate pursuit of next generation video compression since 2012. With the support of investors including Xilinx, NGCodec’s agile startup team has created Reality Codec™, a compressor-decompressor technology optimized for ultra-low latency, high-quality applications. Headquartered in Sunnyvale, California, NGCodec leverages FPGA acceleration in the Cloud to lower encoding costs by 10x over traditional CPU encoders. Learn more at www.ngcodec.com@NGCodec
Launched in 2011, Twitch is a global community that comes together each day to create multiplayer entertainment: unique, live, unpredictable experiences created by the interactions of millions. It brings the joy of co-op to everything, from casual gaming to world-class esports to anime marathons, music, and art streams. Twitch also hosts TwitchCon, our biggest brand-led event of the year, where we bring everyone together to celebrate, learn, and grow their personal interests and passions. We’re always live at Twitch. Stay up to date on all things Twitch on Twitterand on our Blog.
I was at Demuxed this year where I spoke on "How to build a high quality 'live' HD VP9 Encoder ". I was joined in the talk by my colleague and hardware expert Nguyen. Demuxed is an annual conference of video engineers held in downtown SFO. This year was a 2-day event and every session was packed to an audience of over 300 people. It was a great experience to meet and interact with talented professionals in the streaming video industry. The event was streamed live on Twitch and you can access the recorded clip.
Over the last year, I have been a part of the NGCodec team focused on implementation of a live VP9 encoder that can deliver compression efficiency similar to HEVC. This was the focus of our talk this year where we described the architecture of an FPGA-based VP9 encoder that can deliver pristine quality video at high resolutions and lower bitrates. Using the coding tools in VP9 coupled with intelligent mode decisions, rate control and adaptive quantization mechanisms, we were able to achieve 30% improvement in compression efficiency over H.264. In the talk, we explained how FPGAs can be used to design a system that can perform heavy computations like motion estimation and complex mode decisions to optimize video quality. This system is software-like with flexible upgrades and yet high density making it deployable on-prem or in the Cloud.
I also had the wonderful opportunity to announce details of my upcoming book on Video Coding. Titled Decode To Encode, the book answers the hows and whys of elements in H.264, H.265 and VP9. It provides video engineers and students the fundamentals of video compression in a simple language with many examples and illustrations. The book will be released by thanksgiving this year and you can have early access by signing up here.
Thanks to the entire #Demuxed2018 team for providing this wonderful platform to interact and share ideas with experts in the streaming industry.
The case for energy saving using FPGA architecture is compelling. If you look at most data centres, 99% of data centers are using Intel Xeon Processors which with software running on standard operating system, just continues to burn power. This could be cut by almost 85% when using FPGA architecture.
So how does this save the polar bears?
To answer this, we need to understand FPGA. And the data center consumption of global power.
What is FPGA architecture?
FPGA stands for Field Programmable Gate Array and is basically a semiconductor chip made up of many silicon layers which sits within its own circuit board. Its function, as in the case of NGCodec with their encoding applications, are hard wired into the chip as firmware. It means a separate software application running on a specific operating system is not required as the FPGA will take over much of the CPU and GPU functionality. But the upshot is is that FPGA boards do not require a host computer to operate, since they have their own input and output interface. This has many advantages in terms of efficiency and performance, but one major advantage is that is utilizes a lot less power. This alone has a huge environmental benefit.
Can using FPGA save the world?
“Scientific evidence for warming of the climate system is unequivocal.”
- Intergovernmental Panel on Climate Change
To put things in prospective, cloud services are hugely reliant on data centers with many of the major internet and IT companies building their own massive infrastructures including Microsoft, Apple, Cisco and Amazon. Data centers in the USA alone use more than 90 billion kilowatt-hours of electricity a year. This requires approximately 34 giant (500-megawatt) coal-powered plants. On a global scale, data centres used roughly 416 terawatts (4.16 x 1014 watts) or about 3% of the total world’s electricity which equates to approximately 40% of the entire electricity consumed by the United Kingdom in 2017. Shockingly this consumption is forecast to double every four years.
In terms of environmental impact, Anders Andrae, a Swedish researcher and Senior Expert Life Cycle Assessment at Huawei is quoted in Climate Change News that the ICT industry is posed to be responsible for up to 3.5% of global emissions by 2020, with this value potentially escalating to 14% by 2040. It is claimed that the data centre sector could be using 20% of all available electricity in the world by 2025 since data is being created at a more accelerated speed than ever before experienced.
Google estimates that a typical search using its search engine needs as much energy as illuminating a light for 17 seconds, and is responsible for emitting 0.2 grams of CO2. It may not sound much, but now think about how many searches you might make in a year and multiple that by the number of people using Google. And Google is data-lite!
Streaming video through the internet is what really increases the data count. IT company Cisco, claims that video will make up 82 percent of internet traffic by 2021, up from 73 percent in 2016. Around a third of internet traffic in North America is already dedicated to streaming Netflix services alone which is based on traditional software encoding.
With additional video services combined with the booming demand of IOT devices, demand will continue to rise dramatically and it is estimated that 50 billion devices will be connected to the internet by 2020. With the addition of driverless cars each with dozens of embedded sensors, and crypto-currencies like Bitcoin needing vast amounts of energy there will be no let up for energy demand for data centers.
The research from Andrae suggests that data centres will be one of the biggest energy consumers on the planet, beating many countries’ energy consumption levels. This could make data centres one of the biggest polluters in just seven years.
Of course the data center industry is not ignoring this and are promoting renewable green energy and starting to place data centers in cooler locations that don’t require a huge amount of air conditioning. It seems strange now that any of the world’s largest centers are in hot or temperate climates, where vast amounts of energy are used just to keep them from overheating.
Almost as important as switching data centers is for them to utilize low energy devices which will help in improving their energy efficiency. This is where FPGA technology can help, especially when deployed in cloud environments where resources can be shared with many disparate customers. This will reduce energy consumption caused by many companies having their own in-house data centers or machine rooms, a major benefit of cloud.
Greenpeace says given the very size of the internet business, and its exposure to criticism for its contribution to climate change means that they need to modify their stance from being part of the problem to being part of the solution, which the industry takes very seriously.
The hope is that they will bring many other giant corporations with them. “The leadership by major internet companies has been an important catalyst among a much broader range of corporations to adopt 100 percent renewable goals,” says Gary Cook, the lead author of a Greenpeace report. “Their actions send an important market signal.”
Gary Cook believes that many companies offering internet services will fully expect to see green labelling for digital sources as routine. The video industry has to move from being energy sappers helping in the demise of the polar ice caps to instead helping save the polar bears from extinction.
But apart from helping save polar bears, there is of course another major advantage of using FPGA. Efficiency. The NGCodec allows full H265 HEVC or VP9 encoding with low bit rates at around 1MB/sec with extremely low latency. But that aside let’s just do our bit to save the planet!
Emerging codecs including H.265/HEVC and VP9 offer higher compression rates with the same or better visual quality and encoding costs. Implementations vary – the compression rate and visual quality of a given encoder can be dramatically different. NGCodec has developed state-of-the-art encoders, using Xilinx FPGAs, providing over 30% better compression with H.265/HEVC and VP9 compared to existing H.264/AVC encoders targeting live workloads. For video streaming, better compression means reduction in bandwidth. We estimate that cost savings from using next-generation codecs range from 15% to 20% even when applied only to the most popular subset of videos or channels. In addition, better visual quality can improve user and content creator satisfaction. This paper provides a quantitative framework for estimating the economic benefit of adopting NGCodec’s H.265/HEVC and VP9 vs legacy H.264/AVC for live streaming.
NGCodec is a Silicon Valley-based startup that develops next-generation Cloud-based live video encoders. Unlike almost all competing encoders, NGCodec does not use software running on X86 CPU’s, instead, NGCodec uses Xilinx’s programmable hardware called Field Programmable Gate Arrays (FPGAs) running in public clouds including AWS, Alibaba, Huawei, Tencent, Baidu etc.
Visual Quality and User Growth are Driving Exponential Bandwidth Costs
The emergence of advanced video codecs promises to improve visual quality and reduce the bandwidth required for real-time video streams. Our customers are demonstrating strong interests in emerging video categories including live game and camera broadcasts combined with community social streams. Over the next four years, global video streaming bandwidth is expected to grow rapidly. Mobile, social streaming is expected to grow at an 18.9% CAGR, and live video and game streaming is growing at an 26.8% CAGR. Cisco predicts that overall live video bandwidth will grow 15-fold between 2016 and 2021.
As live video streams increase in popularity, their bandwidth and encoding requirements become a larger portion of service operating expenses. Successful app-based live streaming services are seeing an increase in annual bandwidth costs exceeding 75% year over year as their services ramp up and they work to improve visual quality. As TVs, PCs and mobile phones increase in resolution, users are increasingly sensitive to poor quality streams. Leading streaming platforms are increasingly offering high bitrate content, with 8-10mbps for leading providers at resolutions of 1080p60 and beyond. The combined result is fast growth in bandwidth and encoding requirements for video services, and a corresponding increase in operating costs.
The primary drivers of cost for a video encoding infrastructure are video encoding computation, IT power consumption and stream size/ bit rate. Advancements in video encoding promise to improve visual quality and reduce bandwidth requirements. Video service providers can reduce costs by reducing the size of streams and through the adoption of efficient video encoders. The performance advantages of H.265/HEVC and VP9 over the 15-year-old H.264/AVC codec can be dramatic. Below, NGCodec used its RealityCodec real-time encoder to create a 1.5 Mbps 1080p30 stream. The stream is high quality and has a surprisingly low bitrate. This is just an example of the dramatic improvement in visual quality and bandwidth sipping potential of next-generation codecs.
x264 H.264/AVC at 3 Mbps
NGCodec H.265/HEVC at 1.5Mbps
Visual quality as measure by VMAF is significantly better using the NGCodec Xilinx FPGA-based hardware encoder spanning the most common bitrates for the next-generation codecs including H.265/HEVC and VP9. This chart measures average results for four varied video clips. The gap between any two lines is the bandwidth savings between those encoding approaches.
The Combined Reach of the H.265/HEVC and VP9 Codecs - Almost 100% of Today’s Client Platforms
The VP9 codec is a royalty-free, open source codec. It is not under the control of a single vendor. VP9’s performance characteristics are similar to the H.265/HEVC standard. VP9 is the precursor to AV1 which is being standardized by the Alliance of Open Media, and already has gained a wide adoption, supported by the entire Android ecosystem and by the Chrome, Firefox, and Edge browsers. Active users on these VP9 platforms represent approximately 70% of all PC and Smartphone users. The similar, but not royalty free H.265/HEVC codec is further supported by all iOS devices for the last few years, giving the combination of H.265/HEVC and VP9 almost 100% client platform support.
Given that the next generation of codecs can reduce bandwidth and improve visual quality, we see that the costs of operating services, including bandwidth costs and even encoding costs might be reduced. These factors are important in translating the performance of the emerging codecs into cost savings calculations and are introduced below:
CLIENTS: The adoption of video clients supporting next-generation codecs. Today, H.265/HEVC and VP9 have support in almost 100% of client platforms.
CONTENT: The distribution of video popularity can impact savings. Most services report that 10% of content accounts for 60% to 90% of bandwidth. So encoding that top content tier with an efficient codec will provide the majority of the bandwidth benefit and cost savings.
ENCODER BIT RATE: The ability of an encoder to provide the same visual quality at a reduced bandwidth requirement forms the best, apples to apples comparison of encoder performance.
ENCODER PERFORMANCE PER WATT: The performance of video encoders to process video streams can vary dramatically. Hardware based encoders are generally more efficient but less flexible. Software based encoders offer more flexibility but generally consume significantly more power. Emerging FPGA and other silicon-based approaches perform encoding with specialized processors designed specifically for higher performance.
Understanding Bandwidth and Encoding Costs For Live Streaming
The formula for calculating the percentage of savings on bandwidth costs can be summarized as a product of the percentage of clients supporting the codec, the percentage of content encoded that is watched, and the relative efficiency of the codec.
Bandwidth Cost Savings = Percent of Clients Supported * Percent of Content Encoded that is Watched * Encoder Bit Rate Savings
The cost savings using a high performance H.265/HEVC and VP9 codec can be dramatic. Today, the NGCodec RealityCodec encoder using H.265/HEVC or VP9 provides 30% savings in bitrate over best-in-class H.264/AVC encoding at the same or better visual quality. It is also currently the highest performance H.265/HEVC and VP9 encoder able to encode H.265/HEVC or VP9 stream at almost exactly the same power consumption per channel as a commercial, high performance H.264/AVC CPU video encoder. The bandwidth cost savings is driven by the higher quality of H.265/HEVC and VP9 relative to H.264/AVC at the same bit rate. For many customers, the 20% most popular live streaming channels, generate 80% or more of the network traffic. This implies that a platform with 80% of the traffic using the top 20% percent of streams, can substantially reduce network traffic by encoding the top 20% of the streams using VP9 or HEVC. For instance, if VP9 with 70% platform coverage were employed to encode only the top 20% of streams for 80% of watched content, the cost reduction would be given by:
16.8% Bandwidth Cost Savings = Percent of Clients Supported [70%] * Percent of Content Encoded that is Watched [80%] * Encoder Bit Rate Savings [30%]
Estimates for bandwidth cost savings range from approximately 15% to 20% as we vary the percent of channels encoded and the platform coverage - H.265/HEVC plus VP9 cover approximately 100% of client platforms. To demonstrate the potential savings, the table below shows costs per Gigabyte ranging from 4 cents to .25 cents and calculates the cost per million concurrent streams at 2 megabits average bandwidth per year. Savings at 16.8% bandwidth reduction range from $52 Million to $3.2 Million per million streams. Regardless of bandwidth costs, by adopting H.265/HEVC and VP9 significant bandwidth cost savings can be achieved.
Let’s take a look at the cost difference between a hardware and software solution. Encoding costs with a hardware encoder are substantially lower than encoding with software. A high end Xeon E5 CPU with a commercial software-based encoder can process approximately 8 1080p60 H.264 channels at approximately 30% higher bitrate achieving “medium” to “low” preset quality. This same server would not be able to encode a single 1080p60 1080p60 stream using HEVC or VP9 low bit rate encoding.
As such, it is impossible to encode real-time HD streams with the Visual Quality achieved by the hardware-based encoder employing H.265/HEVC or VP9 codecs. The NGCodec Xilinx FPGA-based server solution can process 8 channels of 1080p60 H.265/HEVC or VP9 channels at a 30% lower bitrate achieving much better subjective visual quality - beyond H.264/AVC “slow” preset quality. Taking the improved visual quality as a large benefit, the encoding cost of an H.265/HEVC or VP9 solution is much lower than the cost of encoding to H.264/AVC/VP9/HEVC with software encoders.
Conclusion: Save Money While Improving Visual Quality
Creating the next successful video streaming application or operating an established live streaming service is becoming increasingly, and potentially exponentially more expensive. Live video service providers can save millions of dollars in bandwidth cost, thanks to a 30% reduction in bandwidth using hardware video encoding. Employing H.265/HEVC and VP9 for real time streams provides the best possible visual quality and similar encoding costs. Adopting these next generation codecs can improve user experiences while reducing costs, paving the way for future innovation in live streaming video. NGCodec is committed to further bit rate reductions and has a roadmap of VQ improvements which will be achieved through software updates, in addition next year NGCodec will deliver a live AV1 encoder which will allow the investment made in Xilinx FPGA infrastructure to be reused with even further compression gains.
What’s in a codec? HEVC versus VP9
As video resolutions evolve from HD to UHD and beyond to 8K, encoding technologies to compress video also evolve. They must do so to accommodate the larger resolutions as well as the bandwidth and storage requirements of the broadcast and media industry. Today we are experiencing a codec war between two competing standards, H.265/HEVC and VP9. This is a continuation of the previous generation codec battle between H.264/AVC and VP8.
Analysis shows that H.265/HEVC and VP9 are in many practical situations equally efficient, providing similar levels of compression efficiency and visual quality with the same given hardware. So when choosing between the two codecs, pure quality should not be the major differentiator.
In last decade’s battle between H264/AVC and VP8, both standards were widely adopted by encoding products, browsers and players alike - but arguably H.264/AVC won out by being employed on just about every operating system, browser, player and encoding device in existence. Apple’s decision not to implement VP8 may have had a contribution to the success of H.264/AVC, but even so it did not stop other mobile players from supporting VP8.
With the current generation H.265/HEVC and VP9 encoders, there are many reasons why companies may deploy either in their products and services, which we will examine further in this article.
One of the main obstacles delaying H.265/HEVC adoption is the licensing and royalty situation, which is confusing and still subject to uncertainty. Many companies contributed to the development of the H.265/HEVC standard, and as a result there are three patent pools that are demand royalties – MPEG LA, HEVC Advance and Velos Media with a further number of IP holders not in any patent pool. See diagram below by Jonatan Samuelsson from IBC 2018 paper.
Conversely, the VP9 standard is an open source, royalty free format that was developed by Google. Google originally developed VP9 mainly for YouTube content. In fact if you want to utilize YouTube in 4K, you have no choice but to use VP9. Google has licensed a number of patents to provide cover for VP9 and future codecs.
VP9 has a much larger potential user base, as decoding support has been adopted within Firefox, Edge and Chrome, the most popular global browsers. H.265/HEVC has only been adopted by Edge and Safari. Given every Android phone ships with Chrome, the numbers of potential decoders is huge.
For video contribution within the enterprise space, only H.265/HEVC solutions have been adopted by the major manufacturers as the next generation after their H.264/AVC products. However these are specialized solutions used for ingest of streaming video from a remote live location back to a broadcast center, typically used for live remote productions. At the receiving end they are typically encoded again for production.
However for OTT, consumer products, and use in set top boxes, the penetration has been rather different, with VP9 being adopted at least as much if not more than H.265/HEVC:-
There has been little focus in the media industry on H.266/VVC (Versatile Video Coding), considered to be the next encoding standard after H.265/HEVC from MPEG/ITU. Its aim is to give a +30%-bit rate improvement over H.265/HEVC and will lend itself to formats for 4K, 8K and even 16K.
Instead, there is now an industry focus on AV1. This is generally considered to be the successor to VP9 and initial tests are showing a 25% improvement in bit rate. The general feeling is that it could be very widely adopted due to the nature of its open source approach. There are also potential improvements over H.265/HEVC, although this has yet to be fully quantified.
AV1 was initially called VP10, and was being developed by Google as the successor to VP9. Along the way Google decided to throw the development of the new standard open to a larger community, creating the Alliance for Open Media (AOM) which was announced on September 1, 2015, with Google contributing its VP10 program to AOM. The founding members were Amazon, Cisco, Google, Intel Corporation, Microsoft, Mozilla, and Netflix. At the time, the formation consolidated the development of three potentially competitive open source codecs including VP10 but also Cisco's Thor and Mozilla's Daala.
There is some concern about potential litigation around AV1 – Nokia sued Apple in 2016 around patent infringement of H.264/AVC, which was settled out of court. There is no suggestion a lawsuit is imminent with AV1 but there have been some rumblings in the industry, with so much at stake. But despite that, the prospects for AV1 seems positive, especially given the royalty issues around H.265/HEVC. At the end of the day, it’s all about adoption rates. It could take a few years to determine a winner. Nevertheless AV1 has some very large companies behind it.
As an encoding company that specializes in FPGA technology for low bitrate encoding, including VP9 and HEVC, NGCodec firmly believes in a multi codec approach for their customers. By providing the backbone to many cloud based encoding solutions, NGCodec technology is optimized for low bandwidth, high-quality applications for H.264/AVC, H.265/HEVC, VP9, AV1, and will be keeping a close eye on H.266/VVC as part of its future roadmap. So in summary, despite the extra complexity, customers will demand all four codecs: H.264/AVC for legacy devices, H.265/HEVC for premium and enterprise, VP9 for mass market Android support, and AV1 as a possible successor for the whole market.
Available soon in the Huawei Cloud, H.265 Standard from NGCodec Powered by Xilinx FPGAs
SHANGHAI, Oct. 12, 2018 – HUAWEI CONNECT – Xilinx, Inc. (NASDAQ: XLNX), Huawei, and NGCodec, today announced the development of China’s first cloud-based high-efficiency video coding (HEVC) solution, which is uniquely enabled by Xilinx® Virtex UltraScale+™ FPGAs and NGCodec’s H.265 video encoder. This technology will be offered in the HUAWEI CLOUD FACS (FPGA Acceleration Cloud Server) and will be available before the end of the year. This will enable the first broadcast quality real-time streaming solution using FPGA’s in a scalable cloud infrastructure in China.
“The unique combination of Xilinx’s powerful and adaptable FPGA technology with NGCodec’s HEVC encoder are enabling a set of solutions that were previously not possible,” says Sean Gardner, senior marketing manager, Cloud Video, Xilinx. “Huawei is leading the industry by being the first to implement FPGAs for high-definition, real time video streaming in its cloud infrastructure and bring it to China – the world’s largest market for live video streaming.”
Video streaming is a critical workload for China and companies are struggling to roll out infrastructure at the necessary pace to meet rapid growth combined with peak demand. Traditionally, companies have looked towards GPUs as a viable option, but the Xilinx FPGAs running on the HEVC codec offers 35-45 percent lower bitrate at the same visual quality. This reduction delivers a massive improvement in bandwidth and storage costs.
“To grow our share of this exciting, exploding market, service providers must deliver the best end-user experiences. Other technologies were not able to meet our specifications,” says Anders Zhaogang, Huawei FPGA Acceleration Cloud Service technical leader, Cloud BU-Computing Service Domain Program. “Our high-performance streaming solution now makes it possible.”
The HUAWEI CLOUD HEVC solution enables 1080p picture quality at 60 frames per-second, with a roadmap to 120 frames per-second. It offers fast integration into existing video workflows to accelerate core video functions. And like all Xilinx technology, users can update codecs to achieve higher performance. Moreover, Xilinx has implemented its technology using the open-source FFmpeg media framework opening the door for any user to reap the benefits that FPGAs offer.
The HUAWEI CLOUD HEVC solution will be demonstrated today at Huawei Connect, taking place 10-12 October 2018 at the Shanghai World Expo Exhibition and Convention Center, Shanghai, China.
Xilinx develops highly flexible and adaptive processing platforms that enable rapid innovation across a variety of technologies – from the endpoint to the edge to the cloud. Xilinx is the inventor of the FPGA, hardware programmable SoCs and the ACAP, designed to deliver the most dynamic processor technology in the industry and enable the adaptable, intelligent and connected world of the future. For more information, visit www.xilinx.com.
© Copyright 2018 Xilinx, Inc. Xilinx, the Xilinx logo, Xilinx® Virtex UltraScale+™ and other designated brands included herein are trademarks of Xilinx in the United States and other countries. All other trademarks are the property of their respective owners.
It was a fantastic IBC and we are super excited to announce our new products. We demonstrated our live HEVC and VP9 FPGA encoders for OTT encoding on the Advantech Vega 4000 boards. Broadcast quality 1080p encoding at 1Mbps is now a reality thanks to our superior RealityCodec technology.
For customers seeking public cloud solutions, RealityCodec is available today in the Amazon EC2 F1 FPGA-based cloud. For customers seeking on prem the company has announced a Hardware Evaluation Kit for evaluating performance in real-world streaming workflows.
We also launched our brand new encoder evaluation kit that supports 4K HDMI and SDI inputs. It can be ordered today!
Check the video below where Oliver Gunasekara, our CEO explains and showcases the technology.
NGCodec Leverages Accelize’s Digital Right Management (DRM) Platform for Flexible and Secure Deployment of its Next-Generation Video Compression Solutions
September 10, 2018 – Aix-en-Provence, France and Mountain View, CA– Accelize®, a leading provider and enabler of data center workload acceleration using FPGAs,and NGCodec®, a leader in cloud video processing, today announce that they have signed an agreement for NGCodec to leverage Accelize’s Digital Right Management (DRM) Platform to ensure its video accelerator products can be easily deployed and managed in hybrid multi-cloud environments (Public Cloud, Private Cloud, On-premise or a mix of them).
“NGCodec is a pioneer in its field, having been the first company to deliver an accelerator solution on Amazon F1 instances,” said Jean-Yves Brena, CEO of Accelize. “With its FPGA based expertise NGCodec brings significant video encoding performance gains and reduced cost compared to traditional CPU encoders. This kind of innovation is fueling the cloud transformation. By leveraging our industry-proven licensing and metering solution, NGCodec can focus on innovating and rely on our solution for operational management of their FPGA Video Encoders with the right pay-per-use business model.”
Accelize offers FPGA solution providers a multi-level platform to streamline FPGA accelerator licensing and management. The Accelize solution enables implementation of flexible business models, matching data center operators expectations, in a completely secure way when targeting both Cloud and on-premise deployments.
“The partnership with Accelize is critical for NGCodec in the broad deployment of our video encoders in data center environments,” said Oliver Gunasekara, CEO of NGCodec. “Having the best video encoding engine is inconsequential if not delivered with the right business model. Today, our customers want instantaneous access to our technology and the ability to pay only for what they use. Accelize’s DRM platform gives us the flexible, tuned business model we need along with maximum security and protection of our product.”
Accelize’s DRM platform is part of a broader business platform offering available to all companies creating FPGA based acceleration solutions for today’s public cloud, private cloud and on-premise deployments. Accelize enables developers to create value for data center users by focusing their effort on their unique workload acceleration expertise and leveraging Accelize’s industry-proven DRM technology for flexible licensing and monetization across all FPGA platforms in a fully secured way. Leveraging Accelize’s DRM platform, FPGA solution providers can implement innovative business models that are more tightly aligned with needs of their target customers in the Cloud and data centers at large. Business models such as “per-node” (node locked or floating) and metered (time based or data usage based) are natively supported by the Accelize DRM technology which is available for immediate integration in FPGA products and deployment in datacenter infrastructures.
Companies interested in leveraging Accelize’s Business Platforms to bring their products to market in the Cloud and/or on-premise infrastructures should contact Accelize to evaluate how theses platforms apply can be implemented.
About NGCodec Inc.
NGCodec® has been in passionate pursuit of next generation video compression since 2012. With the support of investors including Xilinx, NGCodec’s agile startup team has created Reality Codec™, a compressor-decompressor technology optimized for ultra-low latency, high-quality applications. Headquartered in Sunnyvale, California, NGCodec licenses its H.265/HEVC video codec silicon intellectual property (IP) to semiconductor and equipment manufacturers around the world. Learn more at www.ngcodec.com.
With strong expertise in FPGA development, IP licensing and extensive operational experience serving the Cloud market, Accelize provides ready-to-use FPGA-accelerated libraries through its AccelStore™, and a multi-level platform offering that enables flexible development, licensing and monetization of FPGA products in datacenters. Accelize aims to facilitate efficient, scalable and secured deployment of FPGA-based acceleration solutions in public cloud, private cloud and on-premise infrastructures. For more information, visit www.accelize.com.
Exhibiting at IBC 2018 in Amsterdam on the Ontario, Canada Booth, Hall #2.A41
Demonstrating both H.265/HEVC and VP9 live adaptive bitrate (ABR) video encoding
HEVC is also being shown at the Advantec booth Hall #11.C32 on the VEGA-4000
NGCodec RealityCodec is already powering Hyperscale infrastructures around the world
New hardware Eval Kit available, with 4K HDMI and SDI capture
Efficient video encoding promises to reduce bandwidth requirements and improve video quality for online video. At IBC 2018, NGCodec is demonstrating breakthrough performance on Adaptive Bit Rate encoding for H.265/HEVC and VP9 codecs. The demonstration is powered by RealityCodec, an FPGA based solution for video encoding with the best possible performance per watt and scalability for broadcast-quality 4K UHD video. For customers seeking public cloud solutions, RealityCodec is available today in the Amazon EC2 F1 FPGA-based cloud. For customers seeking on prem the company has announced a Hardware Evaluation Kit for evaluating performance in real-world streaming workflows.
“NGCodec delivers the lowest bitrate for a given visual quality for live VP9 and HEVC with full support for adaptive bitrate encoding,” said Oliver Gunasekara, CEO and Founder, NGCodec. “Ultimately the bitrate reduction of around 30% over H.264 can save millions of dollars in CDN delivery costs and significantly improve the user experience.”
Emerging HEVC and H.265/HEVC solutions can now be implemented for hyperscale video encoding. NGCodec has pioneered low latency video encoding on programmable FPGA processors. In the public cloud, a single Amazon F1 instance powered by NGCodec delivers significant bit rate reduction over an C5 instances running open source encoders. In the private cloud, NGCodec provides the lowest TCO for broadcast-quality live video streaming.
To book a demo and meeting time at IBC please use the widget on: https://ngcodec.com/ibc
About NGCodec Inc.
NGCodec® has been in passionate pursuit of next generation video compression since 2012. With the support of investors including Xilinx, NGCodec’s agile startup team has created Reality Codec™, a compressor-decompressor technology optimized for ultra-low latency, high-quality applications. Headquartered in Sunnyvale, California, NGCodec leverages FPGA acceleration in the Cloud to lower encoding costs by 10x over traditional CPU encoders. The company’s technology is already being adopted for hyperscale video streaming by clients around the world. Learn more at www.ngcodec.com
Backlash to poor quality live-streaming is now a thing. Live events are cornerstones of broadcast, and Amazon’s recent experience with the US Open broadcast in the UK is an example of the challenges of hyperscale live streaming. While consumers may have pushed back on the price, content available, and advertising on cable and satellite, they enjoyed a consistent and reliable video experience. Streaming has broken into live broadcast for headline events, broadening access and opening business models. Yet consumer experiences have been inconsistent, challenged with poor video quality and feature gaps creating newsworthy backlash. As expectations grow for live streaming of major events, technical hurdles including core technology, hyperscaling, and user experience remain.
Live streaming is a fast moving opportunity. CISCO predicts live video will grow at an astounding rate, increasing 15-fold from 2016 to 2021, ultimately accounting for 13 percent of Internet video traffic, with video traffic being 82 percent of all Internet traffic in 2021. Live video sourced from consumers is being curated and broadcast from tens to hundreds of thousands on emerging mobile-social platforms. Streaming broadcasters are picking up sporting events and season coverage. Leagues are rapidly ramping direct access. Comprehensive coverage of complex events is ideal for streaming, allowing viewers to choose the events, stars and matches they want, and seamlessly move between live, delayed, and recorded content.
Video quality matters. Streaming broadcasters are still working to nail the basics of hyperscale broadcast of live events. And consumer expectations are not standing still, with 4K HDR and 360 VR ramping and viewer comparisons to high-quality cable broadcast and VOD streaming. Viewers want click-to-watch simplicity with broadcast quality. And they notice when they have paid for streaming services but those services fail to deliver on simple - but technically challenging - video quality and reliability expectations.
Efficient video encoding promises to reduce bandwidth requirements and improve live stream quality. The emergence of H.265/HEVC and VP9 are making it possible to deliver broadcast quality 2K and 4K video with reasonable bandwidth for recorded content. Unfortunately good software encoders do not run in realtime. Live software encoders supporting H.265/HEVC and VP9 have little or no compression efficiency gains over the legacy H.264/AVC resulting in a huge difference in Visual Quality (VQ) between live and recorded content.
NGCodec has been developing solutions for encoding live streams and has launched cloud and private cloud encoders ready to meet the challenge of broadcast-quality live streaming. At the upcoming International Broadcasting Convention - IBC 2018 - NGCodec is demonstrating HD at 1Mbps using Adaptive Bit Rate encoding with the H.265/HEVC and VP9 codecs. The solution is powered by RealityCodec, an FPGA based solution for video encoding with the best possible performance per watt and scalability for broadcast-quality HD and 4K UHD video.
NGCodec, a pioneer in cloud video processing, is partnering with Chinese FPGA acceleration specialist CTAccel to deliver FPGA-based video encoding solutions in China. CTAccel will be the exclusive agent for NGcodec products in both mainland China and Hong Kong.
CTAccel will offer the next generation NGCodec video encoder, the RealityCodec™ H.265/HEVC encoder, using FPGA hardware acceleration for low latency while maximizing video quality, meeting the highest broadcasting standards.
“Opportunities for cloud video encoding are expanding, but traditional software approaches need massive, expensive CPU resources and cannot deliver the video quality or latency required by emerging applications,” said Oliver Gunasekara, Chief Executive Officer and Founder, NGCodec. “CTAccel is experienced and respected in FPGA acceleration and we look forward to offering state of the art video encoding solutions to customers in China through CTAccel”
CTAccel has offered image processing solutions employing a high-throughput, low latency FPGA-based accelerator with proven value among customers in China.
“CTAccel is very experienced in FPGA-based acceleration. Being NGcodec’s exclusive agent in China, we are making both party’s solutions more complete. This partnership allows both CTAccel and NGcodec to explore new ways to offer our products to a wide range of customers around the world.” says Harry Yu, CEO and Co-Founder of CTAccel.
NGcodec and CTAccel have already collaborated on several projects and are committed to providing the best FPGA-based solutions to customers.
NGCodec® has been in passionate pursuit of next generation video compression since 2012. With the support of investors including Xilinx, NGCodec’s agile startup team has created Reality Codec™, a compressor-decompressor technology optimized for ultra-low latency, high-quality applications. Headquartered in Sunnyvale, California, NGCodec leverages FPGA acceleration in the Cloud to lower encoding costs by 10x over traditional CPU encoders. Learn more at https://ngcodec.com/
CTAccel Ltd. was founded in March 2016 by a team of FPGA experts from Clustertech Ltd. The company has been involved in the development of FPGA based heterogeneous computing solution for the datacenter since 2013. The company’s patented CTAccel Image Processor (CIP) improves the performance and efficiency of image processing in datacenters. Learn more about CTAccel: http://www.ct-accel.com/home-2/
If you live in the UK you can watch the World Cup live via the BBC iPlayer OTT service in 4K and HDR (HLG). This is very cool. You can read all about it here:
The only downside is that they have a very poor live HEVC encoder. From the BBC Blog:
"The distribution to iPlayer will be Main 10 Profile, Level 5.1 HEVC with HLG signalling. This is the same as we used for our Blue Planet II trials. For Blue Planet II we were able to use non real-time encoding. For our World Cup 2018 coverage we must use real-time encoding. The current state of the art for live HEVC encoding means we need to use increased bitrates and reduce the number of representations we can make available. For the World Cup we will be offering:
- 3840x2160p50 36Mbit/s
- 2560x1440p50 16Mbit/s
- 1920x1080p50 10Mbit/s
- 1280x720p50 7Mbit/s
They really need our Chestnut HEVC live encoder. We could reduce the bitrates by ½ to ⅔ with the same quality and generate all the HLS/DASH renditions in parallel.