WebRTC M80 release notes announced for the first time some support for AV1 in #webrtc. This is the closest one gets from official announcement of support, but it is also difficult from the one-liner in the release note to understand what it is about and how to use it. This blog post will present some details on video codecs support in libwebrtc, position the m80 release in that scope and provide some visibility on the AV1 support in #webrtc timeline.
I. AV1 — The codec everybody wants
AV1 is the best codec (in terms of compression) out there, and has been used in production for almost 2 years by most of the industry for VOD encoding: Netflix, YouTube, and others. In non-realtime business models, the coding efficiency is the key to reducing costs, and the adoption of the new codec has been very fast as it translates into real cost savings.
For more than two years now, Millicast and CosMo has been at the forefront of ** RealTime** AV1 by participating, as a member of the Alliance for Open Media, to the standardisation of the AV1 RTP Payload. The AV1 RTP payload specification enables usage of the AV1 codec in the Real-Time Transport Protocol (RTP) and by extension, in WebRTC which uses RTP for the media transport layer. More details and a demo can be found in this previous post.
With Medooze as the reference open-source SFU for Real-Time, KITE the de-facto standard test engine for webrtc (test available for most existing webrtc Media Servers and Platforms), already used by Google, Apple, and many others, and Millicast the main Real-Time Streaming PaaS out there, CoSMo is ideally positioned to first bring real-time AV1 to market.
Millicast is also at the forefront of realtime AV1 adoption, integrating this new codec at different points in the system: streaming, of course, but also recording, and server-side Ad-Insertion. Real Time server-side transcoding of H264/HLS ad-content into AV1/WebRTC in collaboration with INTEL, was announced at INTEL’s Visual Cloud event at IBC in 2019.
For those interested in more details, here are various presentation made on the subject over the past 6 months:
Tech Presentation — Focus on RealTime — IIT-RTC 14–16 octobre 2019
http://www.rtc-conference.com/2019/presentation/?hid=759
Industry Presentation with Google & MozillaNAB Streaming Summit NYC 16 octobre 2019
https://www.nabstreamingsummit.com/session/av1-latest/
Tech / R&D Presentation with Akamai, Netflix, Google, Twitch and Facebook @ The 1st AOM Research Symposium, SFO (*)
a. AV1 for WebRTC by Dr Alex, Millicast CTO:
https://www.youtube.com/watch...
b. AV1 for large scale Real-Time Streaming by R. Blakely, Millicast CEO:
(*) All presentations, and all slides here:
https://aomedia.org/aomedia-research-symposium-2019/
II. Media Engine = Code + RTP, but not in that order!
The libwebrtc media engine design allows to inject encoder/decoder implementation at the application level. This is a smart design which allow for example chrome to reuse some of the existing hardware accelerated encoder/decoder they use for the <VIDEO> element into the webrtc stack. It also allows native apps to inject specific encoder/decoders, including system live one (i.e.e directly in java on android, respectively obj-c on iOS). It helps application to differentiate, without modifying the libwebrtc code itself. Bring your own Codec, which was a proposal for WebRTC NV, is already a reality for native apps using libwebrtc. Well, almost.
However, this comes at a price. First, since you cannot inject a new RTP payload, you can only inject Codecs for which there is already RTP payload support available. That is, untill m80, only H264, VP8, and VP9. Then you can only support image format and chroma sampling that are already supported, that means for now 8bits, and 4:2:0. With the addition of VP9 mode-2, to get HDR and 10 bits image, some classes have been added to support 10bits. The design is … improvable, and the chroma sampling is still limited to 4:2:0. One can expect this part of the code to be heavily refactored when the time come to add real HDR, 12 bits, 4:2:2 or 4:4:4 in the future. Of course, there is no roadmap item, bug, or official timeline for that.
With our demo of AV1 Real-Time at CommConUk, we promised to make it open source as fast as possible and in a way that would not endanger the spec process. Our implementation was not fully spec compliant, for example it did not use the aggregation header. We agreed with google that it would be better if they implemented the core, and we helped with the SVC part, the server-side and the testing part.
The announcement on the m80 release note correspond to the delivery of the very first phase: RTP support for AV1. It maps to the RTP payload specification of AOMedia. That should be enough to send and receive non-SVC AV1. However, it is not linked to any encoder or decoder in the libwebrtc stack, so it is not usable as such, neither in chrome nor in libwebrtc.
The next step to be done to complete the implementation of the AOM real-Time group, is the implementation of the RTP dependency descriptor, which enables SVC. This is in progress.
When can I expect AV1 in chrome
This is evidently THE question. Equipped with CoSMo’s injectable AV1 codec, you could use real-time AV1 today in desktop native apps, modified chrome, electron, and the like. Medooze SFU already supports it, KITE already supports it, and it is being tested by a happy few Millicast customers.
In Q1, with the additional the RTP Descriptor, SVC will get enabled, allowing even better quality and resilience. Again, with the CoSMo products, it will be useable right away.
Full support in Chrome requires adding a wrapper around an encoder/decoder for AV1. Adding support in a browser for a new encoder/decoder is extremely difficult. The bug surface is huge, the impacts on performances and battery are equally high, and one need to add support across all platforms the browser wants to support. It is likely not going to happen fast, and at least, nobody is being assigned to it today or for the close future.
IV. Talk to us
Join the Real-Time streaming conversation with Millicast, feel free to schedule a call anytime.
Originally published at https://webrtcbydralex.com on January 27, 2020.
