H265 / HEVC hardware acceleration in #WebRTC

INTRODUCTION AND DESIGN

Here is a satellite view of the native WebRTC stack.

webrtc::CreatePeerConnectionFactory( networkThread, networkThread, signalingThread, audioModule, webrtc::CreateBuiltinAudioEncoderFactory(), webrtc::CreateBuiltinAudioDecoderFactory(), createEncoderFactory(), createDecoderFactory(), Nullptr, Nullptr );
public VideoEncoder createEncoder(VideoCodecInfo input)
EncoderInfo info; // Disable texture support, take I420 or I444 as input info.supports_native_handle = false; info.is_hardware_accelerated = true; info.has_internal_source = false; // external capturer to feed us with frame info.implementation_name = "IntelMediaSDK";// Disable frame-dropper for MSDK. info.has_trusted_rate_controller = true;// Disable SVC / Simulcast for MSDK. info.scaling_settings = VideoEncoder::ScalingSettings::kOff; return info;

Section 1: Adding a codec implementation for H264, VP8 or VP9

Here we do not need to extend internal support for new VideoCodecType, nor for a new RTP payload type. This is the simpler case, and also the case most people are apparently interested in.

modules/video_coding/codecs/h264/h264_encoder_impl.h

Section 2: Add support for new codecs, like H265 and AV1

ADD CODEC SUPPORT — BASE

kVideoCodecVP8, kVideoCodecVP9, kVideoCodecH264, #ifndef DISABLE_H265 kVideoCodecH265, #endif kVideoCodecMultiplex,};
#ifndef DISABLE_H265 struct VideoCodecH265 { bool operator==(const VideoCodecH265& other) const; bool operator!=(const VideoCodecH265& other) const { return !(*this == other); } bool frameDroppingOn; int keyFrameInterval; const uint8_t* vpsData; size_t vpsLen; const uint8_t* spsData; size_t spsLen; const uint8_t* ppsData; size_t ppsLen; }; #endif
union VideoCodecUnion { VideoCodecVP8 VP8; VideoCodecVP9 VP9;  VideoCodecH264 H264;  #ifndef DISABLE_H265 VideoCodecH265 H265; #endif };
class RTC_EXPORT VideoCodec { [...] const VideoCodecVP9& VP9() const;  VideoCodecH264* H264(); const VideoCodecH264& H264() const;  #ifndef DISABLE_H265 VideoCodecH265* H265(); const VideoCodecH265& H265() const; #endif [...]

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Millicast

Millicast

The Fastest Streaming on Earth. Realtime WebRTC CDN built for large-scale video broadcasting on any device with sub-500ms latency.