Google

747 6th St S Kirkland WA 98033 USA justin@uberti.name

RAI example This document provides information and requirements for how Forward Error Correction (FEC) should be used by WebRTC implementations.

Introduction In situations where packet loss is high, or perfect media quality is essential, Forward Error Correction (FEC) can be used to proactively recover from packet losses. This specification provides guidance on which FEC mechanisms to use, and how to use them, for WebRTC implementations.

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Types of FEC FEC describes the sending of redundant information in an outgoing packet stream so that information can still be recovered even in the face of packet loss. There are multiple ways this can be accomplished for RTP media streams ; this section enumerates the various mechanisms available and describes their tradeoffs.

Separate FEC Stream This approach, as described in , Section 4.3, sends FEC packets as an independent RTP stream with its own synchronization source (SSRC, ) and payload type, multiplexed with the primary encoding. While this approach can protect multiple packets of the primary encoding with a single FEC packet, each FEC packet will have its own IP+UDP+RTP+FEC header, and this overhead can be excessive in some cases, e.g., when protecting each primary packet with a FEC packet. This approach allows for recovery of entire RTP packets, including the full RTP header.

Redundant Encoding This approach, as described in , allows for redundant data to be piggybacked on an existing primary encoding, all in a single packet. This redundant data may be an exact copy of a previous payload, or for codecs that support variable-bitrate encodings, possibly a smaller, lower-quality representation. In certain cases, the redundant data could include encodings of multiple prior audio frames. Since there is only a single set of packet headers, this approach allows for a very efficient representation of primary + redundant data. However, this savings is only realized when the data all fits into a single packet (i.e. the size is less than a MTU). As a result, this approach is generally not useful for video content. As described in , Section 4, this approach cannot recover certain parts of the RTP header, including the marker bit, CSRC information, and header extensions.

Codec-Specific In-band FEC Some audio codecs, notably Opus and AMR , support their own in-band FEC mechanism, where redundant data is included in the codec payload. This is similar to the redundant encoding mechanism described above, but as it adds no additional framing, it can be slightly more efficient. For Opus, audio frames deemed important are re-encoded at a lower bitrate and appended to the next payload, allowing partial recovery of a lost packet. This scheme is fairly efficient; experiments performed indicate that when Opus FEC is used, the overhead imposed is only about 20-30%, depending on the amount of protection needed. Note that this mechanism can only carry redundancy information for the immediately preceding audio frame; as such the decoder cannot fully recover multiple consecutive lost packets, which can be a problem on wireless networks. See , Section 2.1.7, and this Opus mailing list post for more details. For AMR/AMR-WB, packets can contain copies or lower-quality encodings of multiple prior audio frames. See , Section 3.7.1 for details on this mechanism. In-band FEC mechanisms cannot recover any of the RTP header.

FEC for Audio Content The following section provides guidance on how to best use FEC for transmitting audio data. As indicated in below, FEC should only be activated if network conditions warrant it, or upon explicit application request.

Recommended Mechanism When using variable-bitrate codecs without an internal FEC, redundant encoding (as described in ) with lower-fidelity version(s) of the previous packet(s) is RECOMMENDED. This provides reasonable protection of the payload with only moderate bitrate increase, as the redundant encodings can be significantly smaller than the primary encoding. When using the Opus codec, use of the built-in Opus FEC mechanism is RECOMMENDED. This provides reasonable protection of the audio stream against individual losses, with minimal overhead. Note that, as indicated above, the built-in Opus FEC only provides single-frame redundancy; if multi-packet protection is needed, the aforementioned redundant encoding with reduced-bitrate Opus encodings SHOULD be used instead. When using the AMR/AMR-WB codecs, use of their built-in FEC mechanism is RECOMMENDED. This provides slightly more efficient protection of the audio stream than redundant encoding. When using constant-bitrate codecs, e.g., PCMU , redundant encoding MAY be used, but this will result in a potentially significant bitrate increase, and suddenly increasing bitrate to deal with losses from congestion may actually make things worse. Because of the lower packet rate of audio encodings, usually a single packet per frame, use of a separate FEC stream comes with a higher overhead than other mechanisms, and therefore is NOT RECOMMENDED. As mentioned above, the recommended mechanisms do not allow recovery of parts of the RTP header that may be important in certain audio applications, e.g., CSRCs and RTP header extensions like those specified in and . Implementations SHOULD account for this and attempt to approximate this information, using an approach similar to those described in , Section 4, and , Section 5.

Negotiating Support Support for redundant encoding of a given RTP stream SHOULD be indicated by including audio/red as an additional supported media type for the associated m= section in the SDP offer . Answerers can reject the use of redundant encoding by not including the audio/red media type in the corresponding m= section in the SDP answer. Support for codec-specific FEC mechanisms are typically indicated via "a=fmtp" parameters. For Opus, a receiver MUST indicate that it is prepared to use incoming FEC data with the "useinbandfec=1" parameter, as specified in . This parameter is declarative and can be negotiated separately for either media direction. For AMR/AMR-WB, support for redundant encoding, and the maximum supported depth, are controlled by the 'max-red' parameter, as specified in , Section 8.1. Receivers MUST include this parameter, and set it to an appropriate value, as specified in , Table 6.3.

FEC for Video Content The following section provides guidance on how to best use FEC for transmitting video data. As indicated in below, FEC should only be activated if network conditions warrant it, or upon explicit application request.

Recommended Mechanism Video frames, due to their size, often require multiple RTP packets. As discussed above, a separate FEC stream can protect multiple packets with a single FEC packet. In addition, the "flexfec" FEC mechanism described in is also capable of protecting multiple RTP streams via a single FEC stream, including all the streams that are part of a BUNDLE group. As a result, for video content, use of a separate FEC stream with the flexfec RTP payload format is RECOMMENDED. To process the incoming FEC stream, the receiver can demultiplex it by SSRC, and then correlate it with the appropriate primary stream(s) via the CSRC(s) present in the RTP header of flexfec repair packets, or the SSRC field present in the FEC header of flexfec retransmission packets.

Negotiating Support Support for a SSRC-multiplexed flexfec stream to protect a given RTP stream SHOULD be indicated by including one of the formats described in , Section 5.1.2, as an additional supported media type for the associated m= section in the SDP offer . As mentioned above, when BUNDLE is used, only a single flexfec repair stream will be created for each BUNDLE group, even if flexfec is negotiated for each primary stream. Answerers can reject the use of SSRC-multiplexed FEC, by not including the offered FEC formats in the corresponding m= section in the SDP answer. Use of FEC-only m-lines, and grouping using the SDP group mechanism as described in , Section 4.1 is not currently defined for WebRTC, and SHOULD NOT be offered. Answerers SHOULD reject any FEC-only m-lines, unless they specifically know how to handle such a thing in a WebRTC context (perhaps defined by a future version of the WebRTC specifications).

FEC for Application Content WebRTC also supports the ability to send generic application data, and provides transport-level retransmission mechanisms to support full and partial (e.g. timed) reliability. See for details. Because the application can control exactly what data to send, it has the ability to monitor packet statistics and perform its own application-level FEC, if necessary. As a result, this document makes no recommendations regarding FEC for the underlying data transport.

Implementation Requirements To support the functionality recommended above, implementations MUST be able to receive and make use of the relevant FEC formats for their supported audio codecs, and MUST indicate this support, as described in . Use of these formats when sending, as mentioned above, is RECOMMENDED. The general FEC mechanism described in SHOULD also be supported, as mentioned in . Implementations MAY support additional FEC mechanisms if desired, e.g., .

Adaptive Use of FEC Because use of FEC always causes redundant data to be transmitted, and the total amount of data must remain within any bandwidth limits indicated by congestion control and the receiver, this will lead to less bandwidth available for the primary encoding, even when the redundant data is not being used. This is in contrast to methods like RTX or flexfec's retransmission mode ( , Section 1.1.7), which only transmit redundant data when necessary, at the cost of an extra roundtrip and thereby increased media latency. Given this, WebRTC implementations SHOULD prefer using RTX or flexfec retransmissions instead of FEC when the connection RTT is within the application's latency budget, and otherwise SHOULD only transmit the amount of FEC needed to protect against the observed packet loss (which can be determined, e.g., by monitoring transmit packet loss data from RTCP Receiver Reports ), unless the application indicates it is willing to pay a quality penalty to proactively avoid losses. Note that when probing bandwidth, i.e., speculatively sending extra data to determine if additional link capacity exists, FEC data SHOULD be used as the additional data. Given that extra data is going to be sent regardless, it makes sense to have that data protect the primary payload; in addition, FEC can typically be applied in a way that increases bandwidth only modestly, which is necessary when probing. When using FEC with layered codecs, e.g., , where only base layer frames are critical to the decoding of future frames, implementations SHOULD only apply FEC to these base layer frames. Finally, it should be noted that although applying redundancy is often useful in protecting a stream against packet loss, if the loss is caused by network congestion, the additional bandwidth used by the redundant data may actually make the situation worse, and can lead to significant degradation of the network.

Security Considerations In the WebRTC context, FEC is specifically concerned with recovering data from lost packets; any corrupted packets will be discarded by the SRTP decryption process. Therefore, as described in , Section 10, the default processing when using FEC with SRTP is to perform FEC followed by SRTP at the sender, and SRTP followed by FEC at the receiver. This ordering is used for all the SRTP Protection Profiles used in DTLS-SRTP , which are enumerated in , Section 4.1.2. Additional security considerations for each individual FEC mechanism are enumerated in their respective documents.

IANA Considerations This document requires no actions from IANA.

Acknowledgements Several people provided significant input into this document, including Bernard Aboba, Jonathan Lennox, Giri Mandyam, Varun Singh, Tim Terriberry, Magnus Westerlund, and Mo Zanaty.

References Normative References 3GPP Informative References This specification defines a new Session Description Protocol (SDP) Grouping Framework extension, 'BUNDLE'. The extension can be used with the SDP Offer/Answer mechanism to negotiate the usage of a single transport (5-tuple) for sending and receiving media described by multiple SDP media descriptions ("m=" sections). Such transport is referred to as a BUNDLE transport, and the media is referred to as bundled media. The "m=" sections that use the BUNDLE transport form a BUNDLE group. This specification updates RFC 3264, to also allow assigning a zero port value to a "m=" section in cases where the media described by the "m=" section is not disabled or rejected. This specification updates RFC 5888, to also allow an SDP 'group' attribute to contain an identification-tag that identifies a "m=" section with the port set to zero. This specification defines a new RTP Control Protocol (RTCP) source description (SDES) item and a new RTP header extension that can be used to correlate bundled RTP/RTCP packets with their appropriate "m=" section. This specification updates RFC 7941, by adding an exception, for the MID RTP header extension, to the requirement regarding protection of an SDES RTP header extension carrying an SDES item for the MID RTP header extension. The WebRTC framework specifies protocol support for direct interactive rich communication using audio, video, and data between two peers' web-browsers. This document specifies the non-media data transport aspects of the WebRTC framework. It provides an architectural overview of how the Stream Control Transmission Protocol (SCTP) is used in the WebRTC context as a generic transport service allowing WEB-browsers to exchange generic data from peer to peer. Xiph

Change log Changes in draft -10:

• Additional editorial changes from IETF LC.

Changes in draft -09:

• Editorial changes from IETF LC.
• Added new reference for Opus FEC.

Changes in draft -08:

• Switch to RFC 8174 boilerplate.

Changes in draft -07:

• Clarify how bandwidth management interacts with FEC.
• Make 3GPP reference normative.

Changes in draft -06:

• Discuss how multiple streams can be protected by a single FlexFEC stream.
• Discuss FEC for bandwidth probing.
• Add note about recovery of RTP headers and header extensions.
• Add note about FEC/SRTP ordering.
• Clarify flexfec demux text, and mention retransmits.
• Clarify text regarding offers/answers.
• Make RFC2198 support SHOULD strength.
• Clean up references.

Changes in draft -05:

• No changes.

Changes in draft -04:

• Discussion of layered codecs.
• Discussion of RTX.
• Clarified implementation requirements.
• FlexFEC MUST -> SHOULD.
• Clarified AMR max-red handling.
• Updated references.

Changes in draft -03:

• Added overhead stats for Opus.
• Expanded discussion of multi-packet FEC for Opus.
• Added discussion of AMR/AMR-WB.
• Removed discussion of ssrc-group.
• Referenced the data channel doc.
• Referenced the RTP/RTCP RFC.
• Several small edits based on feedback from Magnus.

Changes in draft -02:

• Expanded discussion of FEC-only m-lines, and how they should be handled in offers and answers.

Changes in draft -01:

• Tweaked abstract/intro text that was ambiguously normative.
• Removed text on FEC for Opus in CELT mode.
• Changed RFC 2198 recommendation for PCMU to be MAY instead of NOT RECOMMENDED, based on list feedback.
• Explicitly called out application data as something not addressed in this document.
• Updated flexible-fec reference.

Changes in draft -00:

• Initial version, from sidebar conversation at IETF 90.