首先来介绍下h265(HEVC)nal单元头,与h264的nal层相比,h265的nal unit header有两个字节构成,如下图所示:
从图中可以看出hHEVC的nal包结构与h264有明显的不同,hevc加入了nal所在的时间层的ID,取去除了nal_ref_idc,此信息合并到了naltype中,通常情况下F为0,layerid为0,TID为1。
nal单元的类型有如下几种:
- enum NalUnitType
- {
- NAL_UNIT_CODED_SLICE_TRAIL_N = 0,
- NAL_UNIT_CODED_SLICE_TRAIL_R,
-
- NAL_UNIT_CODED_SLICE_TSA_N,
- NAL_UNIT_CODED_SLICE_TLA,
-
- NAL_UNIT_CODED_SLICE_STSA_N,
- NAL_UNIT_CODED_SLICE_STSA_R,
-
- NAL_UNIT_CODED_SLICE_RADL_N,
- NAL_UNIT_CODED_SLICE_DLP,
-
- NAL_UNIT_CODED_SLICE_RASL_N,
- NAL_UNIT_CODED_SLICE_TFD,
-
- NAL_UNIT_RESERVED_10,
- NAL_UNIT_RESERVED_11,
- NAL_UNIT_RESERVED_12,
- NAL_UNIT_RESERVED_13,
- NAL_UNIT_RESERVED_14,
- NAL_UNIT_RESERVED_15, NAL_UNIT_CODED_SLICE_BLA,
- NAL_UNIT_CODED_SLICE_BLA,
- NAL_UNIT_CODED_SLICE_BLANT,
- NAL_UNIT_CODED_SLICE_BLA_N_LP,
- NAL_UNIT_CODED_SLICE_IDR,
- NAL_UNIT_CODED_SLICE_IDR_N_LP,
- NAL_UNIT_CODED_SLICE_CRA,
- NAL_UNIT_RESERVED_22,
- NAL_UNIT_RESERVED_23,
-
- NAL_UNIT_RESERVED_24,
- NAL_UNIT_RESERVED_25,
- NAL_UNIT_RESERVED_26,
- NAL_UNIT_RESERVED_27,
- NAL_UNIT_RESERVED_28,
- NAL_UNIT_RESERVED_29,
- NAL_UNIT_RESERVED_30,
- NAL_UNIT_RESERVED_31,
-
- NAL_UNIT_VPS,
- NAL_UNIT_SPS,
- NAL_UNIT_PPS,
- NAL_UNIT_ACCESS_UNIT_DELIMITER,
- NAL_UNIT_EOS,
- NAL_UNIT_EOB,
- NAL_UNIT_FILLER_DATA,
- NAL_UNIT_SEI,
- NAL_UNIT_SEI_SUFFIX,
- NAL_UNIT_RESERVED_41,
- NAL_UNIT_RESERVED_42,
- NAL_UNIT_RESERVED_43,
- NAL_UNIT_RESERVED_44,
- NAL_UNIT_RESERVED_45,
- NAL_UNIT_RESERVED_46,
- NAL_UNIT_RESERVED_47,
- NAL_UNIT_UNSPECIFIED_48,
- NAL_UNIT_UNSPECIFIED_49,
- NAL_UNIT_UNSPECIFIED_50,
- NAL_UNIT_UNSPECIFIED_51,
- NAL_UNIT_UNSPECIFIED_52,
- NAL_UNIT_UNSPECIFIED_53,
- NAL_UNIT_UNSPECIFIED_54,
- NAL_UNIT_UNSPECIFIED_55,
- NAL_UNIT_UNSPECIFIED_56,
- NAL_UNIT_UNSPECIFIED_57,
- NAL_UNIT_UNSPECIFIED_58,
- NAL_UNIT_UNSPECIFIED_59,
- NAL_UNIT_UNSPECIFIED_60,
- NAL_UNIT_UNSPECIFIED_61,
- NAL_UNIT_UNSPECIFIED_62,
- NAL_UNIT_UNSPECIFIED_63,
- NAL_UNIT_INVALID,
- };
下面接收下fu分组打包方式,fu分组包头格式如下:
fus包头包含了两个字节的payloadhdr,一个字节的fu header,fu header与h264一样,结构如下图,包含开始位(1b)、停止位(1b)、futype(6b)
paylodhdr两个自己的赋值,其实就是把hevc帧数据的nal unit header的naltype替换为49即可,下面是从ffmpeg源码中截取出来的fu打包方式代码片段:
- static void nal_send(AVFormatContext *ctx, const uint8_t *buf, int len, int last_packet_of_frame)
- {
- RTPMuxContext *rtp_ctx = ctx->priv_data;
- int rtp_payload_size = rtp_ctx->max_payload_size - RTP_HEVC_HEADERS_SIZE;
- int nal_type = (buf[0] >> 1) & 0x3F;
-
-
- if (len <= rtp_ctx->max_payload_size)
- {
-
- ff_rtp_send_data(ctx, buf, len, last_packet_of_frame);
-
- }
- else
- {
-
- rtp_ctx->buf[0] = 49 << 1;
- rtp_ctx->buf[1] = 1;
-
-
-
-
- rtp_ctx->buf[2] = nal_type;
-
- rtp_ctx->buf[2] |= 1 << 7;
-
-
- buf += 2;
- len -= 2;
- while (len > rtp_payload_size)
- {
-
- memcpy(&rtp_ctx->buf[RTP_HEVC_HEADERS_SIZE], buf, rtp_payload_size);
- ff_rtp_send_data(ctx, rtp_ctx->buf, rtp_ctx->max_payload_size, 0);
-
- buf += rtp_payload_size;
- len -= rtp_payload_size;
-
-
- rtp_ctx->buf[2] &= ~(1 << 7);
-
- }
-
-
- rtp_ctx->buf[2] |= 1 << 6;
-
-
- memcpy(&rtp_ctx->buf[RTP_HEVC_HEADERS_SIZE], buf, len);
- ff_rtp_send_data(ctx, rtp_ctx->buf, len + 2, last_packet_of_frame);
-
- }
- }
通过rtp发送hevc视频数据,当hevc帧数据大于mtu时,应该进行fu分组发送,从上面代码流程就是对超过max_payload_size数据进行fu分组的流程,这个h264 fu-A很类似,很容易理解。
参考规范:
https://tools.ietf.org/html/draft-ietf-payload-rtp-h265-14
ffmpeg相关代码
https://www.ffmpeg.org/doxygen/2.5/rtpenc__hevc_8c_source.html