FFmpeg （libav） 需要花费大量时间在具有很少变体的主节点上执行avformat_open_input

FFmpeg (libav) is taking a lot of time to execute avformat_open_input over a master with few variants

提问人：forlayo 提问时间：10/9/2023 更新时间：10/10/2023 访问量：74

问：

我正在尝试使用 libav 从 youtube 解复用 HLS 流，到目前为止它有效，但我的问题是调用avformat_open_input需要很多时间，有时甚至 1 分钟。

对 Exoplayer 执行相同的操作（例如），它可以正常工作。我有一种感觉，在将 libav 用于 hls 解复用方面错过了一些重要的东西。

我正在使用的主文件如下所示：

#EXTM3U
#EXT-X-INDEPENDENT-SEGMENTS
#EXT-X-MEDIA:URI="http://localhost:35835/itag/233/mediadata.m3u8",TYPE=AUDIO,GROUP-ID="233",DEFAULT=YES,AUTOSELECT=YES,NAME="Default"
#EXT-X-MEDIA:URI="http://localhost:35835/itag/234/mediadata.m3u8",TYPE=AUDIO,GROUP-ID="234",DEFAULT=YES,AUTOSELECT=YES,NAME="Default"
#EXT-X-STREAM-INF:BANDWIDTH=1354423,CODECS="avc1.4D401F,mp4a.40.2",RESOLUTION=854x480,AUDIO="234",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/231/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=318204,CODECS="avc1.4D4015,mp4a.40.5",RESOLUTION=426x240,AUDIO="233",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/229/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=402807,CODECS="avc1.4D4015,mp4a.40.2",RESOLUTION=426x240,AUDIO="234",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/229/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=811077,CODECS="avc1.4D401E,mp4a.40.2",RESOLUTION=640x360,AUDIO="234",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/230/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=177519,CODECS="avc1.4D400C,mp4a.40.5",RESOLUTION=256x144,AUDIO="233",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/269/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3851098,CODECS="avc1.4D4020,mp4a.40.2",RESOLUTION=1280x720,AUDIO="234",FRAME-RATE=60,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/311/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=6324518,CODECS="avc1.64002A,mp4a.40.2",RESOLUTION=1920x1080,AUDIO="234",FRAME-RATE=60,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/312/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=95792,CODECS="vp09.00.10.08,mp4a.40.5",RESOLUTION=256x144,AUDIO="233",FRAME-RATE=15,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/602/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=170851,CODECS="vp09.00.11.08,mp4a.40.5",RESOLUTION=256x144,AUDIO="233",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/603/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=292033,CODECS="vp09.00.20.08,mp4a.40.5",RESOLUTION=426x240,AUDIO="233",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/604/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=376636,CODECS="vp09.00.20.08,mp4a.40.2",RESOLUTION=426x240,AUDIO="234",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/604/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=716510,CODECS="vp09.00.21.08,mp4a.40.2",RESOLUTION=640x360,AUDIO="234",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/605/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=1145989,CODECS="vp09.00.30.08,mp4a.40.2",RESOLUTION=854x480,AUDIO="234",FRAME-RATE=30,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/606/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=3217654,CODECS="vp09.00.40.08,mp4a.40.2",RESOLUTION=1280x720,AUDIO="234",FRAME-RATE=60,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/612/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=6078208,CODECS="vp09.00.41.08,mp4a.40.2",RESOLUTION=1920x1080,AUDIO="234",FRAME-RATE=60,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/617/mediadata.m3u8
#EXT-X-STREAM-INF:BANDWIDTH=14086878,CODECS="vp09.00.50.08,mp4a.40.2",RESOLUTION=2560x1440,AUDIO="234",FRAME-RATE=60,VIDEO-RANGE=SDR,CLOSED-CAPTIONS=NONE
http://localhost:35835/itag/623/mediadata.m3u8

为了举例说明 EXT-STREAM-INF 链接内容的外观，下面是一个示例：

#EXTM3U
#YT-EXT-CONDENSED-URL:BASE-URI="https://rr6---sn-gqn-h5qs.googlevideo.com/videoplayback/id/189d2f25f6103cad/itag/233/source/youtube/cpn/2TC6rJxXeKRmE8UD/expire/1696647973/ei/xXYgZczEE-idp-oPltGVuAI/ip/2a0c:5a84:f209:1800:b842:b4cf:48d7:ca1c/requiressl/yes/ratebypass/yes/goi/133/sgoap/clen%3D4352437%3Bdur%3D713.642%3Bgir%3Dyes%3Bitag%3D139%3Blmt%3D1677742885047413/hls_chunk_host/rr6---sn-gqn-h5qs.googlevideo.com/mh/M-/mm/31,29/mn/sn-gqn-h5qs,sn-h5qzen7y/ms/au,rdu/mv/m/mvi/6/pl/36/ctier/A/pfa/5/initcwndbps/1466250/hightc/yes/siu/1/vprv/1/playlist_type/DVR/txp/5532434/mt/1696626071/fvip/2/keepalive/yes/fexp/24007246/beids/24350018/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,goi,sgoap,ctier,pfa,hightc,siu,vprv,playlist_type/sig/AGM4YrMwRQIhAIxbCFktH4mxamWq1_maS3YbKeNCpYaOd8JCh0fMaCblAiBFrvxzoCzIjaLpMkxvv6Aw0dSNpYDl_KI-P1aNw7McyQ%3D%3D/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl,initcwndbps/lsig/AK1ks_kwRQIhAOKU632ETDnLoNcjq8c7oB8fd2bxioPtYANy3Bo7LKrvAiBfywDIDKfA4RY7c0CzqpRWOso0X0Gkbx6aLox0EG11KQ%3D%3D/playlist/index.m3u8",PARAMS="begin,len,goap,gosq",PREFIX="s/"
#EXT-X-VERSION:3
#EXT-X-PLAYLIST-TYPE:VOD
#EXT-X-TARGETDURATION:7
#EXTINF:3.1,
https://rr6---sn-gqn-h5qs.googlevideo.com/videoplayback/id/189d2f25f6103cad/itag/233/source/youtube/cpn/2TC6rJxXeKRmE8UD/expire/1696647973/ei/xXYgZczEE-idp-oPltGVuAI/ip/2a0c:5a84:f209:1800:b842:b4cf:48d7:ca1c/requiressl/yes/ratebypass/yes/goi/133/sgoap/clen%3D4352437%3Bdur%3D713.642%3Bgir%3Dyes%3Bitag%3D139%3Blmt%3D1677742885047413/hls_chunk_host/rr6---sn-gqn-h5qs.googlevideo.com/mh/M-/mm/31,29/mn/sn-gqn-h5qs,sn-h5qzen7y/ms/au,rdu/mv/m/mvi/6/pl/36/ctier/A/pfa/5/initcwndbps/1466250/hightc/yes/siu/1/vprv/1/playlist_type/DVR/txp/5532434/mt/1696626071/fvip/2/keepalive/yes/fexp/24007246/beids/24350018/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,goi,sgoap,ctier,pfa,hightc,siu,vprv,playlist_type/sig/AGM4YrMwRQIhAIxbCFktH4mxamWq1_maS3YbKeNCpYaOd8JCh0fMaCblAiBFrvxzoCzIjaLpMkxvv6Aw0dSNpYDl_KI-P1aNw7McyQ%3D%3D/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl,initcwndbps/lsig/AK1ks_kwRQIhAOKU632ETDnLoNcjq8c7oB8fd2bxioPtYANy3Bo7LKrvAiBfywDIDKfA4RY7c0CzqpRWOso0X0Gkbx6aLox0EG11KQ%3D%3D/playlist/index.m3u8/s/0/3100/slices%3D0-62986/0
#EXTINF:4.266666,
https://rr6---sn-gqn-h5qs.googlevideo.com/videoplayback/id/189d2f25f6103cad/itag/233/source/youtube/cpn/2TC6rJxXeKRmE8UD/expire/1696647973/ei/xXYgZczEE-idp-oPltGVuAI/ip/2a0c:5a84:f209:1800:b842:b4cf:48d7:ca1c/requiressl/yes/ratebypass/yes/goi/133/sgoap/clen%3D4352437%3Bdur%3D713.642%3Bgir%3Dyes%3Bitag%3D139%3Blmt%3D1677742885047413/hls_chunk_host/rr6---sn-gqn-h5qs.googlevideo.com/mh/M-/mm/31,29/mn/sn-gqn-h5qs,sn-h5qzen7y/ms/au,rdu/mv/m/mvi/6/pl/36/ctier/A/pfa/5/initcwndbps/1466250/hightc/yes/siu/1/vprv/1/playlist_type/DVR/txp/5532434/mt/1696626071/fvip/2/keepalive/yes/fexp/24007246/beids/24350018/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,goi,sgoap,ctier,pfa,hightc,siu,vprv,playlist_type/sig/AGM4YrMwRQIhAIxbCFktH4mxamWq1_maS3YbKeNCpYaOd8JCh0fMaCblAiBFrvxzoCzIjaLpMkxvv6Aw0dSNpYDl_KI-P1aNw7McyQ%3D%3D/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl,initcwndbps/lsig/AK1ks_kwRQIhAOKU632ETDnLoNcjq8c7oB8fd2bxioPtYANy3Bo7LKrvAiBfywDIDKfA4RY7c0CzqpRWOso0X0Gkbx6aLox0EG11KQ%3D%3D/playlist/index.m3u8/s/3100/4267/slices%3D0-62986/1
h5qs.googlevideo.com/videoplayback/id/189d2f25f6103cad/itag/233/source/youtube/cpn/2TC6rJxXeKRmE8UD/expire/1696647973/ei/xXYgZczEE-idp-oPltGVuAI/ip/2a0c:5a84:f209:1800:b842:b4cf:48d7:ca1c/requiressl/yes/ratebypass/yes/goi/133/sgoap/clen%3D4352437%3Bdur%3D713.642%3Bgir%3Dyes%3Bitag%3D139%3Blmt%3D1677742885047413/hls_chunk_host/rr6---sn-gqn-h5qs.googlevideo.com/mh/M-/mm/31,29/mn/sn-gqn-h5qs,sn-h5qzen7y/ms/au,rdu/mv/m/mvi/6/pl/36/ctier/A/pfa/5/initcwndbps/1466250/hightc/yes/siu/1/vprv/1/playlist_type/DVR/txp/5532434/mt/1696626071/fvip/2/keepalive/yes/fexp/24007246/beids/24350018/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,goi,sgoap,ctier,pfa,hightc,siu,vprv,playlist_type/sig/AGM4YrMwRQIhAIxbCFktH4mxamWq1_maS3YbKeNCpYaOd8JCh0fMaCblAiBFrvxzoCzIjaLpMkxvv6Aw0dSNpYDl_KI-P1aNw7McyQ%3D%3D/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl,initcwndbps/lsig/AK1ks_kwRQIhAOKU632ETDnLoNcjq8c7oB8fd2bxioPtYANy3Bo7LKrvAiBfywDIDKfA4RY7c0CzqpRWOso0X0Gkbx6aLox0EG11KQ%3D%3D/playlist/index.m3u8/s/693833/5034/slices%3D0-640,4202012-4262932/148
#EXTINF:4.3,
https://rr6---sn-gqn-h5qs.googlevideo.com/videoplayback/id/189d2f25f6103cad/itag/233/source/youtube/cpn/2TC6rJxXeKRmE8UD/expire/1696647973/ei/xXYgZczEE-idp-oPltGVuAI/ip/2a0c:5a84:f209:1800:b842:b4cf:48d7:ca1c/requiressl/yes/ratebypass/yes/goi/133/sgoap/clen%3D4352437%3Bdur%3D713.642%3Bgir%3Dyes%3Bitag%3D139%3Blmt%3D1677742885047413/hls_chunk_host/rr6---sn-gqn-h5qs.googlevideo.com/mh/M-/mm/31,29/mn/sn-gqn-h5qs,sn-h5qzen7y/ms/au,rdu/mv/m/mvi/6/pl/36/ctier/A/pfa/5/initcwndbps/1466250/hightc/yes/siu/1/vprv/1/playlist_type/DVR/txp/5532434/mt/1696626071/fvip/2/keepalive/yes/fexp/24007246/beids/24350018/sparams/expire,ei,ip,id,itag,source,requiressl,ratebypass,goi,sgoap,ctier,pfa,hightc,siu,vprv,playlist_type/sig/AGM4YrMwRQIhAIxbCFktH4mxamWq1_maS3YbKeNCpYaOd8JCh0fMaCblAiBFrvxzoCzIjaLpMkxvv6Aw0dSNpYDl_KI-P1aNw7McyQ%3D%3D/lsparams/hls_chunk_host,mh,mm,mn,ms,mv,mvi,pl,initcwndbps/lsig/AK1ks_kwRQIhAOKU632ETDnLoNcjq8c7oB8fd2bxioPtYANy3Bo7LKrvAiBfywDIDKfA4RY7c0CzqpRWOso0X0Gkbx6aLox0EG11KQ%3D%3D/playlist/index.m3u8/s/698867/4300/slices%3D0-640,4202012-4323843/149
[.. a lot more parts ...]
#EXT-X-ENDLIST

注意：

  • 我已经尝试解析 master.m3u8 以过滤流并生成一个新的 master.m3u8，其中包含我想要的流（音频 1 + 视频 1 ）。计时有所改善，但仍然超过 15 秒！
  • 我有一种感觉，avformat_open_input正在读取/打开/解析每个 EXT-X-STREAM-INF 或类似的东西中的所有部分，否则这是没有意义的。不知道如何解决这个问题。
  • 我已经尝试过一些选项，例如 http_multiple 或 http_seekable，结果没有任何变化。

有什么想法吗？

视频 ffmpeg http-live-streaming m3u8 libav

评论

0赞 Gyan 10/9/2023
libavformat 将需要发现主播放列表中引用的所有变体流的流编解码器参数。这可能意味着从所有变体播放列表中获取前几个片段。
0赞 forlayo 10/9/2023
正如我评论的那样，我创建了一个只有 3 个视频 + 1 个音频的 master.m1u8，并得到了相同的结果。有什么方法可以避免或限制这种情况吗？我知道我想使用哪些流。

答：

1赞 forlayo 10/10/2023 #1

我找到了一种方法，可以大大减少avformat_open_input打开流、获取流信息等的时间。

诀窍是：

  • 解析 m3u8 文件以获取段的 uri。
  • 创建一个 avio 上下文来读取这些区段。
  • 在致电avformat_open_input之前请av_probe_input_buffer。

现在，花了我 20 秒甚至有时超过一分钟的相同视频在不到 1 秒的时间内打开。

在这里，你有一个快速而肮脏的示例来演示解决方案，使用 libav + hlparse：

    #define AVIO_CTX_BUFSIZE 1024 * 32

static int avioContextRead(void *context, uint8_t *buf, int bufSize)
{
    HlsSegmentsParser *demuxer = (HlsSegmentsParser *)context;
    if (demuxer == nullptr)
    {
        return 0;
    }
    return demuxer->readBuffer(buf, bufSize);
}

int HlsSegmentsParser::readBuffer(uint8_t *buf, int bufSize)
{
    while(true){
        if(cur_segment == nullptr){
            cur_segment = segments->data;
            qDebug()<<"AVFORMAT: Reading segment:"<<cur_segment->sequence_num;
            if (avio_open2(&segment_avio_context, cur_segment->uri, AVIO_FLAG_READ, NULL, NULL) < 0) {
                // Errol;
                return -1;
            }
        }

        int bytesRead = avio_read(segment_avio_context, buf, bufSize);
        if(bytesRead<0)
        {
            qDebug()<<"AVFORMAT: Finished segment:"<<cur_segment->sequence_num;
            avio_close(segment_avio_context);
            segment_avio_context = nullptr;

            cur_segment = nullptr;
            segments = segments->next;
            continue;
        }

        return bytesRead;
    }
}

HlsSegmentsParser::HlsSegmentsParser()
{
    // Inicializa libavformat
    avformat_network_init();
}

int HlsSegmentsParser::init(const std::string &filePath)
{

    m_formatContext = avformat_alloc_context();
    m_avioCtxBuffer = static_cast<uint8_t *>(av_malloc(AVIO_CTX_BUFSIZE));
    m_avioContext = avio_alloc_context(m_avioCtxBuffer, AVIO_CTX_BUFSIZE, 0, this, &avioContextRead, nullptr, nullptr);

    auto *formatContext = (AVFormatContext *)m_formatContext;
    formatContext->pb = static_cast<AVIOContext *>(m_avioContext);
    formatContext->flags |= AVFMT_FLAG_CUSTOM_IO;


    auto content = readURLContent(filePath.c_str());
    qDebug()<<"AVFORMAT: Content: " << content.c_str();

    master_t master_playlist;
    if (HLS_OK == hlsparse_master_init(&master_playlist)) {
         if (hlsparse_master(content.c_str(), content.length(), &master_playlist)) {

            auto videoUri = master_playlist.stream_infs.data->uri;
            auto stream_contnet = readURLContent(videoUri);

            qDebug()<<"AVFORMAT: StreamContent: " << stream_contnet.c_str();
            media_playlist_t media_playlist;
            if(HLS_OK == hlsparse_media_playlist_init(&media_playlist)){
                if(hlsparse_media_playlist(stream_contnet.c_str(), stream_contnet.length(), &media_playlist)){
                    segments = &media_playlist.segments;
                    qDebug() << "AVFORMAT: Opening Input (video): " << videoUri;
                }
            }

        }
    }

    int ret = 0;

    qDebug()<<"AVFORMAT: av_probe_input_buffer: ";
    const AVInputFormat * fmt = nullptr;
    ret = av_probe_input_buffer(m_avioContext, &fmt,  nullptr, nullptr, 0 , 0);
    if (ret < 0)
    {
        qDebug() << "AVFORMAT: Unable to av_probe_input_buffer: " << ret << av_err2str(ret);
    }
    qDebug()<<"AVFORMAT: av_probe_input_buffer done! ";

    qDebug()<<"AVFORMAT: avformat_open_input: ";
    ret = avformat_open_input(&formatContext, nullptr, fmt, nullptr);
    if (ret < 0)
    {
        qDebug() << "AVFORMAT: Unable to avformat_open_input: " << ret << av_err2str(ret);
    }

    qDebug()<<"AVFORMAT: avformat_open_input done! ";


    qDebug()<<"AVFORMAT: avformat_find_stream_info" ;
    ret = avformat_find_stream_info(formatContext, nullptr);
    if (ret < 0)
    {
        qDebug() << "AVFORMAT: Unable to avformat_find_stream_info: " << ret << av_err2str(ret);
    }

    qDebug()<<"AVFORMAT: avformat_find_stream_info done!";

    const AVCodec *vcodec = nullptr;
    auto m_vStreamIdx = av_find_best_stream(formatContext,
                                       AVMEDIA_TYPE_VIDEO,
                                       -1,
                                       -1,
                                       &vcodec,
                                            0);


    qDebug()<<"AVFORMAT: m_vStreamIdx"<<m_vStreamIdx;

    AVCodecParameters *codecParam = formatContext->streams[m_vStreamIdx]->codecpar;
    AVRational fpsRatio = formatContext->streams[m_vStreamIdx]->avg_frame_rate;

    m_width = codecParam->width;
    m_height = codecParam->height;
    m_vCodec = codecParam->codec_id;

    qDebug()<<"AVFORMAT: VCodec "<<avcodec_get_name(codecParam->codec_id)<<m_width<<"x"<<m_height<<fpsRatio.den<<fpsRatio.num;

    return -1;
}

std::string HlsSegmentsParser::readURLContent(const char* url) {
    AVIOContext *avioCtx = NULL;
    uint8_t buffer[8192];
    int bytesRead;
    std::string content;

    if (avio_open2(&avioCtx, url, AVIO_FLAG_READ, NULL, NULL) < 0) {
        fprintf(stderr, "No se pudo abrir la URL.\n");
        avformat_network_deinit();
        return "";
    }

    do {
        bytesRead = avio_read(avioCtx, buffer, sizeof(buffer));
        if (bytesRead > 0) {
            content.append(reinterpret_cast<char*>(buffer), bytesRead);
        }
    } while (bytesRead > 0);

    avio_close(avioCtx);
    return content;
}

这将产生如下输出：

2023-10-09 23:00:36.228  D  AVFORMAT: Opening Input (video):  http://localhost:42257/itag/311/mediadata.m3u8
2023-10-09 23:00:36.228  D  AVFORMAT: av_probe_input_buffer: 
2023-10-09 23:00:36.228  D  AVFORMAT: Reading segment: 0
2023-10-09 23:00:36.423  D  AVFORMAT: av_probe_input_buffer done! 
2023-10-09 23:00:36.424  D  AVFORMAT: avformat_open_input: 
2023-10-09 23:00:36.425  D  AVFORMAT: avformat_open_input done! 
2023-10-09 23:00:36.425  D  AVFORMAT: avformat_find_stream_info
2023-10-09 23:00:36.454  D  AVFORMAT: avformat_find_stream_info done!
2023-10-09 23:00:36.454  D  AVFORMAT: m_vStreamIdx 0
2023-10-09 23:00:36.454  D  AVFORMAT: VCodec  h264 1280 x 720 1001 60000

上一个：带有 mediacodec 表面渲染的 FFMpeg （libav） hls demux 播放速度太快

下一个：使用 QSV 和 ffmpeg、libav 进行颜色转换和缩放的硬件加速