MediaSource API - Safari 在缓冲区下溢时暂停视频-解网

问：

我正在尝试使用 MediaSource API 将低延迟视频流式传输到浏览器。作为上下文，我通过 WebRTC 数据通道（使用自定义可靠传输协议）接收实时 H.264 视频，在浏览器中多路复用到碎片化的 MP4 容器中，并将此数据提供给 MediaSource API。

在这种情况下，我通常没有足够的数据来提供给 API，因为：1）没有从服务器发送帧，因为没有任何视觉变化，或者 2）网络打嗝导致数据延迟。

在这种“缓冲区下溢”的情况下，我注意到 Safari 和 Chrome 之间的行为差异：

Chrome 会很高兴地等待更多数据到达，并继续从中断的地方播放视频，在 .HTMLVideoElement.currentTimeSourceBuffer

一旦没有更多可用数据，Safari 将暂停视频元素，我必须强制继续播放视频，并将视频更新到接近结束时间。这会导致视频播放出现明显的故障。HTMLVideoElement.playHTMLVideoElement.currentTimeSourceBuffer

有什么方法可以实现在 Chrome 和 Safari 中看到的行为吗？我认为这可能与 Chrome 的 MediaSource 实现中存在的“低延迟”模式有关（我相信在从馈送的视频流推断信息后将其打开） - 有没有办法在 Safari 中触发类似的东西？

请参阅下面的重现代码。在这种情况下，我已经将视频复用为碎片化的 MP4，并且我使用而不是通过 WebRTC 数据通道将其馈送到 MediaSource API，但最终结果是相同的：setInterval

<!DOCTYPE html>
<html>
  <body>
    <p id="text">Click anywhere to start streaming video</p>
    <video autoplay playsinline id="video" width="800" height="450"></video>

    <script>
      /** @type HTMLVideoElement */
      const videoElement = document.getElementById('video');

      /** @type HTMLParagraphElement */
      const textElement = document.getElementById('text');

      async function start() {
        let fileReadPointer = 0;

        // Queue up video fragments to be appended to the SourceBuffer - we
        // can't append them immediately as we must wait until the SourceBuffer
        // has finished updating.
        let fragmentQueue = [];

        // File contents is fragmented MP4 (output from JMuxer) chunks.
        // Each chunk is prefixed with a 4-byte chunk length.
        const file = await fetch('video.dat');
        const fileBytes = await file.arrayBuffer();

        const CODEC_MIME_TYPE = 'video/mp4; codecs="avc1.424028"';

        // Create MediaSource
        const mediaSource = new MediaSource();
        mediaSource.addEventListener('sourceopen', handleMediaSourceOpen);

        // Assign MediaSource to <video> element
        const mediaSourceUrl = URL.createObjectURL(mediaSource);
        videoElement.src = mediaSourceUrl;

        // Source buffer. We will create this later once the MediaSource has opened.
        let sourceBuffer;

        // Read next fragment from data file and append it to the fragment queue.
        function receiveNextFragment() {
          // Each fragment is stored as a 4-byte length header, followed by the
          // actual bytes in the fragment.
          if (fileReadPointer + 4 > fileBytes.byteLength) {
            return;
          }

          // Read length from data file
          const length = (new DataView(fileBytes)).getUint32(fileReadPointer);
          fileReadPointer += 4;

          // Read next bytes from data file
          const data = new Uint8Array(fileBytes, fileReadPointer, length);
          fileReadPointer += length;

          fragmentQueue.push(data);

          // Feed source buffer with the latest data if it's not already
          // updating.
          if (!sourceBuffer.updating) {
            feedSourceBufferFromQueue();
          }
        }

        function feedSourceBufferFromQueue() {
          if (fragmentQueue.length === 0) {
            return;
          }

          const nextData = fragmentQueue[0];
          fragmentQueue = fragmentQueue.slice(1);
          sourceBuffer.appendBuffer(nextData);
        }

        function handleMediaSourceOpen() {
          // Create source buffer
          sourceBuffer = mediaSource.addSourceBuffer(CODEC_MIME_TYPE);

          sourceBuffer.addEventListener('updateend', ev => {
            // If there is any more data waiting to be appended to the
            // SourceBuffer, append it now.
            feedSourceBufferFromQueue();
          });

          // Receive video fragments much more slowly than their duration, to
          // cause buffer underflow.
          //
          // Note that the video will continue to play on Chrome as
          // new frames are received, but will pause on Safari as soon as the
          // buffer runs out.
          setInterval(() => {
            receiveNextFragment();

            // Update browser text to reflect video paused state.
            textElement.innerText = `Video paused: ${videoElement.paused}`;
          }, 100);
        }
      };

      // Start once the user clicks in the document (to avoid autoplay video
      // issues)

      let started = false;

      function handleClick() {
        if (started) return;
        started = true;

        start();
      }

      document.addEventListener('click', handleClick);
    </script>

    <style>
      video {
        border: 1px solid black;
      }
    </style>
  </body>
</html>

Google-Chrome Safari 视频流 webkit 媒体源

答：

4赞 Will 12/1/2022 #1

我找到了答案，即在打开对象后将对象的持续时间显式设置为 +Inf：MediaSource

mediaSource.addEventListener('sourceopen', () => {
    mediaSource.duration = Number.POSITIVE_INFINITY;
});

上一个：TypeError：预览。CaptureStream 不是 Apple 手机中的函数

下一个：素数图表中的文本失真

MediaSource API - Safari 在缓冲区下溢时暂停视频

MediaSource API - Safari pauses video on buffer underflow

评论