广播上传扩展的 ProcessSampleBuffer 受到限制

ProcessSampleBuffer of Broadcast Upload Extension being throttled

提问人:Pete Jones 提问时间:11/16/2023 最后编辑:Pete Jones 更新时间:11/16/2023 访问量:43

问:

我正在开发一个 Xamarin Forms iOS 应用程序,该应用程序通过 WebRTC 使用广播上传扩展流式传输用户的屏幕。它在大多数情况下运行良好,但是当该方法没有太多处理要做时,我看到了异常行为,ProcessSampleBuffer 方法的调用频率较低。代码如下:

public override void ProcessSampleBuffer(CoreMedia.CMSampleBuffer sampleBuffer, RPSampleBufferType sampleBufferType)
    {

        switch (sampleBufferType)
        {
            case RPSampleBufferType.Video:
          
                    // Handle audio sample buffer
                    var imageBuffer = sampleBuffer.GetImageBuffer() as CVPixelBuffer;


                    var orientation = GetOrientation(sampleBuffer);

                    var newOrientation = RTCVideoRotation.Rotation0;

                    if (orientation == InterfaceOrientationType.Portrait)
                    {
                        newOrientation = RTCVideoRotation.Rotation0;

                    }
                    else if (orientation == InterfaceOrientationType.LandscapeLeft)
                    {
                        newOrientation = RTCVideoRotation.Rotation180;

                    }
                    else if (orientation == InterfaceOrientationType.LandscapeRight)
                    {
                        newOrientation = RTCVideoRotation.Rotation270;

                    }


                    // UNCOMMENTING THE FOLLOWING 3 LINES MAKES THE METHOD FIRE ON A MORE REGULAR BASIS.                     
                    //  var ciImage = CIImage.FromImageBuffer(imageBuffer);
                    //  var image = new UIImage(ciImage, 1.0f, UIImageOrientation.Up);
                    //  var jpeg = image.AsJPEG(0.05f);


                    long timeStampNs = (long)sampleBuffer.PresentationTimeStamp.Seconds * 1000000000;


                    if (WebRtcService2._videoCapturer != null && WebRtcService2._videoCapturer.Delegate != null)
                    {
                        WebRtcService2._videoCapturer.Delegate.Capturer(WebRtcService2._videoCapturer, ConvertToRTCVideoFrame(imageBuffer, timeStampNs, newOrientation));
                    }

                break;
            case RPSampleBufferType.AudioApp:
                // Handle audio sample buffer for app audio
                break;
            case RPSampleBufferType.AudioMic:
                // Handle audio sample buffer for app audio
                break;
        }
    }

当包含上面注释掉的 3 行以转换为 JPEG 时,该方法会更频繁地调用,每秒多次。使用上面的代码,该方法每分钟左右以 10-20 帧的短时间爆发触发。

为什么会这样?我不想包含 Jpeg 代码,因为它会影响流的性能(帧速率)。

我希望每秒多次调用该方法(当然,当屏幕有要重绘的内容时)的帧速率一致,而无需包含以下代码行:

var ciImage = CIImage.FromImageBuffer(imageBuffer);
var image = new UIImage(ciImage, 1.0f, UIImageOrientation.Up);
var jpeg = image.AsJPEG(0.05f);
Xamarin Xamarin.Forms iOS-App-Extension

评论


答: 暂无答案