提问人:Pete Jones 提问时间:11/16/2023 最后编辑:Pete Jones 更新时间:11/16/2023 访问量:43
广播上传扩展的 ProcessSampleBuffer 受到限制
ProcessSampleBuffer of Broadcast Upload Extension being throttled
问:
我正在开发一个 Xamarin Forms iOS 应用程序,该应用程序通过 WebRTC 使用广播上传扩展流式传输用户的屏幕。它在大多数情况下运行良好,但是当该方法没有太多处理要做时,我看到了异常行为,ProcessSampleBuffer 方法的调用频率较低。代码如下:
public override void ProcessSampleBuffer(CoreMedia.CMSampleBuffer sampleBuffer, RPSampleBufferType sampleBufferType)
{
switch (sampleBufferType)
{
case RPSampleBufferType.Video:
// Handle audio sample buffer
var imageBuffer = sampleBuffer.GetImageBuffer() as CVPixelBuffer;
var orientation = GetOrientation(sampleBuffer);
var newOrientation = RTCVideoRotation.Rotation0;
if (orientation == InterfaceOrientationType.Portrait)
{
newOrientation = RTCVideoRotation.Rotation0;
}
else if (orientation == InterfaceOrientationType.LandscapeLeft)
{
newOrientation = RTCVideoRotation.Rotation180;
}
else if (orientation == InterfaceOrientationType.LandscapeRight)
{
newOrientation = RTCVideoRotation.Rotation270;
}
// UNCOMMENTING THE FOLLOWING 3 LINES MAKES THE METHOD FIRE ON A MORE REGULAR BASIS.
// var ciImage = CIImage.FromImageBuffer(imageBuffer);
// var image = new UIImage(ciImage, 1.0f, UIImageOrientation.Up);
// var jpeg = image.AsJPEG(0.05f);
long timeStampNs = (long)sampleBuffer.PresentationTimeStamp.Seconds * 1000000000;
if (WebRtcService2._videoCapturer != null && WebRtcService2._videoCapturer.Delegate != null)
{
WebRtcService2._videoCapturer.Delegate.Capturer(WebRtcService2._videoCapturer, ConvertToRTCVideoFrame(imageBuffer, timeStampNs, newOrientation));
}
break;
case RPSampleBufferType.AudioApp:
// Handle audio sample buffer for app audio
break;
case RPSampleBufferType.AudioMic:
// Handle audio sample buffer for app audio
break;
}
}
当包含上面注释掉的 3 行以转换为 JPEG 时,该方法会更频繁地调用,每秒多次。使用上面的代码,该方法每分钟左右以 10-20 帧的短时间爆发触发。
为什么会这样?我不想包含 Jpeg 代码,因为它会影响流的性能(帧速率)。
我希望每秒多次调用该方法(当然,当屏幕有要重绘的内容时)的帧速率一致,而无需包含以下代码行:
var ciImage = CIImage.FromImageBuffer(imageBuffer);
var image = new UIImage(ciImage, 1.0f, UIImageOrientation.Up);
var jpeg = image.AsJPEG(0.05f);
答: 暂无答案
评论