在 ARKit 上使用前置摄像头翻转图像-解网

问：

我正在使用 SwiftUI 在 RealityKit 中的前置和后置摄像头之间切换。但是，在使用前置摄像头时，我遇到了拍摄图像显示为镜像的问题。

我认为这导致应用程序无法检测到我在资产下的“AR 资源”中添加的图像。

我怎样才能翻转该图像？

您可以参考下面的捕获图像：func switchCamera()

func switchCamera() {
    
    print("DEBUGG: switchCamera on Model")
    guard var newConfig = arView.session.configuration else {
        return
    }
    
    switch newConfig {
    case is ARWorldTrackingConfiguration:
        newConfig = ARFaceTrackingConfiguration()
    case is ARFaceTrackingConfiguration:
        newConfig = ARWorldTrackingConfiguration()
    default:
        newConfig = ARWorldTrackingConfiguration()
    }
    arView.session.run(newConfig)
}

捕获的图像：

更新解决方案 - 我找到了我的问题的简单解决方案

通过使用两行代码指定应用于视图的转换，我可以达到预期的效果：

对于前置摄像头，我将使用以下代码：

arView.transform = CGAffineTransform(scaleX: -1, y: 1)
对于后置摄像头，我将使用：

arView.transform = CGAffineTransform(scaleX: 1, y: 1)

通过这些更改，我的代码将修改如下：

func switchCamera() {
    
    print("DEBUGG: switchCamera on Model")
    guard var newConfig = arView.session.configuration else {
        return
    }
    
    switch newConfig {
    case is ARWorldTrackingConfiguration:
        newConfig = ARFaceTrackingConfiguration()
        arView.transform = CGAffineTransform(scaleX: -1, y: 1) // Add here
    case is ARFaceTrackingConfiguration:
        newConfig = ARWorldTrackingConfiguration()
        arView.transform = CGAffineTransform(scaleX: 1, y: 1) // Add here
    default:
        newConfig = ARWorldTrackingConfiguration()
    }
    arView.session.run(newConfig)
}

该解决方案适用于后置和前置摄像头：

Swift SwiftUI Arkit RealityKit

func flipImage(buffer: CVPixelBuffer) -> CVPixelBuffer? {
    // 1
    var ci = CIImage(pixelBuffer: buffer)
    // 2
    // scale image by -1 x, making it mirrored horizontally
    ci = ci.transformed(by: CGAffineTransform(scaleX: -1, y: 1))
    // image is moved to the left, so we need to put it back
    ci = ci.transformed(by: CGAffineTransform(translationX: ci.extent.width, y: 0))

    // 3
    // creating a CIContext with Metal device for rendering to new cvpixelbuffer on GPU (making it faster)
    guard let mtlDevice = MTLCreateSystemDefaultDevice() else { return nil }
    // creating new buffer
    guard let newBuffer = createPixelBuffer(width: CVPixelBufferGetWidth(buffer), height: CVPixelBufferGetHeight(buffer), pixelFormat: CVPixelBufferGetPixelFormatType(buffer))
    // creating context
    let context = CIContext(mtlDevice: mtlDevice)
    // rendering our flipped ci to newBuffer
    context.render(ci, to: newBuffer)
    return newBuffer
}

然后在您的照片处理函数中调用此函数（可能的方法）

// get your buffer and cam data
var buffer = <your buffer>
let isFront = <is front camera?>
if isFront {
    // flip
    buffer = flipImage(buffer: buffer) ?? buffer // if it fails fallback to original
}
// handle your buffer

Helper 函数，用于从 CoreMLHelpers 创建空像素缓冲区

fileprivate func metalCompatiblityAttributes() -> [String: Any] {
    let attributes: [String: Any] = [
        String(kCVPixelBufferMetalCompatibilityKey): true,
        String(kCVPixelBufferOpenGLCompatibilityKey): true,
        String(kCVPixelBufferIOSurfacePropertiesKey): [
            String(kCVPixelBufferIOSurfaceOpenGLESTextureCompatibilityKey): true,
            String(kCVPixelBufferIOSurfaceOpenGLESFBOCompatibilityKey): true,
            String(kCVPixelBufferIOSurfaceCoreAnimationCompatibilityKey): true
        ]
    ]
    return attributes
}

func createPixelBuffer(width: Int, height: Int, pixelFormat: OSType) -> CVPixelBuffer? {
    let attributes = metalCompatiblityAttributes() as CFDictionary
    var pixelBuffer: CVPixelBuffer?
    let status = CVPixelBufferCreate(nil, width, height, pixelFormat, attributes, &pixelBuffer)
    if status != kCVReturnSuccess {
        print("Error: could not create pixel buffer", status)
        return nil
    }
    return pixelBuffer
}

在 ARKit 上使用前置摄像头翻转图像

Flip image with front camera on ARKit

评论

评论