且构网

分享程序员开发的那些事...
且构网 - 分享程序员编程开发的那些事

使用AVCaptureSession& amp;翻转相机时进行无缝音频录制AVAssetWriter

更新时间:2023-10-29 18:57:46

嘿,我遇到了同样的问题,并发现在切换相机后,下一帧被推离了适当的位置.此后似乎会移动每一帧,从而导致视频和音频不同步.我的解决方案是在切换相机后将所有错位的帧移动到正确的位置.

Hey I was facing the same issue and discovered that after switching cameras the next frame was pushed far out of place. This seemed to shift every frame after that thus causing the the video and audio to be out of sync. My solution was to shift every misplaced frame to it's correct position after switching cameras.

很抱歉,我的答案将是Swift 4.2

您必须使用AVAssetWriterInputPixelBufferAdaptor才能在指定的演示时间戳记上附加示例缓冲区.

You'll have to use AVAssetWriterInputPixelBufferAdaptor in order to append the sample buffers at a specify presentation timestamp.

previousPresentationTimeStamp是上一帧的显示时间戳,而currentPresentationTimestamp是您猜测当前帧的显示时间戳. maxFrameDistance在测试时效果很好,但是您可以根据自己的喜好进行更改.

previousPresentationTimeStamp is the presentation timestamp of the previous frame and currentPresentationTimestamp is as you guessed the presentation timestamp of the current. maxFrameDistance worked every well when testing but you can change this to your liking.

let currentFramePosition = (Double(self.frameRate) * Double(currentPresentationTimestamp.value)) / Double(currentPresentationTimestamp.timescale)
let previousFramePosition = (Double(self.frameRate) * Double(previousPresentationTimeStamp.value)) / Double(previousPresentationTimeStamp.timescale)
var presentationTimeStamp = currentPresentationTimestamp
let maxFrameDistance = 1.1
let frameDistance = currentFramePosition - previousFramePosition
if frameDistance > maxFrameDistance {
    let expectedFramePosition = previousFramePosition + 1.0
    //print("[mwCamera]: Frame at incorrect position moving from \(currentFramePosition) to \(expectedFramePosition)")

    let newFramePosition = ((expectedFramePosition) * Double(currentPresentationTimestamp.timescale)) / Double(self.frameRate)

    let newPresentationTimeStamp = CMTime.init(value: CMTimeValue(newFramePosition), timescale: currentPresentationTimestamp.timescale)

    presentationTimeStamp = newPresentationTimeStamp
}

let success = assetWriterInputPixelBufferAdator.append(pixelBuffer, withPresentationTime: presentationTimeStamp)
if !success, let error = assetWriter.error {
    fatalError(error.localizedDescription)
}

也请注意-之所以有效,是因为我保持帧速率一致,因此请确保在此过程中您完全控制了捕获设备的帧速率.

Also please note - This worked because I kept the frame rate consistent, so make sure that you have total control of the capture device's frame rate throughout this process.

我在此处使用此逻辑进行了存储库