VLCKit with LCEVC
This section provides an overview of how to build and use VLCKit, how VLCKit uses Apple’s VideoToolbox and how LCEVC decoding can be achieved inside VLCKit.
The LCEVC-enabled VLCKit has been extensively tested and validated by third party test house Kineton to ensure that the addition of LCEVC to the player maintained all of its functionality and performance. The results of the tests are available here.
The VLCKit repository can be found in VideoLan’s Gitlab. The vanilla VLCKit is available through CocoaPods, Carthage and you can build it locally on your machine. The VLCKit that is commercially used is not on master branch yet, it’s stored in branch 3.0. For the purposes of this document, we will use the 3.0 branch and build it locally on the machine.
Before starting the build, make sure you have enough storage space (~6GB).
By going into the root directory of VLCKit, you can start the build process with the following commands:
./buildMobileVLCKit.sh -f
You can also specify which architecture you want to build VLCKit for:
./buildMobileVLCKit.sh -a aarch64
Once the build finishes, you can find the MobileVLCKit.framework in the build directory. You can link this framework directly inside XCode:
- 1.Go to Project > Targets > ${YOUR_APPLICATION} > General and add framework to Frameworks, Libraries and Embedded content section.
- 2.Make sure Framework Search Paths and Header Search Paths in Project > Targets > ${YOUR_APPLICATION} > Build Settings can find the .framework and its Headers directory.
VLCKit has a mechanism for automatically applying patches to libvlc (VLC’s core) before starting the build. The required changes for applying LCEVC to VLCKit consist of multiple patches and a static library binary. You just need to copy the patches into Resources/MobileVLCKit/patches folder and start the build. Once the build is finished, integrate the framework just like before.
You also need to embed the lcevc_dec_videotoolbox.a binary into your application. Same as frameworks:
- 1.Go to Project > Targets > ${YOUR_APPLICATION} > General and add framework to Frameworks, Libraries and Embedded content section.
- 2.Make sure Library Search Paths in Project > Targets > ${YOUR_APPLICATION} > Build Settings can find the lcevc_dec_videotoolbox.a file.
VLCKit should now be able to decode LCEVC streams.
In order to use VLCKit, first you need an instance of
VLCMediaPlayer. Then an UIView to set as render target and a VLCMedia instance which will load the asset.PlayerViewController.swift
class PlayerViewController: UIViewController {
var url: URL!
let player = VLCMediaPlayer()
override func viewDidLoad() {
super.viewDidLoad()
setupMediaLibrary()
}
override func viewDidAppear(_ animated: Bool) {
super.viewDidAppear(animated)
player.play()
}
private func setupMediaLibrary() {
player.delegate = self
player.drawable = view
player.media = VLCMedia(url: url)
}
}
extension PlayerViewController: VLCMediaPlayerDelegate {
func mediaPlayerStateChanged(_ aNotification: Notification!) {
if player.state == .stopped {
self.dismiss(animated: true, completion: nil)
}
}
}
In order to work with Apple’s VideoToolbox Decoder module (
VTDecompressionSession), the stream must be demuxed into CMSampleBuffers and fed into VTDecompressionSession for decompression. In response VTDecompressionSession will return CVPixelBuffers for given CMSampleBuffers in decoding order. After reordering CVPixelBuffers to presentation order, they’re ready for presentation on screen.All the logics mentioned earlier are encapsulated into a single module named videtoolbox in VLCs ecosystem. The process is shown below:
In order to apply LCEVC decoding, we need to extract LCEVC payload before decompression, and apply LCEVC once decompression is done and frames are ordered by presentation. The process is shown below:
- 1.
Last modified 1yr ago