VLCKit with LCEVC
1 Introduction
This section provides an overview of how to build and use VLCKit, how VLCKit uses Apple’s VideoToolbox and how LCEVC decoding can be achieved inside VLCKit.
A general overview of VLCKit and its capabilities is available at https://wiki.videolan.org/VLCKit.
The LCEVC-enabled VLCKit has been extensively tested and validated by third party test house Kineton to ensure that the addition of LCEVC to the player maintained all of its functionality and performance. The results of the tests are available here.
2 Integrating VLCKit
The VLCKit repository can be found in VideoLan’s Gitlab. The vanilla VLCKit is available through CocoaPods, Carthage and you can build it locally on your machine. The VLCKit that is commercially used is not on master branch yet, it’s stored in branch 3.0. For the purposes of this document, we will use the 3.0 branch and build it locally on the machine.
2-1 Building VLCKit
Before starting the build, make sure you have enough storage space (~6GB).
2-1-1 Vanilla VLCKit
By going into the root directory of VLCKit, you can start the build process with the following commands:
You can also specify which architecture you want to build VLCKit for:
Once the build finishes, you can find the MobileVLCKit.framework in the build directory. You can link this framework directly inside XCode:
Go to Project > Targets > ${YOUR_APPLICATION} > General and add framework to Frameworks, Libraries and Embedded content section.
Make sure Framework Search Paths and Header Search Paths in Project > Targets > ${YOUR_APPLICATION} > Build Settings can find the .framework and its Headers directory.
For more information on how to integrate frameworks, check Apple’s documentations.
2-1-2 VLCKit + LCEVC
VLCKit has a mechanism for automatically applying patches to libvlc (VLC’s core) before starting the build. The required changes for applying LCEVC to VLCKit consist of multiple patches and a static library binary. You just need to copy the patches into Resources/MobileVLCKit/patches folder and start the build. Once the build is finished, integrate the framework just like before.
You also need to embed the lcevc_dec_videotoolbox.a binary into your application. Same as frameworks:
Go to Project > Targets > ${YOUR_APPLICATION} > General and add framework to Frameworks, Libraries and Embedded content section.
Make sure Library Search Paths in Project > Targets > ${YOUR_APPLICATION} > Build Settings can find the lcevc_dec_videotoolbox.a file.
VLCKit should now be able to decode LCEVC streams.
2-2 Using VLCKit
In order to use VLCKit, first you need an instance of VLCMediaPlayer
. Then an UIView
to set as render target and a VLCMedia
instance which will load the asset.
For more information on how to use VLCKit you can check their examples, documentations and VLC-iOS.
In order to work with Apple’s VideoToolbox Decoder module (VTDecompressionSession
), the stream must be demuxed into CMSampleBuffer
s and fed into VTDecompressionSession
for decompression. In response VTDecompressionSession
will return CVPixelBuffer
s for given CMSampleBuffer
s in decoding order. After reordering CVPixelBuffer
s to presentation order, they’re ready for presentation on screen.
3-1 VLCKit videotoolbox module
All the logics mentioned earlier are encapsulated into a single module named videtoolbox in VLCs ecosystem. The process is shown below:
3-2 VLCKit + LCEVC
In order to apply LCEVC decoding, we need to extract LCEVC payload before decompression, and apply LCEVC once decompression is done and frames are ordered by presentation. The process is shown below:
Last updated