# VLCKit

## 1 Introduction

This section provides an overview of how to build and use VLCKit, how VLCKit uses Apple’s VideoToolbox and how LCEVC decoding can be achieved inside VLCKit.

A general overview of VLCKit and its capabilities is available at <https://wiki.videolan.org/VLCKit>.

The LCEVC-enabled VLCKit has been extensively tested and validated by third party test house Kineton to ensure that the addition of LCEVC to the player maintained all of its functionality and performance. The results of the tests are available [here](https://www.v-nova.com/wp-content/uploads/dlm_uploads/Validation_and_Certification_for_LCEVC_on_Mobile_Devices_V-Nova_Test_Report.pdf).

![](https://content.gitbook.com/content/II3HaKNLFEqGvuN59zwF/blobs/RAH90AhmZpvIZOkxnGbt/Logo-Kineton-1-wide.png)

## 2 Integrating VLCKit

The VLCKit repository can be found in [VideoLan’s Gitlab](https://code.videolan.org/videolan/VLCKit). The vanilla VLCKit is available through **CocoaPods**, **Carthage** and you can **build it locally** on your machine. The VLCKit that is commercially used is not on master branch yet, it’s stored in [branch 3.0](https://code.videolan.org/videolan/VLCKit/-/tree/3.0). For the purposes of this document, we will use the 3.0 branch and build it locally on the machine.

### 2-1 Building VLCKit

Before starting the build, make sure you have enough storage space (\~6GB).

#### 2-1-1 Vanilla VLCKit

By going into the root directory of VLCKit, you can start the build process with the following commands:

```bash
./buildMobileVLCKit.sh -f
```

You can also specify which architecture you want to build VLCKit for:

```bash
./buildMobileVLCKit.sh -a aarch64
```

Once the build finishes, you can find the **MobileVLCKit.framework** in the **build** directory. You can link this framework directly inside XCode:

1. Go to **Project > Targets > ${YOUR\_APPLICATION} > General** and add framework to **Frameworks, Libraries and Embedded content** section.
2. Make sure **Framework Search Paths** and **Header Search Paths** in **Project > Targets > ${YOUR\_APPLICATION} > Build Settings** can find the *.***framework** and its **Headers** directory.

For more information on how to integrate frameworks, check [Apple’s documentations](https://developer.apple.com/library/archive/documentation/MacOSX/Conceptual/BPFrameworks/Tasks/IncludingFrameworks.html).

#### 2-1-2 VLCKit + LCEVC

VLCKit has a mechanism for automatically applying patches to libvlc (VLC’s core) before starting the build. The required changes for applying LCEVC to VLCKit consist of multiple patches and a static library binary. You just need to copy the patches into **Resources/MobileVLCKit/patches** folder and start the build. Once the build is finished, integrate the framework just like before.

You also need to embed the **lcevc\_dec\_videotoolbox.a** binary into your application. Same as frameworks:

1. Go to **Project > Targets > ${YOUR\_APPLICATION} > General** and add framework to **Frameworks, Libraries and Embedded content** section.
2. Make sure **Library Search Paths** in **Project > Targets > ${YOUR\_APPLICATION} > Build Settings** can find the **lcevc\_dec\_videotoolbox.a** file.

VLCKit should now be able to decode LCEVC streams.

### 2-2 Using VLCKit

In order to use VLCKit, first you need an instance of `VLCMediaPlayer`. Then an `UIView` to set as render target and a `VLCMedia` instance which will load the asset.

{% code title="PlayerViewController.swift" %}

```swift
class PlayerViewController: UIViewController {
    var url: URL!
    let player = VLCMediaPlayer()
    
    override func viewDidLoad() {
        super.viewDidLoad()
        setupMediaLibrary()
    }
    
    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)
        player.play()
    }
    
    private func setupMediaLibrary() {
        player.delegate = self
        player.drawable = view
        player.media = VLCMedia(url: url)
    }
}

extension PlayerViewController: VLCMediaPlayerDelegate {
    func mediaPlayerStateChanged(_ aNotification: Notification!) {
        if player.state == .stopped {
            self.dismiss(animated: true, completion: nil)
        }
    }
}
```

{% endcode %}

For more information on how to use VLCKit you can check their [examples](https://code.videolan.org/videolan/VLCKit/-/tree/master/Examples), [documentations](https://code.videolan.org/videolan/VLCKit/-/tree/master/doc/html) and [VLC-iOS](https://code.videolan.org/videolan/vlc-ios/).

## 3 [Apple’s VideoToolbox](https://developer.apple.com/documentation/videotoolbox)

In order to work with Apple’s VideoToolbox Decoder module (`VTDecompressionSession`), the stream must be demuxed into `CMSampleBuffer`s and fed into `VTDecompressionSession` for decompression. In response `VTDecompressionSession` will return `CVPixelBuffer`s for given `CMSampleBuffer`s in decoding order. After reordering `CVPixelBuffer`s to presentation order, they’re ready for presentation on screen.

### 3-1 VLCKit videotoolbox module

All the logics mentioned earlier are encapsulated into a single module named videtoolbox in VLCs ecosystem. The process is shown below:

![](https://content.gitbook.com/content/II3HaKNLFEqGvuN59zwF/blobs/0FuDwHnR0STZLhxEhjHb/1.png)

### 3-2 VLCKit + LCEVC

In order to apply LCEVC decoding, we need to extract LCEVC payload before decompression, and apply LCEVC once decompression is done and frames are ordered by presentation. The process is shown below:

![](https://content.gitbook.com/content/II3HaKNLFEqGvuN59zwF/blobs/XY6WBdxPuoMsvHarBxrt/2.png)

1.