AVPlayer with LCEVC

Introduction

This page provides a guide to adding LCEVC-enhanced video decoding capabilities to an iOS application based on the native AVPlayer.

Including the LCEVC Libraries

As a first step you need to add the provided LCEVC libraries and dependencies to your project. If you don't already have access to the LCEVCAVPlayer Github repository please speak to your V-Nova representative or submit a request here. There are several possible approaches to this depending on your preference:

CocoaPods

CocoaPods is a dependency manager for Cocoa projects. For usage and installation instructions, visit their website. To integrate LCEVCAVPlayerKit into your Xcode project using CocoaPods, specify it in your Podfile:

pod "LCEVCAVPlayerKit", :source => 'https://github.com/v-novaltd/LCEVCAVPlayerKit'

Swift Package Manager

The Swift Package Manager is a tool for automating the distribution of Swift code and is integrated into the swift compiler.

Once you have your Swift package set up, adding LCEVCAVPlayerKit as a dependency is as easy as adding it to the dependencies value of your Package.swift. It can also be added via Xcode: more info here

dependencies: [
    .package(url: "https://github.com/v-novaltd/LCEVCAVPlayerKit.git", .upToNextMajor(from: "1.0.025"))
]

Manually

If you prefer not to use any of the aforementioned dependency managers, you can integrate LCEVCAVPlayerKit into your project manually.

Embedded Framework

~$ git clone https://github.com/v-novaltd/LCEVCAVPlayerKit

• Open the new LCEVCAVPlayerKit folder, and drag the LCEVCAVPlayerKit.xcframework & VNova-ffmpeg.xcframework into the Project Navigator of your application's Xcode project. When it asks to copy, please check both files to copy them under your project folder.

• These universal frameworks (xcframework) are dynamic frameworks and have to be embedded and shipped within your app, otherwilse run time error will happen.

Integration

There are two options when it comes to integrating the LCEVC-enhanced player. One provides an out of the box solution but does not allow customization, while the other requires a few extra steps but enables you to customize the player, buttons, look and feel and behavior.

LCEVC Enhancement Pipeline

It is important to keep in mind how the enhancement is applied to the base video. The LCEVC encoded video includes extra information that the LCEVC enhancer can use to improve the quality and resolution of the image once it is rendered.

The result of the enhancement process is a new frame to be displayed. The rendering of the frames is handled by the LCEVCAVPlayerKit libraries which provide a custom UI View to be used to render the video.

Integration Key Components

The LCEVC process has been encapsulated inside a single class to facilitate a simple and rapid integration:

• LCEVCAVPlayer: A subtype of AVPlayer that adds the required interceptions and processes to generate the enhanced video. This class is compatible with the standard components that make use of AVPlayer and adds:

  • outputVideoView: A read-only property that allows at any point to get a reference to the view containing the enhanced version of the video. This view is maintained and created by the player and it can be easily integrated into any existing view.

  • renderIn: This function handles the initialization and configuration of the outputVideoView as a sub-View of the provided parent view, creating the required constraints for it to fill its parent. If the client application has custom layout management this call can be skipped.

  • width: A read-only property returning the width of the enhanced video. Normally this property is only useful for debugging purposes, but it can be essential if the client application needs to manipulate the layout of the application.

  • height: A read-only property returning the width of the enhanced video. Normally this property is only useful for debugging purposes, but it can be essential if the client application needs to manipulate the layout of the application.

Integration QuickStart 1: Using AVPlayerViewController

The LCEVCAVPlayerKit libraries are compatible with apps that use the native iOS Player controls provided by AVFoundation using the AVPlayerViewcontroller. The integration itself has been simplified as much as possible to ensure that minimal effort is required.

Instructions

1. import the frameworks (AVPlayerViewController needs Apple's AVKit framework)

import AVKit
import LCEVCAVPlayerKit

2. Create an instance of LCEVCAVPlayer providing the remote URL of the asset you want to play. This is a subclass of AVPlayer and can be used in any place you would normally use your AVPlayer instance.

let player = LCEVCAVPlayer(url: url)

3. Once that instance is created you can provide it to the standard AVPlayerViewController using its player property and call the renderIn method of LCEVCAVPlayer which will render the enhanced video to the superlayer you provide.

let view = AVPlayerViewController()
view.player = player
self.present(view, animated: true, completion: nil)
player.renderIn(view.contentOverlayView!)

The end result will be that the LCEVC-enhanced video will be displayed with the native iOS player controls provided by AVFoundation.

Integration QuickStart 2: Using LCEVCAVPlayer directly

This option is suitable if you need more control over the playback UI controls or further customization of the player.

Instructions

1. Add an instance of UIView into your application. This view will end up holding the enhanced video. This can be done either programmatically or using storyboards as below:

1. Import the framework

import LCEVCAVPlayerKit

2. In your ViewController, create and maintain an instance of LCEVCAVPlayer providing the URL of the remote stream to be played. This player exposes all the capabilities of the standard AVPlayer with some exceptions that are outlined in the following section of this document.

let player = LCEVCAVPlayer(url: url)

3. Once your view has been created, for example when the viewDidLoad trigger gets called, you can call the addObserver function on the player to monitor and react to the different status changes by using the key path AVPlayer.status:

player.addObserver(self, forKeyPath: #keyPath(AVPlayer.status), options: [.old, .new], context: &playerItemContext)

This will fire a callback for every player status change.

You should also provide the LCEVCAVPlayer with the required rendering target at this point.

player.renderIn(targetView)

The targetView argument refers to the UIView where the LCEVCAVPlayer will insert the output rendering target. It will be configured to stretch to fill its parent. In case that you want to manage the layout manually you can access that view using the outputVideoView property.

1. Once those steps are completed the player is configured and ready to go, the only thing left is to trigger the playback. A good place for that is to use the observer previously defined by catching the status readyToPlay where it is safe to use the default play method of the AVPlayer to start the playback session.

Other Notes and Considerations

• LCEVC AVPlayer makes use of a local web server to intercept requests performed by AVFoundation. This approach is secure because the included server is configured to reject any request that does not originate from the local device.

• The current version of the enhancer internally intercepts and modifies the requested URLs in order to extract the LCEVC enhancement data. If you are currently handling your AVAssets manually (by preloading or storing them in any way) some customization may be required.

• LCEVC AVPlayer does not support AirPlay. However, mirror mode can be used.

• LCEVC AVPlayer does not support FairPlay protected content.