LCEVC in Android Open-Source Project

Introduction

This page summarizes the steps for using MPEG-5 LCEVC with the Android Open-Source Project (AOSP). Android provides a well-defined framework for media applications to interact with the system at an operating system level, outlined in the diagram below. The Stagefright media playback engine incudes built-in software-based codecs for popular media formats at a native level. Stagefright audio and video playback features include integration with OpenMAX (OMX) codecs, session management, time-synchronized rendering, transport control and DRM. More information is available in the Android Media framework documentation.

V-Nova LCEVC integration fits at this OMX component level.

The approach for AOSP integration is via a plugin for OMXMaster named libstagefright_lcevc_plugin.so, which is loaded via OMXMaster’s API function OMXMaster::addPlugin.

The plugin is an implementation of OMXPluginBase that registers LCEVC specific components held in independent files. The current implementation provides support for enhancement of AVC/h.264 and HEVC encoding and decoding through the following components:

Decoder

  • "OMX.vnova.video.decoder.avc-lcevc"

  • "OMX.vnova.video.decoder.hevc-lcevc"

Encoder

  • "OMX.vnova.video.encoder.avc-lcevc"

  • "OMX.vnova.video.encoder.hevc-lcevc"

With this approach the changes to AOSP are minimal and confined to a standard way to loading new functionality. They can be applied to any fork of the AOSP project through a simple patch provided by V-Nova on its github repository. If you don't already have access to the repository please speak to your V-Nova representative or submit a request here.

This plugin allows the LCEVC components to be distributed and updated easily and separately by the provision of a simple .so which V-Nova provides together with the necessary scripts.

Summary

Installing any AOSP packages involves replacing the operating system on the host devices. This should be done with care by experts as errors could render the device inoperable. The steps to do this are generally:

  1. Download all packages

  2. Prepare device and platform tools

  3. Flash the LCEVC AOSP image onto the host

  4. Install OMX components to host

In most cases, V-Nova will provide a patch for a customer's fork of the AOSP project. Fully built AOSP images are also available.

AOSP image

Patch

The best way to modify your AOSP fork is by applying the OMXMaster modification as a patch provided by V-Nova on its github repository. The patch is non-invasive, minimal and uses the constructs made available by the Stagefright framework to load the necessary media functionality.

Pre-packaged by V-Nova

V-Nova can provide a full image of ASOP already patched with the necessary modifications to load the necessary LCEVC plug-in. To date, V-Nova have added support for a few chipsets, such as the snapdragon-855, Google Pixel 4a (snapdragon-730G) and various emulators.

The AOSP changes, software and steps to install LCEVC components are as agnostic as possible from the underlying chipsets to easily expand to other integrations.

Preparing devices and platform tools

Some devices need more preparation than others, particularly if the host device is a mobile phone. Possible tasks include:

  • Install Google platform tools (ADB and fastboot) on control PC

  • Enable OEM unlocking and USB debugging on the host device

  • Set up USB drivers on control PC (windows 10 users)

  • Unlocking the bootloader

  • Reverting to stock factory operating system

Specific instructions for devices are included in the documents within the packages.

Installing the AOSP image on the device

Prerequisites

  • LCEVC enabled AOSP image

  • Terminal on control PC

  • adb and Fastboot connections to the device

Installing LCEVC enabled image

  • Enter the BootLoader (fastboot) with adb reboot bootloader

  • In the terminal navigate to the location where you saved the ZIP file.

  • Now enter fastboot -w update aosp_<chipset>-img-android-<version>-lcevc-1.zip

  • The device will reboot into the fastbootd prompt. Wait as the fastboot command is still running in the background, it will ask you to press a button to continue.

  • The device will reboot itself and perform a fresh setup of Android, follow it through by Skipping as much as possible.

  • Upon booting the device you'll see an LCEVC Enhanced logo.

LCEVC OMX Components

Prerequisites

  • Python 3.7

  • adb connection to the device

Together with LCEVC OMX components for encoding and decoding, V-Nova provides all of the scripts necessary to install them, check them, uninstall them.

Note: in the paragraphs below we show the examples for a Snapdragon 855 chipset. Steps for other chipsets will be equivalent.

Installing the components

Extract the build of LCEVC OMX components into some location

tar -xzf lcevc-omx-snapdragon-855-<version>-<date>.tar.gz

On Windows use 7Zip or equivalent. Within the package you will find the following directory structure:

doc/ emulator/ scripts/ snapdragon-855/

including documentation, scripts and the actual OMX binaries.‌

Install the python requirements for installation script:

on Linux:

python3 -m venv env source env/bin/activate pip install -r scripts/omx/requirements.txt

on Windows:

python3 -m venv env cd env/scripts/ activate.bat cd ../../ pip install -r scripts/omx/requirements.txt

After making sure requirements are satisfied, installation of the OMX components is done simply using the omx.py script:‌

python3 scripts/omx/omx.py install --host-path [--device <serial>]

  • --host-path is the path to the .tar.gz file with the OMX components

  • --device <serial> - target device if multiple attached

For more information about the syntax of omx.py do:

python3 scripts/omx/omx.py install --help

Checking that components are installed

Make sure the python requirements are installed.

After following the process above, check that the LCEVC OMX components are correctly installed on the system. You can do this using the scripts provided.

python scripts/omx/omx.py info

The output should look something like this (note that the actual checksum will change with releases of the components):

Verifying installation:

> adb shell md5sum /vendor/lib/libstagefright_lcevc_plugin.so 23ae53bd690917cc4fb7e3be5d223cc3 /vendor/lib/libstagefright_lcevc_plugin.so -> OK > adb shell md5sum /vendor/lib/libOMX.vnova.video.encoder.lcevc.so bd1f9931442a0362ea488c213117cd06 /vendor/lib/libOMX.vnova.video.encoder.lcevc.so -> OK > adb shell md5sum /vendor/lib/libOMX.vnova.video.decoder.lcevc.so b5a67d95cd1cccdde4d0a728ef447556 /vendor/lib/libOMX.vnova.video.decoder.lcevc.so -> OK > adb shell md5sum /vendor/etc/media_codecs.xml f4ced8075048178d497ae6b394910259 /vendor/etc/media_codecs.xml -> OK > adb shell md5sum /vendor/etc/media_codecs_lcevc_enc.xml d18c10022385177466af1a6dad1f1ac9 /vendor/etc/media_codecs_lcevc_enc.xml -> OK > adb shell md5sum /vendor/etc/media_codecs_lcevc_dec.xml 600d19dabc11cda832b67bed6147163d /vendor/etc/media_codecs_lcevc_dec.xml -> OK > adb shell md5sum /vendor/bin/lcevc_omx_test 2622ea3acfe9edc4f6e5fd1349a0e890 /vendor/bin/lcevc_omx_test -> OK > adb shell md5sum /vendor/lib/liblcevc_eil.so 7940d978d5e544831e774fc0622ae202 /vendor/lib/liblcevc_eil.so -> OK > adb shell md5sum /vendor/lib/liblcevc_dil.so fd52f83eb88e8492d5d0c1509a3283a1 /vendor/lib/liblcevc_dil.so -> OK > adb shell md5sum /vendor/lib/liblcevc_epi.so eaeadd5312bbe5cf34507e415bbc8aed /vendor/lib/liblcevc_epi.so -> OK > adb shell md5sum /vendor/lib/liblcevc_dpi.so 293ce36461a482a892409fc1c9f34817 /vendor/lib/liblcevc_dpi.so -> OK

Uninstalling components

Make sure the python requirements are installed.

Uninstall OMX components via omx.py

python3 scripts/omx/omx.py uninstall [--device <serial>]

  • --device <serial> - target device if multiple attached

Using the components

V-Nova provides several applications to test and verify encoding and decoding of LCEVC video using the OMX components installed. Currently, these applications are:

  • An Encoder a simple encoding application that allows LCEVC enhancement to be set as well as calculating several useful statistics:

lcevc_erp_android_prod.apk

  • A Decoder a modified version of Android ExoPlayer that uses the LCEVC OMX components:

Exo-OMX-r<version>-OMX-<date>-noExtensions-withPs-release.apk

However, any application on the device can use the LCEVC OMX components through the Android MediaCodec API. The LCEVC OMX components are listed in the media_codecs_lcevc_enc.xml and media_codecs_lcevc_dec.xml files. They can be called explicitly by name using one of many APIs Android make available (e.g createByCodecName) or by providing roles, here are they names:

  • OMX.vnova.video.encoder.avc-lcevc

  • OMX.vnova.video.encoder.hevc-lcevc

  • OMX.vnova.video.decoder.avc-lcevc

  • OMX.vnova.video.decoder.hevc-lcevc

Roles according to their names:

  • video_encoder.avc

  • video_encoder.hevc

  • video_decoder.avc

  • video_decoder.hevc

Creating the module based on name is easy, just specify the name of the component to use and it will be created, this method is recommended. Creating the module by role can be different depending on what other components are installed on the operating system.

Note: It should also be noted that LCEVC modules should currently only be used when LCEVC-enhanced content is available, for example the LCEVC decoder will not be able to decode vanilla AVC/HEVC bitstreams, only LCEVC-enhanced bitstreams.

PreviousEmbedding LCEVC-enabled demo hls.js playerNextLCEVC in FFmpeg

Last updated