LCEVC in Android Open-Source Project
Last updated
Last updated
This page summarizes the steps for using MPEG-5 LCEVC with the Android Open-Source Project (AOSP). Android provides a well-defined framework for media applications to interact with the system at an operating system level, outlined in the diagram below. The Stagefright media playback engine incudes built-in software-based codecs for popular media formats at a native level. Stagefright audio and video playback features include integration with OpenMAX (OMX) codecs, session management, time-synchronized rendering, transport control and DRM. More information is available in the Android Media framework documentation.
V-Nova LCEVC integration fits at this OMX component level.
The approach for AOSP integration is via a plugin for OMXMaster named libstagefright_lcevc_plugin.so, which is loaded via OMXMaster’s API function OMXMaster::addPlugin.
The plugin is an implementation of OMXPluginBase that registers LCEVC specific components held in independent files. The current implementation provides support for enhancement of AVC/h.264 and HEVC encoding and decoding through the following components:
Decoder
"OMX.vnova.video.decoder.avc-lcevc"
"OMX.vnova.video.decoder.hevc-lcevc"
Encoder
"OMX.vnova.video.encoder.avc-lcevc"
"OMX.vnova.video.encoder.hevc-lcevc"
With this approach the changes to AOSP are minimal and confined to a standard way to loading new functionality. They can be applied to any fork of the AOSP project through a simple patch provided by V-Nova on its github repository. If you don't already have access to the repository please speak to your V-Nova representative or submit a request here.
This plugin allows the LCEVC components to be distributed and updated easily and separately by the provision of a simple .so which V-Nova provides together with the necessary scripts.
Installing any AOSP packages involves replacing the operating system on the host devices. This should be done with care by experts as errors could render the device inoperable. The steps to do this are generally:
Download all packages
Prepare device and platform tools
Flash the LCEVC AOSP image onto the host
Install OMX components to host
In most cases, V-Nova will provide a patch for a customer's fork of the AOSP project. Fully built AOSP images are also available.
The best way to modify your AOSP fork is by applying the OMXMaster modification as a patch provided by V-Nova on its github repository. The patch is non-invasive, minimal and uses the constructs made available by the Stagefright framework to load the necessary media functionality.
V-Nova can provide a full image of ASOP already patched with the necessary modifications to load the necessary LCEVC plug-in. To date, V-Nova have added support for a few chipsets, such as the snapdragon-855, Google Pixel 4a (snapdragon-730G) and various emulators.
The AOSP changes, software and steps to install LCEVC components are as agnostic as possible from the underlying chipsets to easily expand to other integrations.
Some devices need more preparation than others, particularly if the host device is a mobile phone. Possible tasks include:
Install Google platform tools (ADB and fastboot) on control PC
Enable OEM unlocking and USB debugging on the host device
Set up USB drivers on control PC (windows 10 users)
Unlocking the bootloader
Reverting to stock factory operating system
Specific instructions for devices are included in the documents within the packages.
Prerequisites
LCEVC enabled AOSP image
Terminal on control PC
adb and Fastboot connections to the device
Installing LCEVC enabled image
Enter the BootLoader (fastboot) with adb reboot bootloader
In the terminal navigate to the location where you saved the ZIP file.
Now enter fastboot -w update aosp_<chipset>-img-android-<version>-lcevc-1.zip
The device will reboot into the fastbootd prompt. Wait as the fastboot command is still running in the background, it will ask you to press a button to continue.
The device will reboot itself and perform a fresh setup of Android, follow it through by Skipping as much as possible.
Upon booting the device you'll see an LCEVC Enhanced logo.
Prerequisites
Python 3.7
adb connection to the device
Together with LCEVC OMX components for encoding and decoding, V-Nova provides all of the scripts necessary to install them, check them, uninstall them.
Note: in the paragraphs below we show the examples for a Snapdragon 855 chipset. Steps for other chipsets will be equivalent.
Extract the build of LCEVC OMX components into some location
tar -xzf lcevc-omx-snapdragon-855-<version>-<date>.tar.gz
On Windows use 7Zip or equivalent. Within the package you will find the following directory structure:
doc/
emulator/
scripts/
snapdragon-855/
including documentation, scripts and the actual OMX binaries.
Install the python requirements for installation script:
on Linux:
python3 -m venv env
source env/bin/activate
pip install -r scripts/omx/requirements.txt
on Windows:
python3 -m venv env
cd env/scripts/
activate.bat
cd ../../
pip install -r scripts/omx/requirements.txt
After making sure requirements are satisfied, installation of the OMX components is done simply using the omx.py script:
python3 scripts/omx/omx.py install --host-path [--device <serial>]
--host-path is the path to the .tar.gz file with the OMX components
--device <serial> - target device if multiple attached
For more information about the syntax of omx.py do:
python3 scripts/omx/omx.py install --help
Make sure the python requirements are installed.
After following the process above, check that the LCEVC OMX components are correctly installed on the system. You can do this using the scripts provided.
python scripts/omx/omx.py info
The output should look something like this (note that the actual checksum will change with releases of the components):
Verifying installation:
> adb shell md5sum /vendor/lib/libstagefright_lcevc_plugin.so
23ae53bd690917cc4fb7e3be5d223cc3 /vendor/lib/libstagefright_lcevc_plugin.so -> OK
> adb shell md5sum /vendor/lib/libOMX.vnova.video.encoder.lcevc.so
bd1f9931442a0362ea488c213117cd06 /vendor/lib/libOMX.vnova.video.encoder.lcevc.so -> OK
> adb shell md5sum /vendor/lib/libOMX.vnova.video.decoder.lcevc.so
b5a67d95cd1cccdde4d0a728ef447556 /vendor/lib/libOMX.vnova.video.decoder.lcevc.so -> OK
> adb shell md5sum /vendor/etc/media_codecs.xml
f4ced8075048178d497ae6b394910259 /vendor/etc/media_codecs.xml -> OK
> adb shell md5sum /vendor/etc/media_codecs_lcevc_enc.xml
d18c10022385177466af1a6dad1f1ac9 /vendor/etc/media_codecs_lcevc_enc.xml -> OK
> adb shell md5sum /vendor/etc/media_codecs_lcevc_dec.xml
600d19dabc11cda832b67bed6147163d /vendor/etc/media_codecs_lcevc_dec.xml -> OK
> adb shell md5sum /vendor/bin/lcevc_omx_test
2622ea3acfe9edc4f6e5fd1349a0e890 /vendor/bin/lcevc_omx_test -> OK
> adb shell md5sum /vendor/lib/liblcevc_eil.so
7940d978d5e544831e774fc0622ae202 /vendor/lib/liblcevc_eil.so -> OK
> adb shell md5sum /vendor/lib/liblcevc_dil.so
fd52f83eb88e8492d5d0c1509a3283a1 /vendor/lib/liblcevc_dil.so -> OK
> adb shell md5sum /vendor/lib/liblcevc_epi.so
eaeadd5312bbe5cf34507e415bbc8aed /vendor/lib/liblcevc_epi.so -> OK
> adb shell md5sum /vendor/lib/liblcevc_dpi.so
293ce36461a482a892409fc1c9f34817 /vendor/lib/liblcevc_dpi.so -> OK
Make sure the python requirements are installed.
Uninstall OMX components via omx.py
python3 scripts/omx/omx.py uninstall [--device <serial>]
--device <serial> - target device if multiple attached
V-Nova provides several applications to test and verify encoding and decoding of LCEVC video using the OMX components installed. Currently, these applications are:
An Encoder a simple encoding application that allows LCEVC enhancement to be set as well as calculating several useful statistics:
lcevc_erp_android_prod.apk
A Decoder a modified version of Android ExoPlayer that uses the LCEVC OMX components:
Exo-OMX-r<version>-OMX-<date>-noExtensions-withPs-release.apk
However, any application on the device can use the LCEVC OMX components through the Android MediaCodec API. The LCEVC OMX components are listed in the media_codecs_lcevc_enc.xml and media_codecs_lcevc_dec.xml files. They can be called explicitly by name using one of many APIs Android make available (e.g createByCodecName) or by providing roles, here are they names:
OMX.vnova.video.encoder.avc-lcevc
OMX.vnova.video.encoder.hevc-lcevc
OMX.vnova.video.decoder.avc-lcevc
OMX.vnova.video.decoder.hevc-lcevc
Roles according to their names:
video_encoder.avc
video_encoder.hevc
video_decoder.avc
video_decoder.hevc
Creating the module based on name is easy, just specify the name of the component to use and it will be created, this method is recommended. Creating the module by role can be different depending on what other components are installed on the operating system.
Note: It should also be noted that LCEVC modules should currently only be used when LCEVC-enhanced content is available, for example the LCEVC decoder will not be able to decode vanilla AVC/HEVC bitstreams, only LCEVC-enhanced bitstreams.
