Carriage implementations

In this section, we will provide a brief description of the carriages’ implementation with a focus on the standards-based solutions, in a few well-known player applications.

AndroidX Media3

V-Nova maintains a public fork of the AndroidX Media3 media player SDK (formerly known as ExoPlayer) in which the default decoder (MediaCodecAdapter) can be replaced by a wrapper (LcevcMediaCodecAdapter) that has the additional functionality to detect and decode LCEVC-encoded media. This fork is available here:

GitHub - v-novaltd/lcevc-androidx-mediaGitHub

In this fork, functionality is also added to support the dual-track delivery mechanism for base and LCEVC enhancement, as described in Section 3.1.2 above. The relative code changes are shown in this commit:

https://github.com/v-novaltd/lcevc-androidx-media/commit/762cf0b4f4c73e9404c36f1972a54a408ffc7d58

MIME type support

In the current AndroidX Media3 integration, logic is implemented to detect when a stream is LCEVC enabled through the lvc1 FourCC code for the dual-track use case. Not yet implemented is the functionality to subsequently open the appropriate MediaCodec using the relevant MIME type (e.g., video/lcevc-avc, video/lcevc-hevc, video/lcevc-vvc).

Going forward, the integration will be updated in the following way:

1. For an LCEVC enabled stream, where the player detects the stream as e.g. lcevc + hevc, it will query MediaCodec for video/lcevc-hevc MIME type support.

2. If MediaCodec reports Yes, then open MediaCodec with video/lcevc-hevc. This will allow integrators to set up an LCEVC decoder accordingly.

3. If MediaCodec reports No, then open MediaCodec with video/hevc as it is today. This will allow backward compatibility across devices that are not yet LCEVC-enabled.

This logic will be added to the AndroidX Media3 integration in the coming weeks.

MP4 dual track support

Media decoding in Android is provided by the MediaCodec class, which interfaces encoding and decoding services implemented in native code. MediaCodec is a one-input-one-output block, therefore, input from two different streams is not supported out of the box. To circumvent this limitation a “joiner” solution has been used. The joiner prepends each sample of the LCEVC Enhancement track to the corresponding sample of the Base track, thus resulting in a single flow of samples, each new sample being the LCEVC sample followed by the base sample, to the reader and ultimately to MediaCodec. The design of the LCEVC NAL units allows a legacy reader (e.g., the MediaCodec implementation of a base decoder) to interpret them as “reserved” or “unspecified” and thus discard them, which results in the joiner not hindering the decoding of the base component.

The above-linked commit reveals all the code changes. In principle, the joiner solution can be described briefly as:

• Modifying the SampleQueue class to:

  • Add an enhancedSampleQueue, default null, as a link to a parallel queue for the samples from the enhancement (LCEVC) track;

  • Add a method attachEnhancement() to allow setting the enhancedSampleQueue;

  • Add a method isEnhancement() to allow a client to know if the queue is of a base or an enhancement;

  • Prepend the data read from the linked enhancement queue, if present, to the data read from the queue in the read() method, effectively performing the join operation;

• Adding logic to recognise the presence of the LCEVC track, implementing the signalling as defined in ISO/IEC 14496-15:2024;

  • LcevcConfig class

  • Parsable Atoms

  • Sbas atom and setting of scalableId (as is of the base track)

• Adding logic to process main and enhanced tracks in MP4, fragmented MP4, HLS segments, and DASH chunks;

• Adding logic to attach the enhancement SampleQueue to the base SampleQueue, based on the presence of tref/sbas signalling in MP4.