The DIL (Decoder Integration Layer) is the V-Nova library used to integrate LCEVC decoding and rendering to an existing legacy decoder implementation.
The DIL can be used in both:
Application integration - the DIL renders directly onto an on-screen surface, provided by the client, of arbitrary size (generally different from the content resolution). Examples: ExoPlayer (Android).
OS Integration - the DIL decodes onto a buffer or draws onto an off screen texture of the same size of the content final resolution, it doesn’t handle the final render to display, including YUV to RGB conversion, and resizing to the destination surface. Examples: MFT (Windows), OMX (Android).
The following block diagram illustrates the role of the DIL more clearly. In conjunction with a base decoder, typically provided by the OS, it offers a complete solution from buffer to output.
DIL Types and Enumerations
DIL_NALFormatenum
DIL_NALFormat represents the two possible NAL Unit formats, for each of the MPEG base codecs the DIL currently supports: H264 (ISO/IEC 14496-10, aka AVC) and H265 (ISO/IEC 23008-2, aka HEVC) NAL, where Annex B indicates 3 byte 0x000001 or 4 byte 0x00000001 start code prefix as described in Annex B of (H264 or HEVC), LP (Length Prefix) indicates a 4 byte prefix with the length of the NAL unit as MSBF 32 bit unsigned int.
DIL_ReturnCode represents the possible API return codes. Most of the API calls will return one of these values.
typedef enum DIL_ReturnCode
{
DIL_RC_Success = 0, /* The API call completed successfully */
DIL_RC_Error = -1, /* A generic catch all error */
DIL_RC_InvalidParam = -2, /* The user supplied an invalid parameter to the function call */
DIL_RC_NotFound = -3, /* The query call failed to find the item by name, but didn't not generate an error. */
DIL_RC_NotSupported = -4, /* The functionality requested is not supported on the running system. */
DIL_RC_Flushed = -5, /* The requested operation failed because it was flushed via DIL_Flush. */
DIL_RC_Timeout = -6 /* The requested operation failed because it timedout. */
} DIL_ReturnCode;
typedef enum DIL_ColorFormat
{
DIL_UnknownColorFormat = 0, /* Unspecified colour format */
DIL_I420_8, /* 8 bit 4:2:0 YUV Planar colour format: Y, U and V on separate planes */
DIL_NV12_8, /* 8 bit 4:2:0 YUV SemiPlanar colour format: Y plane and UV interleaved plane */
DIL_RGB_8, /* 8 bit Interleaved R, G, B planes 24 bit per sample */
DIL_RGBA_8, /* 8 bit Interleaved R, G, B and A planes 32 bit per sample */
DIL_I420_10LE, /* 10 bit Little Endian 4:2:0 YUV Planar colour format: Y, U and V on separate planes */
} DIL_ColorFormat;
DIL_ImageType enum
This enum identifies the type of planes the DIL_Image consists of; planes are non-overlapping sets of samples belonging to one or more colour channels with a constant sample stride, so RGB is a single plane, because R G and B pointed at separately would overlap UV from NV12 is a single plane, because U and V pointed at separately would overlap Y U and V from I420 are three planes because they do not overlap
typedef enum DIL_ImageType
{
DIL_UnknownImageType = 0, /* Unspecified image type */
DIL_Buffer_Array, /* One buffer per plane, even if image is on a single memory allocation, an array of planes shall be set by assigning the right pointers to plane buffers. DIL allocated buffers have contiguous memory allocation */
DIL_Texture_Array, /* One texture per plane, typically in YUV colour formats, it consists of a single plane, even for YUV, if GL YUV extension is used */
DIL_HardwareBuffer_Array /* One HardwareBuffer per plane, typically in YUV colour formats, it consists of a single plane, even for YUV, if GL YUV extension is used */
} DIL_ImageType;
DIL_ImageDesc struct
Holds configurations of a single DIL_Image instance
typedef struct DIL_ImageDesc
{
bool is_managed; /* True means that Image (and its memory) is managed by DIL */
DIL_ImageType image_type; /* Whether it is a buffer array or a texture array */
DIL_ColorFormat color_format; /* Colour format */
uint32_t width; /* Nominal net width of the image in luma samples, i.e. no alignment to macroblocks or striding */
uint32_t height; /* Nominal net height of the image in luma samples, i.e. no sliceheight or padding */
bool can_modify; /* Specifying wether content of image is readOnly or changable */
bool can_resize; /* Specifying wether allocated memory is resizable or not. (Does not apply to DIL_Buffer_External) */
uint32_t options; /* options used in creating the Image, in the case of HardwareBuffers this is a combination of DIL_HardwareBufferUsageOption */
} DIL_ImageDesc;
DIL_ContextSettings struct
This structure holds parameters related to the window surface the DIL would render into, in case of on screen rendering those parameters are normally provided by the windowing system, e.g. EGL, GLFW.
typedef struct DIL_ContextSettings
{
void* gl_external_window; /* Surface window, from the windowing system */
void* gl_external_context; /* Context, from the windowing system */
bool is_secure; /* True if the client has initialised window and context as secure for protected content */
} DIL_ContextSettings;
DIL_RenderInformation struct
This structure captures the properties related to the (on screen) window rendering, they do not apply for buffer or texture output (off screen). NOTE: rotation is a per frame property because it may change frame by frame due to for ex. the encoding device rotating while shooting.
typedef struct DIL_RenderInformation
{
uint32_t rotation; /* Degrees of rotation for the final rendering on the destination window, in 90 unit increments clockwise */
float pixel_aspect_ratio_num; /* Pixel Aspect Ratio of width to height, this is the numerator, par = (pixel_aspect_ratio_num)/(pixel_aspect_ratio_den) */
float pixel_aspect_ratio_den; /* Pixel Aspect Ratio of width to height, this is the denominator, par = (pixel_aspect_ratio_num)/(pixel_aspect_ratio_den) */
} DIL_RenderInformation;
DIL_DecodeInformation struct
This structure captures properties related to the decoding process, width and height of the output, whether LCEVC data has been found and whether it has been applied.
typedef struct DIL_DecodeInformation
{
uint32_t width; /* width of the decoded frame */
uint32_t height; /* height of the decoded frame */
bool skip_requested; /* decode skip was requested */
bool lcevc_enhanced; /* true if the frame has been enhanced by lcevc decoding */
bool lcevc_available; /* true if lcevc encoded data is available for this frame */
} DIL_DecodeInformation;
DIL_ImagePlaneBufferDesc struct
This struct describes a plane memory buffer
typedef struct DIL_ImagePlaneBufferDesc
{
uint8_t* data; /* Pointer to first byte of the first active sample of the plane buffer, NOTE: active means no cropping, the active origin is 0,0 */
uint32_t sample_byte_stride; /* Distance in bytes between consecutive sample values of the same channel, for ex. UV plane on 8bit depth this will be 2 */
uint32_t row_byte_stride; /* Distance in bytes between the first sample of two consecutive rows */
} DIL_ImagePlaneBufferDesc;
DIL_ImagePlaneTextureDesc struct
This struct describes a plane OpenGL texture
typedef struct DIL_ImagePlaneTextureDesc
{
uint32_t target; /* OpenGL/GLES texture target, as in glBindTexture target */
uint32_t id; /* OpenGL/GLES texture id. Unused indexes should be set to 0 */
uint32_t width; /* OpenGL/GLES texture width, which might have extra padding, NOTE: the active width is in the parent Image struct */
uint32_t height; /* OpenGL/GLES texture height, which might have extra padding, NOTE: the active height is in the parent Image struct */
int32_t internal_format; /* OpenGL/GLES (GLint) sized internal format as from Table 2 of glTexImage2D man page */
} DIL_ImagePlaneTextureDesc;
DIL_HardwareBufferUsageOption enum
This enum use used to set the UsageOption when creating HardwareBuffer DIL_Image types. It should be provided in the "hardware_buffer_usage_option" tag when configuring the DIL object
typedef struct DIL_ImagePlaneHardwareBufferDesc
{
uint32_t width; /* width, which might have extra padding, NOTE: the active width is in the parent Image struct */
uint32_t height; /* height, which might have extra padding, NOTE: the active height is in the parent Image struct */
void* nativeObject; /* platform native object/id for the HardwareBuffer */
} DIL_ImagePlaneHardwareBufferDesc;
DIL Properties API
DIL_QueryProperty
Obtain a specific named property. This function provides a way of looking up a property by name to observe it's current value, this may be of use for a user to query DIL properties
Release a previously queried property, this function provides a way of releasing any memory allocated by a call to DIL_QueryProperty. Any data contained in value maybe invalidated after this call
Release a previously queried property group, this function provides a way of releasing any memory allocated by a call to DIL_QueryPropertyGroup Any data contained in value maybe invalidated after this call
This structure contains all relevant information used to query a property or metadata from the DIL.
typedef struct DILProperty
{
const char* name; /* The name of the property. */
const char* description; /* A description of the property. */
DILPropertyType type; /* The value type for this property. */
union
{
int8_t i8;
int16_t i16;
int32_t i32;
int64_t i64;
uint8_t u8;
uint16_t u16;
uint32_t u32;
uint64_t u64;
float f;
double d;
const char* str;
const void* vptr;
const void* data;
const DILPropertyGroup* pgptr;
} value; /* A union representing the value of the property. The user must read the field that match the type enum value. */
const uint8_t* blob; /* Special field containing a block of contiguous memory when the type enum is set to DIL_PT_Blob. This is allowed to be NULL. */
uint32_t blob_size; /* When the blob is not NULL then the length in bytes of the memory, otherwise 0. */
uint64_t data_size; /* size of the "data" buffer in bytes */
bool can_free; /* flag to show if the pointer values should be free'ed when the DIL_ReleaseProperty is called */
} DILProperty;
DILPropertyGroup struct.
This structure contains a logical grouping of properties, this is a convenience feature to assist with providing a "pretty" command line, or GUI.
All properties are assigned to a group, and properties can only appear in one group.
typedef struct DILPropertyGroup_t
{
const char* name; /* The name for this group of properties. */
const char* description; /* A description of the property group. */
DILProperty* properties; /* An array of properties for this group. */
uint32_t property_count; /* The length of the properties array. */
bool can_free; /* flag to show if the properties pointer should be free'ed when the DIL_ReleaseProperty is called */
} DILPropertyGroup;
Figure 1 Unmodified and LCEVC Pipeline using the DIL
