# Functionality ### Input & output As shown in the diagram above, the DIL needs two inputs: * input buffers - the same buffers that are fed to the base decoder in encoding order; * base decoded frames - produced by the base decoder in presentation order; and produces an output onto either a * buffer or * offscreen texture or * onscreen surface. ### Colour formats The DIL can operate on a base decoded picture in the following colour formats: * YUV420 I420 - planar Y, U and V with 4:2:0 subsampled chroma; * YUV420 NV12 - semi-planar Y (planar), and UV (interleaved) with 4:2:0 subsampled chroma; * RGB/RGBA - raster Red Green Blue, i.e. on one plane with interleaved colour components; ### Modes The DIL can be configured to work with two types of internal pipeline: * CPU - all the LCEVC stages are performed in CPU, only using SIMD acceleration, by means of the DPI only, the GPU is used only for the possible YUV/RGB conversion; * GPU - most of the LCEVC stages are performed in GPU suing GL shaders, including YUV/RGB conversions, while the CPU is only used to produce the LCEVC residual planes at both LOQs ### Output type The DIL offers decoding to either * on screen * off screen On screen refers to a window from the OS window system, currently EGL and GLFW windowing systems are supported. Off screen refers to an array of buffers or GL textures in memory. The latter case has been designed to allow the client to manage decode and presentation separately, for example to allow buffering/queuing already decoded pictures before display. ### Other configurable features On creating a DIL instance, the client can configure additional features, such as: * OpenGL major and minor versions (to force a specific version over the system’s selected one); * Use of OpenGL ES; * Use of 8 bit LCEVC residual planes - instead of 16 bit * Use of Hardware Buffers - (Apple and Android only) * Enable an on screen UI for stats and live config; * Enable dumping stats to local storage; * Enable dumping raw output frames to local storage; The clients configures the DIL at creation time by passing a JSON string and a pair of window system related parameters for context and destination surface. ## API Overview Using the DIL is fairly easy, especially if compared to the more basic DPI, since it operates at a relatively high abstraction level. Apart from obvious creation and destruction of a DIL instance (respectively `DIL_Create()` and `DIL_Destroy()`), the main API calls are effectively two: * Feed input - `DIL_AddNALData()` * Decode - `DIL_DecodeandRender()` (for decoding on screen, alternatively `DIL_Decode()` for decoding off screen followed by `DIL_Render()`) * Feed input - `DIL_AddNALData()` * Decode - `DIL_DecodeandRender()` (for decoding on screen, alternatively `DIL_Decode()` for decoding off screen followed by `DIL_Render()`) The DIL also offers an API for the client to retrieve the size of the output picture, which can be useful to pre-allocate buffers or textures of the right size before decoding: * Get size information - `DIL_GetDecodeInformation()` Note: input buffers must have NAL byte stream format with either Annex B or Length Prefix format. ### Drop/Skip frames LCEVC content may be encoded with a temporal feature that requires each and every frame to be actioned on in order to keep the DPI internal temporal reference data correct. As a result when the client skips frames, for ex. because of a seek in the timeline, or drops frames because they are “late”, it shall let the DIL know by calling the following function: * `DIL_DecodeSkip()` The DIL does know whether the temporal feature is on and therefore can fall back to a no operation case if that is the case. It is not advised that the client tries to implement this behaviour on its side. ### Synchronous and Asynchronous modes The DIL can be used in both synchronous or asynchronous mode. This is applicable to the decode and render functions: * `DIL_Decode()` * `DIL_DecodeAndRender()` * `DIL_DecodeSkip()` * `DIL_Render()` If the client has previously set the callback function by means of the `DIL_SetDecodedAsyncCallback()` or `DIL_SetRenderAsyncCallback()` API calls, the above decode functions will work asynchronously, therefore returning immediately and triggering the client’s callback function when the decode (or render) has completed. If the callback has not been set the functions will work synchronously. ### Base pass-through mode The DIL also supports a pass-through mode, in which the base is simply copied on to the output without applying the LCEVC enhancement. This mode is triggered when no LCEVC data is found for the frame and can be set as forced behaviour, for every frame, from the JSON configuration. ## Example Integration Code In basic terms, a sample integration code using the DIL can be briefly described, among the various phases, as follows: ```cpp // Include DIL header #include // Initialisation phase /////////////////////////////////////////////////////// // Create an instance of the DIL DIL_ContextSettings context_settings = { 0 }; char * json_settings = "{\ \"gl_decode\": true,\ \"gl_es\": true,\ \"gl_major\": 3,\ \"gl_minor\": 1,\ \"swap_interval\": 0,\ \"fullscreen\": false,\ \"hardware_buffers\": false,\ \"use_u8_surface\": false,\ \"force_passthrough\": false\ }"; // Create a DIL instance DIL_Decoder decoder = nullptr; if (DIL_Create(json_settings, &context_settings , &decoder) != DIL_RC_Success) { fprintf(stderr, "Unable to create DIL instance\n"); exit(-1); } // Destruction phase ////////////////////////////////////////////////////////// DIL_Destroy(decoder); decoder = nullptr; // Handling input buffers ///////////////////////////////////////////////////// //... DIL_NALFormat nalFormat; // Set NAL format, from metadata, properties or parsing the input buffers // ... if (DIL_AddNALData(decoder, inputCc, presentationTimeStamp, buffer, length, nalFormat) != DIL_RC_Success) { fprintf(stderr, "Unable to pass input buffer for inputCc=%u pts=%PRId64\n", inputCc, presentationTimeStamp); exit(-2); } // Handling decoded frame from base decoder /////////////////////////////////// // ... // Get size of ouput picture if needed DIL_DecodeInformation decodeInfo; if (DIL_GetDecodeInformation(decoder, inputCc, presentationTimeStamp, baseWidth, baseHeight, &decodeInfo) != DIL_RC_Success) { fprintf(stderr, "DIL get decode info failed for inputCc=%u pts=%PRId64\n", inputCc, baseImage.pts; exit(-3); } // Using decode info // ... DIL_Image baseImage; // Fill the base "Image" struct with the data properties of the base frame // ... DIL_RenderInformation renderInfo; // Fill rotation and pixel aspect ratio // ... if (DIL_DecodeAndRender(decoder, inputCc, presentationTimeStamp, baseImage, &renderInfo, &decodeInfo) != DIL_RC_Success) { fprintf(stderr, "DIL decode failed for pts=%PRId64\n", baseImage.pts; exit(-4); } // Decode info reports if LCEVC is available and has been applied // ... ``` ## DIL API ### DIL\_Create Create an instance of a DIL Decoder * **json\_settings** JSON string with initialisation parameters * **context\_settings** settings from the windowing system * **instance decoder** instance created * **return** DIL API return code ```cpp DIL_ReturnCode DIL_Create( const char* json_settings, const DIL_ContextSettings* context_settings, DIL_Decoder instance ); ``` ### DIL\_Destroy Destroy an instance of a DIL Decoder * **decoder** instance to be destroyed ```cpp void DIL_Destroy( DIL_Decoder decoder); ``` ### DIL\_GetImage Returns a fresh usable instance of DIL\_Image * **decoder** DIL instance * **image\_desc** Describing Image configurations * **image** Result DIL\_Image if creation is successful, NULL otherwise * **return** DIL\_RC\_Error if image\_desc contains incompatible values ```cpp DIL_ReturnCode DIL_GetImage(DIL_Decoder decoder, DIL_ImageDesc* image_desc, DIL_Image* image); ``` ### **DIL\_ReleaseImage** Releases instance of DIL\_Image NOTE: No reference calls to image address should be made after this method * **decoder** DIL instance * **image** DIL\_Image instance to be released * **return** DIL API return code ```cpp DIL_ReturnCode DIL_ReleaseImage(DIL_Decoder decoder, DIL_Image image); ``` ### DIL\_ImageGetDesc Will return DIL\_ImageDesc of the image * **image** DIL\_Image to query * **desc** Contents of the pointer will be rewritten with image descriptions * **return** DIL API return code ```cpp DIL_ReturnCode DIL_ImageGetDesc(DIL_Image image, DIL_ImageDesc* desc); ``` ### DIL\_ImageSetActiveRegion Will set the active region for an image * **image** DIL\_Image instance to modify * **offset\_x** X offset for the region * **offset\_y** Y offset for the region * **width** width for the region * **height** height for the region * **return** DIL\_RC\_Error if crop region is out of bounds, image\_type doesn't support cropping ```cpp DIL_ReturnCode DIL_ImageSetActiveRegion( DIL_Image image, uint32_t offset_x, uint32_t offset_y, uint32_t width, uint32_t height); ``` ### DIL\_ImageGetActiveRegion Will get the active region for an image * **image** DIL\_Image instance to modify * **offset\_x** X offset for the region * **offset\_y** Y offset for the region * **width** width for the region * **height** height for the region * **return** DIL\_RC\_Error if crop region is out of bounds, image\_type doesn't support cropping ```cpp DIL_ReturnCode DIL_ImageGetActiveRegion( DIL_Image image, uint32_t* offset_x, uint32_t* offset_y, uint32_t* width, uint32_t* height); ``` ### DIL\_ImageGetPlanesCount Will return number image's number of planes * **image** DIL\_Image to query * **plane\_count** Will be filled with number of planes * **return** DIL API return code ```cpp DIL_ReturnCode DIL_ImageGetPlanesCount(DIL_Image image, uint32_t* plane_count); ``` ### DIL\_ImageSetPlaneBuffer Will set content and configurations of a plane by data buffer * **image** DIL\_Image instance to modify * **plane\_index** Index of the plane to modify * **buffer\_desc** DIL\_ImagePlaneBufferDesc instance to read from * **return** DIL\_RC\_Error if plane\_index is out of bounds, image\_type is not DIL\_Buffer or image is managed ```cpp DIL_ReturnCode DIL_ImageSetPlaneBuffer( DIL_Image image, uint32_t plane_index, DIL_ImagePlaneBufferDesc* buffer_desc); ``` ### DIL\_ImageGetPlaneBuffer Will return content and configurations of a plane * **image** DIL\_Image to query * **plane\_index** Index of the plane to query * **buffer\_desc** DIL\_ImagePlaneBufferDesc instance to write to * **return** DIL\_RC\_Error if plane\_index is out of bounds or image\_type is not DIL\_Buffer ```cpp DIL_ReturnCode DIL_ImageGetPlaneBuffer( DIL_Image image, uint32_t plane_index, DIL_ImagePlaneBufferDesc* buffer_desc); ``` ### DIL\_ImageSetPlaneTexture Will set content and configurations of a plane by OpenGL/GLES texture * **image** DIL\_Image instance to modify * **plane\_index** Index of the plane to modify * **texture\_desc** DIL\_ImagePlaneTextureDesc instance to read from * **return** DIL\_RC\_Error if plane\_index is out of bounds, image\_type is not DIL\_Texture or image is managed ```cpp DIL_ReturnCode DIL_ImageSetPlaneTexture( DIL_Image image, uint32_t plane_index, DIL_ImagePlaneTextureDesc* texture_desc); ``` ### DIL\_ImageGetPlaneTexture Will return content and configurations of a plane by OpenGL/GLES texture * **image** DIL\_Image instance to modify * **plane\_index** Index of the plane to modify * **texture\_desc** DIL\_ImagePlaneTextureDesc instance to read from * **return** DIL\_RC\_Error if plane\_index is out of bounds, image\_type is not DIL\_Texture or image is managed ```cpp DIL_ReturnCode DIL_ImageGetPlaneTexture( DIL_Image image, uint32_t plane_index, DIL_ImagePlaneTextureDesc* texture_desc); ``` ### DIL\_ImageSetPlaneHardwareBuffer Will set content and configurations of a plane by OS specific Hardware buffer. * **image** DIL\_Image instance to modify * **plane\_index** Index of the plane to modify * **desc** DIL\_ImagePlaneHardwareBufferDesc instance to read from * **return** DIL\_RC\_Error if plane\_index is out of bounds, image\_type is not DIL\_Texture or image is managed ```cpp DIL_ReturnCode DIL_ImageSetPlaneHardwareBuffer( DIL_Image image, uint32_t plane_index, DIL_ImagePlaneHardwareBufferDesc* desc); ``` ### DIL\_ImageGetPlaneHardwareBuffer Will return content and configurations of a plane by OS specific Hardware buffer. * **image** DIL\_Image instance to query * **plane\_index** Index of the plane to query * **desc** DIL\_ImagePlaneHardwareBufferDesc instance to write to * **return** Will return DIL\_RC\_Error if plane\_index is out of bounds or image\_type is not DIL\_HardwareBuffer ```cpp DIL_ReturnCode DIL_ImageGetPlaneHardwareBuffer( DIL_Image image, uint32_t plane_index, DIL_ImagePlaneHardwareBufferDesc* desc); ``` ### DIL\_ImageSetUserData Will set the user data associated with the image * **image** DIL\_Image to set the userdata for * **userdata** The userdata to associate with the image * **return** DIL\_RC\_InvalidParam if the image, DIL\_RC\_Success otherwise ```cpp DIL_ReturnCode DIL_ImageSetUserData(DIL_Image image, void* userdata); ``` ### DIL\_ImageGetUserData Will get the user data associated with the image through DIL\_ImageSetUserData * **image** DIL\_Image to get the userdata for * **userdata** The userdata to associate with the image * **return** DIL\_RC\_InvalidParam if the image is invalid or userdata is invalid, DIL\_RC\_Success otherwise ```cpp DIL_ReturnCode DIL_ImageGetUserData(DIL_Image image, void** userdata); ``` ### DIL\_ImagePlaneHardwareBufferLock Will return content and configurations of a plane buffer obtained by locking the hardware buffer for CPU access * **image** DIL\_Image to access * **plane\_index** Index of the plane to query * **read\_access** Whether we want read access * **write\_access** Whether we want write access * **buffer\_desc** DIL\_ImagePlaneBufferDesc instance to write to * **return** Will return DIL\_RC\_Error if plane\_index is out of bounds or image\_type is not DIL\_HardwareBuffer ```cpp DIL_ReturnCode DIL_ImagePlaneHardwareBufferLock( DIL_Decoder decoder, DIL_Image image, uint32_t plane_index, bool read_access, bool write_access, DIL_ImagePlaneBufferDesc* buffer_desc); ``` ### DIL\_ImagePlaneHardwareBufferUnlock Will unlock the hardware buffer plane once CPU access is completed. * **image** DIL\_Image to unlock * **plane\_index** Index of the plane to query * **return** Returns DIL\_RC\_Error if plane\_index is out of bounds or image\_type is not DIL\_HardwareBuffer ```cpp DIL_ReturnCode DIL_ImagePlaneHardwareBufferUnlock( DIL_Decoder decoder, DIL_Image image, uint32_t plane_index); ``` ### DIL\_AddTSData Feed Transport Stream packet(s) for the DIL to extract and store LCEVC data, feed a buffer of TS packets associated with the Access Unit identified by input cc and timestamp NOTE: input\_cc is a feed counter identifying which feed the passed data belongs to, the client shall increment its value every time the input changes, typically because of a playback position change or a rendition change in an ABR playback NOTE: input\_cc shall have monotonically increasing value over calls * **decoder** DIL instance * **input\_cc** input continuity counter * **data** pointer to the TS buffer * **size** size of the TS buffer * **returns** DIL API return code ```cpp DIL_ReturnCode DIL_AddTSData( DIL_Decoder decoder, uint16_t input_cc, uint8_t* data, uint32_t size ); ``` ### DIL\_AddWebmData Feed Webm payload for the DIL to extract and store LCEVC data * **decoder** DIL instance * **input\_cc** input continuity counter * **data** pointer to the TS buffer * **size** size of the TS buffer * **returns** DIL API return code ```cpp DIL_ReturnCode DIL_AddWebmData( DIL_Decoder decoder, uint16_t input_cc, uint8_t* data, uint32_t size ); ``` ### DIL\_AddNALData Feed NAL unit(s) for the DIL to extract and store LCEVC data, feed a buffer of NAL units associated with the Access Unit identified by input cc and timestamp NOTE: input\_cc is a feed counter identifying which feed the passed data belongs to, the client shall increment its value every time the input changes, typically because of a playback position change or a rendition change in an ABR playback NOTE: input\_cc shall have monotonically increasing value over calls NOTE: timestamp shall have monotonically increasing value over calls within the same input\_cc * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference for the passed NAL units * **data** pointer to the NAL units buffer * **size** size of the NAL units buffer * **nalu\_format** format of the NAL units * **return** DIL API return code ```cpp DIL_ReturnCode DIL_AddNALData( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp, uint8_t* data, uint32_t size, DIL_NALFormat nalu_format ); ``` ### DIL\_AddNALDataEx Feed NAL unit(s) for the DIL to extract and store LCEVC data (Extended), feed a buffer of NAL units associated with the Access Unit identified by input cc and timestamp and additionally let the DIL strip the LCEVC payloads and zero pad the trailing bytes, returning the new size before padding NOTE: input\_cc is a feed counter identifying which feed the passed data belongs to, the client shall increment its value every time the input changes, typically because of a playback position change or a rendition change in an ABR playback NOTE: input\_cc shall have monotonically increasing value over calls NOTE: timestamp shall have monotonically increasing value over calls within the same input\_cc * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference for the passed NAL units * **data** pointer to the NAL units buffer * **size** size of the NAL units buffer * **nalu\_format** format of the NAL units * **return** size of the stripped NAL units buffer ```cpp uint32_t DIL_AddNALDataEx( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp, uint8_t* data, uint32_t size, DIL_NALFormat nalu_format ); ``` ### DIL\_AddRawData Feed LCEVC data to the DIL, feed a buffer of pre parsed LCEVC payload data for the Access Unit identified by input cc and timestamp NOTE: input\_cc is a feed counter identifying which feed the passed data belongs to, the client shall increment its value every time the input changes, typically because of a playback position change or a rendition change in an ABR playback NOTE: input\_cc shall have monotonically increasing value over calls NOTE: timestamp shall have monotonically increasing value over calls within the same input\_cc * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference for the passed LCEVC data * **data** pointer to the NAL units buffer * **size** size of the NAL units buffer * **return** DIL API return code ```cpp DIL_ReturnCode DIL_AddRawData( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp, uint8_t* data, uint32_t size ); ``` ### DIL\_GetDecodeInformation Get details about the decoding process for the Access Unit identified by input cc and timestamp * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference of the picture * **base\_width** width of the base decoded picture * **base\_height** height of the base decoded picture * **information** pointer to decoder information structure that the DIL will fill * **return** DIL API return code ```cpp DIL_ReturnCode DIL_GetDecodeInformation( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp, uint32_t base_width, uint32_t base_height, DIL_DecodeInformation* information ); ``` ### DIL\_Decode Decode on to an off screen DIL\_Image, get an off screen LCEVC enhanced (or base pass-through) picture from the input base picture relative to input cc and timestamp * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference of the picture to be decoded * **max\_decode\_time\_us maximum** amount of time the decode has to complete, use 0 to disable decode timeout * **input** pointer to decoded base picture for input cc and timestamp (it may be modified if DIL operates in CPU mode) * **output** pointer to LCEVC enhanced (or base pass-through) output picture * **decode\_information** pointer to decoder information structure that the DIL will fill (if DIL\_SetDecodedAsyncCallback has not been called) * **return** DIL API return code ```cpp DIL_ReturnCode DIL_Decode( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp, uint32_t max_decode_time_us, DIL_Image* input, DIL_Image* output, DIL_DecodeInformation* decode_information ); ``` ### DIL\_Render Render to back buffer a previously decoded DIL\_Image NOTE: for the scheduled render to be set DIL\_SetRenderAsyncCallback must have been previously called * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference of the picture to be rendered * **image** pointer to a picture to be rendered to back buffer * **render\_information** pointer to a render information structure * **delay\_us** delay in microseconds for the render to eb executed (0 for no delay) * **return** DIL API return code ```cpp DIL_ReturnCode DIL_Render( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp, DIL_Image image, const DIL_RenderInformation* render_information, int64_t delay_us ); ``` ### DIL\_DecodeAndRender Decode and render immediately to back buffer * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference of the picture to be decoded and rendered * **max\_decode\_time\_us maximum** amount of time the decode has to complete, use 0 to disable decode timeout * **input** pointer to decoded base picture for input cc and timestamp (it may be modified if DIL operates in CPU mode) * **render\_information** pointer to a render information structure * **decode\_information** pointer to decoder information structure that the DIL will fill (if DIL\_SetDecodedAsyncCallback has not been called) * **return** DIL API return code ```cpp DIL_ReturnCode DIL_DecodeAndRender( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp, uint32_t max_decode_time_us, DIL_Image input, const DIL_RenderInformation* render_information, DIL_DecodeInformation* decode_information ); ``` ### DIL\_DecodeSkip Let the DIL know that an Access Unit is not being presented (dropped or skipped), DIL will do the minimum processing to keep internal state consistent for the next picture NOTE: this call will generate a DecodeAsyncCallback if previously set NOTE: calling this on an IDR Access Unit is undefined behaviour * **decoder** DIL instance * **input\_cc** input continuity counter * **timestamp** time reference for the Access Unit to be skipped * **return** DIL API return code ```cpp DIL_ReturnCode DIL_DecodeSkip( DIL_Decoder decoder, uint16_t input_cc, int64_t timestamp ); ``` ### DIL\_Flush Synchronize client and DIL by throwing away any pending decodes and renders and waiting for all callbacks to complete NOTE: all callbacks set by DecodeAsyncCallback and RenderAsyncCallback will be called with a the result filed in the callback will indicate the error DIL\_RC\_Flushed making DIL calls in this callback could result in a deadlock use the DIL\_SetFlushAsyncCallback to break this loop * **decoder** DIL instance * **return** DIL API return code ```cpp DIL_ReturnCode DIL_Flush( DIL_Decoder decoder ); ``` ### DIL\_Drain synchronize client and DIL by allowing any pending decodes and renders to complete and waiting for all callbacks to complete NOTE: all callbacks set by DecodeAsyncCallback and RenderAsyncCallback will be called making DIL calls in this callback could result in a deadlock use the DIL\_SetDrainAsyncCallback to break this loop * **decoder** DIL instance * **return** DIL API return code ```cpp DIL_ReturnCode DIL_Drain( DIL_Decoder decoder ); ``` ### DIL\_GetGLContext Get the DIL's GL Context * **decoder** DIL instance * **context** pointer to pointer of GL Context * **return** DIL API return code ```cpp DIL_ReturnCode DIL_GetGLContext( DIL_Decoder decoder, void** context ); ``` ### DIL\_GetGLDisplay Get the DIL's GL Display * **decoder** DIL instance * **display** pointer to pointer of GL Display * **return** DIL API return code ```cpp DIL_ReturnCode DIL_GetGLDisplay( DIL_Decoder decoder, void** display); ``` ### DIL\_SetGLWindow Set the DIL's GL output window * **decoder** DIL instance * **gl\_external\_window** pointer to destination window * **is\_secure** true if the destination window is secure * **return** DIL API return code ```cpp DIL_ReturnCode DIL_SetGLWindow( DIL_Decoder decoder, void* gl_external_window, bool is_secure ); ``` ### DIL\_SetExitRequestedCallback Set ExitRequest call back function * **decoder** DIL instance * **callback** pointer to DILOnExitRequestedCallBack function * **user\_data** pointer to client's user data * **return** DIL API return code ```cpp typedef void (*DILOnExitRequestedCallBack)( void* user_data ); DIL_ReturnCode DIL_SetExitRequestedCallback( DIL_Decoder decoder, DILOnExitRequestedCallBack callback, void* user_data); ``` ### DIL\_SetDecodedAsyncCallback Set asynchronous on decoded call back function * **result** result of the decode request, see DIL\_ReturnCode * **input\_cc** input continuity counter * **timestamp** time reference of the picture that was decoded * **input** pointer to base input picture, the same passed in the previous DIL\_Decode * **output** pointer to LCEVC enhanced (or base pass-through) output picture, the same passed in the previous DIL\_Decode * **information** pointer to decoder information structure (not the same passed in from the previous DIL\_Decode) * **user\_data** pointer to the same client's user data that was passed into the DIL\_SetDecodedAsyncCallback * **decoder** DIL instance * **callback** pointer to DILOnDecodedAsyncCallBack function * **user\_data** pointer to client's user data * **return** DIL API return code ```cpp typedef void (*DILOnDecodedAsyncCallBack)( DIL_ReturnCode result, uint16_t input_cc, int64_t timestamp, DIL_Image input, DIL_Image output, const DIL_DecodeInformation* information, void* user_data ); DIL_ReturnCode DIL_SetDecodedAsyncCallback( DIL_Decoder decoder, DILOnDecodedAsyncCallBack callback, void* user_data ); ``` ### DIL\_SetRenderAsyncCallback Set asynchronous on render callback function * **result** result of render request, see DIL\_ReturnCode * **completion\_time\_us** actual time in microseconds the render completed * **input\_cc** input continuity counter * **timestamp** time reference of the picture that was rendered * **output** pointer to the same picture passed in with the DIL\_Render method * **user\_data** pointer to the same client's user data that was passed into the DIL\_SetRenderAsyncCallback * **decoder** DIL instance * **callback** pointer to DILOnRenderAsyncCallBack function * **user\_data** pointer to client's user data * **return** DIL API return code ```cpp typedef void (*DILOnRenderAsyncCallBack)( DIL_ReturnCode result, int64_t completion_time_us, uint16_t input_cc, int64_t timestamp, DIL_Image output, void* user_data ); DIL_ReturnCode DIL_SetRenderAsyncCallback( DIL_Decoder decoder, DILOnRenderAsyncCallBack callback, void* user_data ); ``` ### DIL\_SetRenderResizeAsyncCallback Called at the start of the render process when the image to be displayed has a different size to the last image rendered. This is a blocking call and there should be no DIL calls in the callback function. * **result** result of render request, see DIL\_ReturnCode * **completion\_time\_us** actual time in microseconds the render completed * **input\_cc** input continuity counter * **timestamp** time reference of the picture that was rendered * **output** pointer to the same picture passed in with the DIL\_Render method * **user\_data** pointer to the same client's user data that was passed into the DIL\_SetRenderAsyncCallback * **decoder** DIL instance * **callback** pointer to DILOnRenderAsyncCallBack function * **user\_data** pointer to client's user data * **return** DIL API return code ```cpp typedef void (*DILOnRenderAsyncCallBack)( DIL_ReturnCode result, int64_t completion_time_us, uint16_t input_cc, int64_t timestamp, DIL_Image output, void* user_data ); DIL_ReturnCode DIL_SetRenderResizeAsyncCallback( DIL_Decoder decoder, DILOnRenderAsyncCallBack callback, void* user_data ); ``` ### DIL\_SetFlushAsyncCallback Callback function for the DIL\_Flush() function * **user\_data** pointer to client's user data * **decoder** DIL instance * **callback** pointer to DILOnSynchronizeCallBack function * **user\_data** pointer to client's user data * **return** DIL API return code ```cpp typedef void (*DILOnSynchronizeCallBack)( void* user_data ); DIL_ReturnCode DIL_SetFlushAsyncCallback( DIL_Decoder decoder, DILOnSynchronizeCallBack callback, void* user_data ); ``` ### DIL\_SetDrainAsyncCallback Callback function for the DIL\_Drain() function * **user\_data** pointer to client's user data * **decoder** DIL instance * **callback** pointer to DILOnSynchronizeCallBack function * **user\_data** pointer to client's user data * **return** DIL API return code ```cpp typedef void (*DILOnSynchronizeCallBack)( void* user_data ); DIL_ReturnCode DIL_SetDrainAsyncCallback( DIL_Decoder decoder, DILOnSynchronizeCallBack callback, void* user_data ); ``` ### DIL\_InputMouseButtonState Pass the DIL the position of the input mouse button * **decoder** DIL instance * **pressed** true if button is pressed * **return** DIL API return code ```cpp DIL_ReturnCode DIL_InputMouseButtonState( DIL_Decoder decoder, bool pressed ); ``` ### DIL\_InputMousePosition Pass the DIL the position of the input mouse * **decoder** DIL instance * **x** horizontal coordinate of the mouse cursor * **y** vertical coordinate of the mouse cursor * **return** DIL API return code ```cpp DIL_ReturnCode DIL_InputMousePosition( DIL_Decoder decoder, float x, float y ); ``` ## DIL Types and Enumerations ### DIL\_NALFormat enum DIL\_NALFormat represents the two possible NAL Unit formats, for each of the MPEG base codecs the DIL currently supports: H264 (ISO/IEC 14496-10, aka AVC) and H265 (ISO/IEC 23008-2, aka HEVC) NAL, where Annex B indicates 3 byte 0x000001 or 4 byte 0x00000001 start code prefix as described in Annex B of (H264 or HEVC), LP (Length Prefix) indicates a 4 byte prefix with the length of the NAL unit as MSBF 32 bit unsigned int. ```cpp typedef enum DIL_NALFormat { DIL_UnknownNALFormat = 0, DIL_H264_AnnexB, DIL_H265_AnnexB, DIL_H264_LP, DIL_H265_LP } DIL_NALFormat; ``` ### DIL\_ReturnCode enum DIL\_ReturnCode represents the possible API return codes. Most of the API calls will return one of these values. ```cpp typedef enum DIL_ReturnCode { DIL_RC_Success = 0, /* The API call completed successfully */ DIL_RC_Error = -1, /* A generic catch all error */ DIL_RC_InvalidParam = -2, /* The user supplied an invalid parameter to the function call */ DIL_RC_NotFound = -3, /* The query call failed to find the item by name, but didn't not generate an error. */ DIL_RC_NotSupported = -4, /* The functionality requested is not supported on the running system. */ DIL_RC_Flushed = -5, /* The requested operation failed because it was flushed via DIL_Flush. */ DIL_RC_Timeout = -6 /* The requested operation failed because it timedout. */ } DIL_ReturnCode; ``` ### DIL\_ColorFormat enum This enum represents the supported colour formats, currently only 8 bit per sample formats are supported For a detailed description of the formats see: ```cpp typedef enum DIL_ColorFormat { DIL_UnknownColorFormat = 0, /* Unspecified colour format */ DIL_I420_8, /* 8 bit 4:2:0 YUV Planar colour format: Y, U and V on separate planes */ DIL_NV12_8, /* 8 bit 4:2:0 YUV SemiPlanar colour format: Y plane and UV interleaved plane */ DIL_RGB_8, /* 8 bit Interleaved R, G, B planes 24 bit per sample */ DIL_RGBA_8, /* 8 bit Interleaved R, G, B and A planes 32 bit per sample */ DIL_I420_10LE, /* 10 bit Little Endian 4:2:0 YUV Planar colour format: Y, U and V on separate planes */ } DIL_ColorFormat; ``` ### DIL\_ImageType enum This enum identifies the type of planes the DIL\_Image consists of; planes are non-overlapping sets of samples belonging to one or more colour channels with a constant sample stride, so RGB is a single plane, because R G and B pointed at separately would overlap UV from NV12 is a single plane, because U and V pointed at separately would overlap Y U and V from I420 are three planes because they do not overlap ```cpp typedef enum DIL_ImageType { DIL_UnknownImageType = 0, /* Unspecified image type */ DIL_Buffer_Array, /* One buffer per plane, even if image is on a single memory allocation, an array of planes shall be set by assigning the right pointers to plane buffers. DIL allocated buffers have contiguous memory allocation */ DIL_Texture_Array, /* One texture per plane, typically in YUV colour formats, it consists of a single plane, even for YUV, if GL YUV extension is used */ DIL_HardwareBuffer_Array /* One HardwareBuffer per plane, typically in YUV colour formats, it consists of a single plane, even for YUV, if GL YUV extension is used */ } DIL_ImageType; ``` ### DIL\_ImageDesc struct Holds configurations of a single DIL\_Image instance ```cpp typedef struct DIL_ImageDesc { bool is_managed; /* True means that Image (and its memory) is managed by DIL */ DIL_ImageType image_type; /* Whether it is a buffer array or a texture array */ DIL_ColorFormat color_format; /* Colour format */ uint32_t width; /* Nominal net width of the image in luma samples, i.e. no alignment to macroblocks or striding */ uint32_t height; /* Nominal net height of the image in luma samples, i.e. no sliceheight or padding */ bool can_modify; /* Specifying wether content of image is readOnly or changable */ bool can_resize; /* Specifying wether allocated memory is resizable or not. (Does not apply to DIL_Buffer_External) */ uint32_t options; /* options used in creating the Image, in the case of HardwareBuffers this is a combination of DIL_HardwareBufferUsageOption */ } DIL_ImageDesc; ``` ### DIL\_ContextSettings struct This structure holds parameters related to the window surface the DIL would render into, in case of on screen rendering those parameters are normally provided by the windowing system, e.g. EGL, GLFW. ```cpp typedef struct DIL_ContextSettings { void* gl_external_window; /* Surface window, from the windowing system */ void* gl_external_context; /* Context, from the windowing system */ bool is_secure; /* True if the client has initialised window and context as secure for protected content */ } DIL_ContextSettings; ``` ### DIL\_RenderInformation struct This structure captures the properties related to the (on screen) window rendering, they do not apply for buffer or texture output (off screen). NOTE: rotation is a per frame property because it may change frame by frame due to for ex. the encoding device rotating while shooting. ```cpp typedef struct DIL_RenderInformation { uint32_t rotation; /* Degrees of rotation for the final rendering on the destination window, in 90 unit increments clockwise */ float pixel_aspect_ratio_num; /* Pixel Aspect Ratio of width to height, this is the numerator, par = (pixel_aspect_ratio_num)/(pixel_aspect_ratio_den) */ float pixel_aspect_ratio_den; /* Pixel Aspect Ratio of width to height, this is the denominator, par = (pixel_aspect_ratio_num)/(pixel_aspect_ratio_den) */ } DIL_RenderInformation; ``` ### DIL\_DecodeInformation struct This structure captures properties related to the decoding process, width and height of the output, whether LCEVC data has been found and whether it has been applied. ```cpp typedef struct DIL_DecodeInformation { uint32_t width; /* width of the decoded frame */ uint32_t height; /* height of the decoded frame */ bool skip_requested; /* decode skip was requested */ bool lcevc_enhanced; /* true if the frame has been enhanced by lcevc decoding */ bool lcevc_available; /* true if lcevc encoded data is available for this frame */ } DIL_DecodeInformation; ``` ### DIL\_ImagePlaneBufferDesc struct This struct describes a plane memory buffer ```cpp typedef struct DIL_ImagePlaneBufferDesc { uint8_t* data; /* Pointer to first byte of the first active sample of the plane buffer, NOTE: active means no cropping, the active origin is 0,0 */ uint32_t sample_byte_stride; /* Distance in bytes between consecutive sample values of the same channel, for ex. UV plane on 8bit depth this will be 2 */ uint32_t row_byte_stride; /* Distance in bytes between the first sample of two consecutive rows */ } DIL_ImagePlaneBufferDesc; ``` ### DIL\_ImagePlaneTextureDesc struct This struct describes a plane OpenGL texture ```cpp typedef struct DIL_ImagePlaneTextureDesc { uint32_t target; /* OpenGL/GLES texture target, as in glBindTexture target */ uint32_t id; /* OpenGL/GLES texture id. Unused indexes should be set to 0 */ uint32_t width; /* OpenGL/GLES texture width, which might have extra padding, NOTE: the active width is in the parent Image struct */ uint32_t height; /* OpenGL/GLES texture height, which might have extra padding, NOTE: the active height is in the parent Image struct */ int32_t internal_format; /* OpenGL/GLES (GLint) sized internal format as from Table 2 of glTexImage2D man page */ } DIL_ImagePlaneTextureDesc; ``` ### DIL\_HardwareBufferUsageOption enum This enum use used to set the UsageOption when creating HardwareBuffer DIL\_Image types. It should be provided in the "hardware\_buffer\_usage\_option" tag when configuring the DIL object ```cpp typedef enum DIL_HardwareBufferUsageOption { UsageOption_None = 0, UsageOption_Read_Never = (1 << 0), UsageOption_Read_Rarely = (1 << 1), UsageOption_Read_Often = (1 << 2), UsageOption_Write_Never = (1 << 4), UsageOption_Write_Rarely = (1 << 5), UsageOption_Write_Often = (1 << 6), UsageOption_Protected_Content = (1 << 7), UsageOption_GPU_Read = (1 << 8), UsageOption_GPU_Write = (1 << 9) } DIL_HardwareBufferUsageOption; ``` ### DIL\_ImagePlaneHardwareBufferDesc struct This struct describes a plane HardwareBuffer ```cpp typedef struct DIL_ImagePlaneHardwareBufferDesc { uint32_t width; /* width, which might have extra padding, NOTE: the active width is in the parent Image struct */ uint32_t height; /* height, which might have extra padding, NOTE: the active height is in the parent Image struct */ void* nativeObject; /* platform native object/id for the HardwareBuffer */ } DIL_ImagePlaneHardwareBufferDesc; ``` ## DIL Properties API ### DIL\_QueryProperty Obtain a specific named property. This function provides a way of looking up a property by name to observe it's current value, this may be of use for a user to query DIL properties ```cpp DIL_ReturnCode DIL_QueryProperty(DIL_Decoder decoder, const char* name, DILProperty* output); ``` ### DIL\_SetProperty Set a specific named property. This function provides a way of changing a property by name ```cpp DIL_ReturnCode DIL_SetProperty(DIL_Decoder decoder, const char* name, DILProperty* value); ``` ### DIL\_QueryPropertyGroup Obtain all the available properties for a named group ```cpp DIL_ReturnCode DIL_QueryPropertyGroup(DIL_Decoder decoder, const char* name, DILPropertyGroup* output); ``` ### DIL\_ReleaseProperty Release a previously queried property, this function provides a way of releasing any memory allocated by a call to DIL\_QueryProperty. Any data contained in value maybe invalidated after this call ```cpp DIL_ReturnCode DIL_ReleaseProperty(DIL_Decoder decoder, DILProperty* value); ``` ### DIL\_ReleasePropertyGroup Release a previously queried property group, this function provides a way of releasing any memory allocated by a call to DIL\_QueryPropertyGroup Any data contained in value maybe invalidated after this call ```cpp DIL_ReturnCode DIL_ReleasePropertyGroup(DIL_Decoder decoder, DILPropertyGroup* value); ``` ### DILProperty struct. This structure contains all relevant information used to query a property or metadata from the DIL. ```cpp typedef struct DILProperty { const char* name; /* The name of the property. */ const char* description; /* A description of the property. */ DILPropertyType type; /* The value type for this property. */ union { int8_t i8; int16_t i16; int32_t i32; int64_t i64; uint8_t u8; uint16_t u16; uint32_t u32; uint64_t u64; float f; double d; const char* str; const void* vptr; const void* data; const DILPropertyGroup* pgptr; } value; /* A union representing the value of the property. The user must read the field that match the type enum value. */ const uint8_t* blob; /* Special field containing a block of contiguous memory when the type enum is set to DIL_PT_Blob. This is allowed to be NULL. */ uint32_t blob_size; /* When the blob is not NULL then the length in bytes of the memory, otherwise 0. */ uint64_t data_size; /* size of the "data" buffer in bytes */ bool can_free; /* flag to show if the pointer values should be free'ed when the DIL_ReleaseProperty is called */ } DILProperty; ``` ### DILPropertyGroup struct. This structure contains a logical grouping of properties, this is a convenience feature to assist with providing a "pretty" command line, or GUI. All properties are assigned to a group, and properties can only appear in one group. ```cpp typedef struct DILPropertyGroup_t { const char* name; /* The name for this group of properties. */ const char* description; /* A description of the property group. */ DILProperty* properties; /* An array of properties for this group. */ uint32_t property_count; /* The length of the properties array. */ bool can_free; /* flag to show if the properties pointer should be free'ed when the DIL_ReleaseProperty is called */ } DILPropertyGroup; ```