Encoder Integration Layer (EIL)

Introduction

This article serves as a reference to the V-Nova LCEVC Encoder Integration Layer and should be read in conjunction with the Encoder Integration Layer (EIL) header, where each function is well documented as to its behaviour, arguments and return conditions. This document outlines the design, flow of functionality, and the properties exposed through the Integration Layers configuration system.

The following figure illustrates the EIL software stack.

V-Nova LCEVC Encoding Software Stack

The EIL combines V-Nova LCEVC with the base codec, and wrappers for supported base codecs. V-Nova provides example source code below to show how to use the Integration Layer.

Functionality

The EIL provides a simple yet powerful API, and is similar in usage to working with other video encoding libraries.

The steps for encoding streams using the V-Nova LCEVC EIL are as follows:

1. Initialise the Integration Layer

2. While there are frames to encode:

   a. Obtain an input frame and pass ownership to the encoder.

   b. Wait for the encoder to complete, then fetch the decoded frame.

   c. Release ownership of the output frame back to the encoder.

3. Shut down the Integration Layer

EIL Integration Process Overview

The integration of the EIL can be summarised in several steps:

1. Creation of the EIL

2. Initialisation of the EIL

3. Passing video frames to the EIL

4. Fetching the encoded bitstream from the EIL

5. Flushing the EIL

6. Destruction of the EIL

The above steps, as well as the various options available to integrations, are described below.

1. Creation of the EIL

An instance of the EIL is created using EIL_Open. An EILOpenSettings object is required, which at minimum, must specify the name of a base encoder. This instance of the EIL should be passed to all other functions. The EILOpenSettings can be initialised to a sensible default state using EIL_OpenSettingsDefault.

The EIL issues internally generated logging messages via a callback. The callback is registered on the EILOpenSettings struct. You must provide a callback function pointer before calling Open to receive messages.

void Log(void* userdata, int32_t level, const char* msg)
{
    fprintf(level == EIL_LL_Error ? stderr : stdout, msg);
}

EILContext OpenEIL()
{
    EILOpenSettings settings;
    EILReturnCode   rc = EIL_OpenSettingsDefault(&settings);

    if (rc != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to initialise default open settings: %s\n",
                EIL_GetErrorString(rc));
        return NULL;
    }

    settings.base_encoder = "x264";
    settings.log_callback = Log;
    EILContext context   = NULL;

    if ((rc = EIL_Open(&settings, &context)) != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to open the EIL: %s\n", EIL_GetErrorString(rc));
        return NULL;
    }

    return context;
}

2. Initialisation of the EIL

The EIL is initialised using EIL_Initialise. An EILInitSettings object is required, in which the encoding properties, such as resolution, framerate and bitrate, are specified. properties_json should be set to a json object string with extra optional encoder options.

Upon successful initialisation, the user can then proceed to encoding YUV input.

// context should have been previously created with EIL_Open
bool InitialiseEIL(EILContext context)
{
    EILInitSettings settings;
    EILReturnCode   rc = EIL_InitSettingsDefault(&settings);

    settings.width     = 1920;
    settings.height    = 1080;
    settings.fps_num   = 25;
    settings.fps_denom = 1;
    settings.bitrate   = 3000; // NB: This is in kbps

    // The properties are usually generated from user input
    settings.properties_json = "{ \"encoding_transform_type\": \"dds\" }";

    if ((rc = EIL_Initialise(context, &settings)) != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to intiailise the EIL: %s\n", 
                EIL_GetErrorString(rc));
        return false;
    }

    return true;
}

3. Passing video frames to the EIL

If the previous steps succeeded, then the EIL is now ready to receive frames.

If the EIL has been configured to not use external input, then a preallocated frame can be fetched from the EIL using EIL_GetPicture; otherwise, an EILPicture object should be created at the integration layer and passed to EIL_Encode.

To encode YUV frames, perform the following steps:

1. Call EIL_GetPicture to obtain an EILPicture object for the type of frame to be encoded. The caller now temporarily owns the picture object.

2. Load YUV data into the pre-allocated memory of the plane member of the picture object.

3. Call EIL_Encode using the EILPicture object to begin encoding the picture. The encoder retakes ownership of the EILPicture object, and it is no longer valid to manipulate it. This is a blocking call for the duration of the encoding of a single frame.

bool Encode(EILContext context, int64_t pts)
{
    EILPicture* picture = NULL;
    EILReturnCode rc = EIL_GetPicture(context, EIL_FrameType_Progressive, &picture);

    if (rc != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to obtain picture from the EIL: %s\n",
                EIL_GetErrorString(rc));
        return false;
    }

    // Write the frame data to picture->plane ensuring that the picture->stride
    // for each plane is obeyed.

    picture->pts = pts;

    if ((rc = EIL_Encode(context, picture)) != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to encode picture: %s\n", EIL_GetErrorString(rc));
        return false;
    }

    return true;
}

It is invalid behaviour to hold onto the EILOutput object across multiple frames. The caller must copy and buffer the data if that is the required behaviour.

4. Fetching the encoded bitstream from the EIL

After receiving enough frames, the EIL is ready to output encoded data. The encoded data is fetched using the EIL_GetOutput function.

• If there is any output data available when EIL_GetOutput is invoked, then EIL_GetOutput returns EIL_RC_Success.

• If there is no data available and no errors occurred, then EIL_GetOutput returns EIL_RC_Finished.

To fetch the encoded bitstream from the EIL, perform the following steps:

1. The loop call EIL_GetOutput returns the following, depending upon the presence of output data: - If there is output data, then EILGetOutput returns EIL_RC_Success and populates EILOutput* with a valid pointer. The caller now owns the output data object. - If there is no more output data, then the EIL_GetOutput returns EIL_RC_Finished, unless an error is generated. Please be aware this step is important, as the EIL may generate more than one output per single input, for example, when generating an interlaced encode.

2. Utilise the output data, either by writing it directly out to file or copying it if it must be buffered for output in a muxing system.

3. Call EIL_ReleaseOutput to release the output object back to the ownership of the encoder. It is then no longer valid to read from this object.

Once the integration has finished with the output data, it should return it to the EIL using EIL_ReleaseOutput. It is perfectly fine to call this method from a separate thread other than EIL_Encode.

int HandleOutput(EILContext context)
{
    EILReturnCode rc     = EIL_RC_Success;
    EILOutput*    output = NULL;

    /* Always loop over output. Flush blocks and accumulates may frames worth of
     output. */
    while ((rc = EIL_GetOutput(context, &output)) == EIL_RC_Success)
    {
        // pass the bitstream to a muxer
        mux(output->pts, output->dts, output->data, output->data_length);
		
        /* Immediately release the output. */
        if (EIL_ReleaseOutput(context, output) != EIL_RC_Success)
        {
            fprintf(stderr, "Unable to get output from the EIL: %s\n",
                    EIL_GetErrorString(rc));
            return -1;
        }
    }

    if (rc != EIL_RC_Finished)
    {
        fprintf("Error: Failed to handle output\n");
        return -1;
    }

    return 0;
}

5. Flushing the EIL

To signal to the EIL that no more input data is available, simply pass NULL to EIL_Encode. The EIL then finishes processing any queued video frames. This should only ever be performed at the end of the stream.

To complete encoding, perform the following steps:

1. Call EIL_Encode once using a null pointer for the picture object argument, to flush the entire pipeline. This is a blocking call whilst the remaining pictures are encoded. Subsequent calls will have no effect.

2. Repeat step 4 from the encoding process IL_RC_Finished is returned.

6. Destruction of the EIL and Shutdown

Once encoding has finished, then the EIL can be destroyed. Call EIL_Close to safely shut down and release the encoding context.

void CloseEIL(EILContext* context)
{
    EIL_Close(*context);
    *context = NULL;
}

Features

External Input

The EIL can be configured to accept externally allocated input frames. To do this, the integration should set external_input in EILInitSettings to true. When configured in this mode, the EIL will not allocate any pictures itself, and so EIL_GetPicture will always fail.

In external input mode, the integration must create the EILPicture object itself. The object can be default initialised using EIL_InitPictureDefault. The integration then must set the plane pointers to memory it has allocated and stride to the number of bytes per line. The other fields are then used in the same manner as when external input is disabled.

The planes must be valid from the point that the picture is passed to EIL_Encode, until the associated output is produced.

The EIL can also accept Android hardware buffers in this mode when the EIL is configured for GPU mode. To do this the picture's memory_type is set to EIL_MT_AHardwareBuffer and the plane pointers are set to AHardwareBuffer.

bool InitialiseExteranlInput(EILContext context)
{
    EILInitSettings settings;
    EILReturnCode   rc = EIL_InitSettingsDefault(&settings);

    settings.width          = 1920;
    settings.height         = 1080;
    settings.fps_num        = 25;
    settings.fps_denom      = 1;
    settings.bitrate        = 3000; // NB: This is in kbps
    settings.external_input = 1;

    // The properties are usually generated from user input
    settings.properties_json = "{ \"encoding_transform_type\": \"dds\" }";

    if ((rc = EIL_Initialise(context, &settings)) != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to intiailise the EIL: %s\n",
                EIL_GetErrorString(rc));
        return false;
    }

    return true;
}

bool EncodeExternalInput(EILContext context, int64_t pts)
{
    EILPicture* picture = (EILPicture*)malloc(sizeof(EILPicture));

    if (EIL_InitPictureDefault(picture) != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to initialise picture object\n");
        return false;
    }

    // For the example the planes are allocated inline. These could just as
    // easily be set to pointers that already contain the frame data to
    // avoid copying the data.
    // Assuming 1080p YUV420p
    picture->pts        = pts;
    picture->num_planes = 3;
    picture->plane[0]   = malloc(1920 * 1080);
    picture->stride[0]  = 1080;
    picture->plane[1]   = malloc(960 * 540);
    picture->stride[1]  = 540;
    picture->plane[2]   = malloc(960 * 540);
    picture->stride[2]  = 540;
    picture->user_data  = picture;

    // populate the allocated planes with frame data

    EILReturnCode rc = EIL_Encode(context, picture);

    if (rc != EIL_RC_Success)
    {
        for (int i = 0; i < picture->num_planes; ++i)
            free(picture->plane[i]);

        fprintf(stderr, "Unable to encode frame: %s\n", EIL_GetErrorString(rc));
        return false;
    }

    return true;
}

bool HandleOutput(EILContext context)
{
    EILOutput*    output = NULL;
    EILReturnCode rc     = EIL_GetOutput(context, &output);

    if (rc != EIL_RC_Success)
    {
        if (rc != EIL_RC_Finished)
        {
            fprintf(stderr, "Unable to get output from the EIL: %s\n",
                    EIL_GetErrorString(rc));
            return false;
        }

        return true;
    }

    // pass the bitstream to a muxer
    mux(output->pts, output->dts, output->data, output->data_length);

    EILPicture* picture = (EILPicture*)output->user_data;

    for (int i = 0; i < picture->num_planes; ++i)
        free(picture->plane[i]);

    if ((rc = EIL_ReleaseOutput(context, output)) != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to release output back to the EIL: %s\n",
                EIL_GetErrorString(rc));
        return false;
    }

    return true;
}

Dynamic Reconfiguration

The EIL is able to reconfigure the target bitrate and framerate, if the base encoder also supports dynamic reconfiguration. This is done by passing EILReconfigureSettings, with the new values, to the EIL_Reconfigure function.

bool Reconfigure(EILContext context)
{
    EILReconfigureSettings settings;

    if (EIL_ReconfigureSettingsDefault(&settings) != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to default initialise reconfigure settings");
        return false;
    }

    settings.flags     = EIL_RF_Bitrate | EIL_RF_Framerate;
    settings.bitrate   = 5000;
    settings.fps_num   = 30;
    settings.fps_denom = 1;

    EILReturnCode rc = EIL_Reconfigure(context, &settings);

    if (rc != EIL_RC_Success)
    {
        fprintf(stderr, "Unable to reconfigure EIL: %s\n",
                EIL_GetErrorString(rc));
        return false;
    }

    return true;
}

Error Handling

Almost all of the APIs available from the EIL return EILReturnCode. The documentation for each function Error Handling outlines the potential return conditions. Every effort has been made to ensure that invalid usage of the IL is caught and presented with a meaningful return code. Additionally, a log error or warning will typically be issued via the logging mechanism.

Base Encoder Plugin

Base layer encoding is implemented via a plugin system. Base encoder plugins, along with their corresponding wrappers, are dynamically loaded at runtime, based upon the name of the Base encoder specified in the EILOpenSettings structure.

The name of the base encoder that the EIL uses for base encoding should not contain the preceding lcevc_eilp_ (liblcevc_eilp_ on Linux) or the platform's dynamic library extension. For example, if a plugin is called liblcevc_eilp_x264.so or lcevc_eilp_x264.dll, then the base_encoder string should be x264.

The EIL utilises several search paths to locate plugins. This is to assist with different program usage behaviours and EIL integration techniques. The order of search is as follows:

1. The location of the application process.

2. The current working directory.

3. System-defined PATH environment variable.

An API is provided for the EIL to interact with plugins. This allows a user to optionally implement their own base encoder through the plugin system without the need to be embedded within the EIL, and provides the benefits of modularity. Note: This requires the base plugin API and further documentation on how to develop your own base encoder plugin.

Encoding Configuration Properties

The EIL provides a very powerful configuration system. All configuration properties, except for the basic frame information, are set via a JSON configuration string in the EILInitSettings structure.

The configuration system is supported by a comprehensive built-in system of optimised parameter defaults. The EIL employs ideal settings for both the LCEVC encoder and base encoder, based upon key parameters passed to it by the application, e.g., resolution, bitrate, tune.

Properties can be anonymously queried by the EIL_QueryPropertyGroups API, and, if the user knows the name of properties, by the EIL_QueryProperty API.

Properties specified within the JSON string override any setting provided by the database. Therefore, it is important that an application takes care to specify only properties within the JSON string that the user has explicitly set. For example, in a command-line program, only the options specified by the user in the command line should be included in the JSON string, or, from a GUI, only the options that the user has manually specified a choice for should be included in the JSON string.

The JSON string is processed utilising strong typing correlating to the target property type. If the types are not adhered to, then the EIL will fail to initialise. The user must take care to generate a JSON string that utilises the correct JSON type for each property type. The following table outlines the types and their JSON equivalents:

EILPropertyType	JSON type
EIL_PT_Int32	Number
EIL_PT_Int64	Number
EIL_PT_Uint32	Number
EIL_PT_Uint64	Number
EIL_PT_Float	Number
EIL_PT_Double	Number
EIL_PT_String	String
EIL_PT_Boolean	Boolean, string (‘0’ = false), or number (0 = false)
EIL_PT_Blob	Not used for properties

The JSON string is formatted with a top-level object, and a member per property, as shown in the following example:

{
 "encoding_downsample_luma" : "cubic",
 "temporal_enabled" : true,
 …
}

For additional convenience, properties have been assigned to logical groupings, and can optionally contain a description about what the property does. This is so a user may dynamically generate a command-line parser, or a UI, for the publicly available properties.

Querying Configuration Properties

All publicly available encoding configuration properties for the base encoder and LCEVC enhancement can be queried through the API using EIL_QueryPropertyGroups. This will return an array of all property groups which each contain an array of all properties in that group. Alternatively, if the name of the parameter is known then it can be directly queried through EIL_QueryProperty.

Each property contains a name, description, the expected type of that property and its current value. Below is an example that queries the API for all properties and prints them.

void PrintProperties(EILContext context)
{
	EILPropertyGroups groups;
	EILReturnCode     rc = EIL_QueryPropertyGroups(context, &groups);

	if (rc != EIL_RC_Success)
	{
		fprintf(stderr, "Unable to query property groups: %s\n", EIL_GetErrorString(rc));
		return;
	}

	for (uint32_t g = 0; g < groups.group_count; ++g)
	{
		EILPropertyGroup* group = &groups.group[g];

		printf("%s:\n", group->name);

		for (uint32_t p = 0; p < group->property_count; ++p)
		{
			EILProperty* property = &group->properties[p];

			printf("\t%s (", property->name);

			switch (property->type)
			{
				case EIL_PT_Int16: printf("int16"); break;
				case EIL_PT_Int32: printf("int32"); break;
				case EIL_PT_Int64: printf("int64"); break;
				case EIL_PT_Uint16: printf("uint16"); break;
				case EIL_PT_Uint32: printf("uint32"); break;
				case EIL_PT_Uint64: printf("uint64"); break;
				case EIL_PT_Float: printf("float"); break;
				case EIL_PT_Double: printf("double"); break;
				case EIL_PT_String: printf("string"); break;
				case EIL_PT_Boolean: printf("boolean"); break;
				case EIL_PT_Blob: printf("blob"); break;
			}

			printf("): %s\n", property->description);
		}
	}
}

Metadata

Metadata works much in the same way as properties, except that they are queried only from the system, and they cannot be set externally. They can be treated as read-only properties.

Metadata can only be queried by name using the EIL_QueryMetadata API.

Metadata additionally implements an extension to properties allowing access to a contiguous block of memory (EIL_PT_Blob). This system is incorporated with the Base encoder plugin, such that a Base encoder can output some data to the integrator of the IL, with the IL being ignorant to its usage.

This system is used to allow the EIL to report to the user some internal values, or data that are either not interpreted as properties, or that cannot be expressed as properties.

For example, the ffmpeg sample integration utilises this feature to obtain the h.264 global headers from the libx264 Base encoder plugin to be used for HLS.

Metadata Values

Metadata Values Overview

This section describes the metadata values available from the EIL. It is broken down into sections for each discrete component of the EIL. The following tables indicate the name of the metadata, its associated property type, and a description of what it is and how it is used.

Integration Layer

Metadata	Type	Description
base_type	EIL_PT_String	Describes the underlying technology of the base encoder plugin utilised. For example, the libx264 plugin will output h264 in this metadata.
base_width	EIL_PT_Uint32	Width of the base encoded video.
base_height	EIL_PT_Uint32	Height of the base encoded video.

x264 Integration Layer Plugin

Metadata	Type	Description
h264_header_data	EIL_PT_Blob	This is available when the ‘global-header’ x264 property is set to 1. This is a contiguous block of memory containing the H.264 headers (PPS, SPS, etc.), and can be used when the output container is segmented, so that the headers can be injected at the start of each segment.

API (version 3.9)

EIL_GetErrorString

VN_EIL_PublicAPI const char* EIL_GetErrorString(EILReturnCode error);

Utility method for getting a string representation of an EIL return code in English.

EIL_OpenSettingsDefault and EIL_Open

EIL_OpenSettingsDefault

VN_EIL_PublicAPI EILReturnCode EIL_OpenSettingsDefault(EILOpenSettings* settings);

Initialises the members of an EILOpenSettings object to sensible default state. Some fields of the settings must still be populated by the integration before the settings can be used by the EIL.

EIL_Open

#define VN_EIL_OpenFnName VN_EIL_PP_Concat(EIL_Open_, VN_EIL_API_Version)

VN_EIL_PublicAPI EILReturnCode VN_EIL_OpenFnName(EILOpenSettings* settings, EILContext* output);

#define EIL_Open VN_EIL_OpenFnName

This is the entry point for creating an instance of the EIL. The open function will have the version of the API appended at the end of the function name to ensure that a newer or older version of the EIL that contains breaking changes cannot be mistakenly used. As such, the EIL_Open define is provided to aid in calling the correct open function.

If the EIL fails to open, then it is typically due to not being able to find and open the requested base encoder. A detailed message as to why the encoder failed to open should be sent to the log callback, if it is provided.

EILContext

typedef struct EILContextImpl* EILContext;

The EILContext is an opaque handle to the EIL's encoder instance. An EILContext is created when the EIL is opened using EIL_Open, and the same instance should be passed to all other functions. It is destroyed when EIL_Close is called.

EILOpenSettings

EILOpenSettings Overview

typedef struct EILOpenSettings
{
    const char* base_encoder;
    void*       base_encoder_userdata;
    void        (*log_callback)(void* userdata, int32_t level, const char* msg);
    void*       log_userdata;
    const char* log_filepath;
    void*       reserved;
    void*       reserved2;
} EILOpenSettings;

EILOpenSettings contains the information required to open the EIL. It can be default initialised using EIL_OpenSettingsDefault.

base_encoder

    const char* base_encoder;

[ Required ]

The name of the base encoder that the EIL uses for base encoding should not contain the preceding lcevc_eilp_ (liblcevc_eilp_ on Linux) or the platform's dynamic library extension. For example, if a plugin is called liblcevc_eilp_x264.so or lcevc_eilp_x264.dll, then the base_encoder string should be x264.

There are two special strings for base_encoder: none and inject:

• none is a pass-through plugin that sets the output recon to the input.

• inject allows for reserved2 to reference an EILPAPI object that contains function pointers for all of a plugin's methods.

base_encoder_userdata

    void*       base_encoder_userdata;

[ Dependent on the plugin | Default: NULL ]

The base_encoder_userdata is a handle that allows some arbitrary data to be passed through to a base plugin.

log_callback

    void        (*log_callback)(void* userdata, int32_t level, const char* msg);
    void*       log_userdata;

[ Optional | Default: NULL ]

log_callback can be set to a method that receives all log messages.

userdata is the value set in log_userdata, and level is one of the levels in the enumeration EILLogLevel. If left unset, then all logging is disabled.

All log messages are in the format level | group | message\n, and are limited to 4096 bytes.

log_filepath

    const char* log_filepath;

[ Optional | Default: NULL ]

log_filepath is an optional file path, that when specified , contains all log messages.

reserved, reserved2

    void*       reserved;
    void*       reserved2;

Reserved fields that should not be modified, unless specified elsewhere.

EIL_InitSettingsDefault and EIL_Initialise

EIL_InitSettingsDefault

VN_EIL_PublicAPI EILReturnCode EIL_InitSettingsDefault(EILInitSettings* settings);

Initialises the members of an EILInitSettings object to sensible defaults. Some fields of the structure must still be populated by the integration before the settings can be used by the EIL.

EIL_Initialise

VN_EIL_PublicAPI EILReturnCode EIL_Initialise(EILContext context, EILInitSettings* settings);

Initialises the EIL so that it is ready to receive pictures for encoding. The context must have previously been created using EIL_Open.

EILInitSettings

EILInitSettingsOverview

typedef struct EILInitSettings
{
    EILColourFormat color_format;
    uint32_t        width;
    uint32_t        height;
    uint32_t        fps_num;
    uint32_t        fps_denom;
    uint32_t        bitrate;
    int32_t         gop_length;
    const char*     properties_json;
    uint8_t         external_input;
} EILInitSettings;

EILInitSettings contains the information required to initialise the EIL. It can be default initialised using EIL_InitSettingsDefault.

color_format

    EILColourFormat color_format;

[ Optional | Default: EIL_YUV_420P ]

The colour format of the input image. If it is not one of the YUV formats, then there will be an extra conversion step that converts the input into the YUV format. The matrix used for the colour conversion can be controlled using the picture_colorspace config option.

width, height

    uint32_t        width;
    uint32_t        height;

[ Required ]

The resolution of the input image. Once the EIL is initialised. then these are modified by the EIL to become the resolution of the base video. These modified values should be used by any operation or process that consumes the bitstream. Consult the LCEVC standard for supported resolutions.

fps_num

    uint32_t        fps_num;
    uint32_t        fps_denom;

[ Required ]

The framerate's numerator and denominator, e.g., 25/1 for 25fps or 30000/1001 for 29.97fps

bitrate

    uint32_t        bitrate;

[ Required ]

The target bitrate, in kilobits/s, at which to encode.

gop_length

    int32_t         gop_length;

[ Optional | Default: EIL_GOPLengthAuto ]

The GOP length in frames. There are three values to which gop_length can be set, subject to the base encoder's abilities:

• EIL_GOPLengthAuto: The EIL determines which GOP length to use.

• EIL_GOPLengthInfinite: Only the first frame is an I-frame.

• EIL_GOPLengthIntraOnly: Every frame is an I-frame.

properties_json

    const char*     properties_json;

[ Required ]

A string containing json properties for the EIL's user-specified options.

external_input

    uint8_t         external_input;

[ Optional | Default: 0 ]

Whether or not the input image's planes should be allocated by the integration or the EIL.

• When set to 0, the EIL allocates and manages the lifetimes of an image's planes.

• When set to 1, the planes are expected to be allocated and managed external to the EIL.

EIL_GetPicture

EIL_GetPicture Overview

VN_EIL_PublicAPI EILReturnCode EIL_GetPicture(EILContext context, EILFrameType frameType, EILPicture** picture);

When the EIL is configured to operate in internal input mode (see external_input under EIL_InitSettings), this method is used to obtain a picture object with preallocated planes. The planes can then be populated with frame data and passed to EIL_Encode. A picture obtained using this method is owned by the integration until that picture is passed back through EIL_Encode.

The EIL internally creates a pool of pictures from which this method extracts. The most common reason for the failure of this method, is that the pool of pictures is empty. This can occur if this function is called too frequently without returning the pictures through EIL_Encode. The size of the pool depends on the base encoder's frame delay plus the EIL's frame delay, where frame delay is the number of frames that must be provided to the encoder before any output is produced.

EILPicture

EILPicture Overview

typedef struct EILPicture
{
    EILMemoryType   memory_type;
    uint32_t        num_planes;
    void*           plane[EIL_MaxPlanes];
    uint32_t        stride[EIL_MaxPlanes];
    EILBaseType     base_type;
    EILFrameType    frame_type;
    EILFieldType    field_type;
    int64_t         pts;
    void*           user_data;
    const void*     reserved;
} EILPicture;

An EILPicture is in the input to the encoder, containing all information needed to encode a single frame of video. If the EIL is configured to operate in external input mode, then it is up to the integration to set the fields of this struct, in which case it can be default initialised using EIL_InitPictureDefault. Otherwise, the EIL will populate the fields of the struct, and the integration should populate the preallocated planes.

memory_type

    EILMemoryType   memory_type;

[ Optional if externally allocated | Default EIL_MT_Host ]

memory_type specifies the type of memory stored in the picture's planes. This can currently only be either:

• EIL_MT_Host is memory that is accessible from the device's CPU, i.e., allocated using malloc or some device memory that has been mapped. If the memory is externally allocated, then the memory must be aligned for 8- or 16-bit access, depending upon the bitdepth.

• EIL_MT_AHardwareBuffer indicates that the planes are Android hardware buffers

  (AHardwareBuffer). The encoder must be configured to use the GPU for this to be accepted, when used in conjunction with external input.

Other memory types:

• EIL_MT_TextureGL is currently unsupported as input to the EIL. The EIL may pass this type of

  memory to the base encoder, depending upon the configuration.

• EIL_MT_Unknown is used only to indicate that the memory type is not yet defined. memory_type should not be set to EIL_MT_Unknown by the point of ingest.

num_planes, plane[EIL_MaxPlanes]

    uint32_t        num_planes;
    void*           plane[EIL_MaxPlanes];

[ Required if externally allocated ]

Pointers to the appropriate structures holding the plane data, as defined by the memory_type. These planes can be interleaved in the cases of the RGB formats. As such, num_planes indicates the number of plane pointers rather than the number of channels. In the case of contiguous memory, each pointer should be set to the starting address of each plane, and the number of planes should be set as if the memory is not contiguous.

stride[EIL_MaxPlanes]

    uint32_t        stride[EIL_MaxPlanes];

[ Required if externally allocated ]

The stride, in bytes, for each plane defined in plane.

base_type

    EILBaseType     base_type;

[ Optional | Default: EIL_BT_Uknown ]

If set to a value other than the default, then when this picture is encoded, the EIL will attempt to force the requested type of picture from the base encoder. Whether or not the requested type of frame is produced, is subject to a base encoder's capabilities.

frame_type, field_type

    EILFrameType    frame_type;
    EILFieldType    field_type;

[ Optional if externally allocated | Default: EIL_FrameType_Progressive ]

Whether this frame is progressive, a single field, or interlaced, and if it is a single field, then whether the frame is the top or bottom field.

pts

    int64_t         pts;

[ Optional | Defaut: -1 ]

The presentation timestamp of the picture. If left as the default, then it is determined by the EIL.

user_data

    void*           user_data;

[ Optional | Default: NULL ]

A handle to some arbitrary data that is set on the EILOutput produced by the picture.

reserved

    const void*     reserved;

Internally used field. Do not modify.

EIL_Encode

VN_EIL_PublicAPI EILReturnCode EIL_Encode(EILContext context, EILPicture* picture);

Passes a picture that is ready for encode to the EIL. A picture can be obtained from the EI, either:

• Using EIL_GetPicture, if external input is disabled; or,

• Created and populated by the integration, if external input is enabled.

EIL_GetOutput and EIL_ReleaseOutput

Get_Output

VN_EIL_PublicAPI EILReturnCode EIL_GetOutput(EILContext context, EILOutput** output);

Gets the next output available from the EIL. The integration has ownership of the EILOutput until it has been passed back through EIL_ReleaseOutput. This can return one of two return codes upon success:

• EIL_RC_Success: Successfully got the next frame of output.

• EIL_RC_Finished: No errors occurred but there there is currently no output available.

Release_Output

VN_EIL_PublicAPI EILReturnCode EIL_ReleaseOutput(EILContext context, EILOutput* output);

Returns an output obtained using EIL_GetOutput back to the encoder.

EILOutput

• EILOutput Overview

typedef struct EILOutput
{
    EILFrameType    frame_type;
    EILFieldType    field_type;
    EILBaseType     base_type;
    int8_t          keyframe;
    int32_t         qp;
    int64_t         pts;
    int64_t         dts;
    int8_t          config;
    const uint8_t*  data;
    uint32_t        data_length;
    const uint8_t*  lcevc;
    uint32_t        lcevc_length;
    void*           user_data;
    const void*     reserved;
} EILOutput;

The EILOutput struct contains all information for an encoded frame of video. These should never have to be created manually, and should be treated as read-only.

• frame_type, field_type

    EILFrameType    frame_type;
    EILFieldType    field_type;

Whether the frame that produced this output is progressive, interlaced or a field. field_type is applicable, only if the type is EIL_FT_Field.

• base_type, keyframe

    EILBaseType     base_type;
    int8_t          keyframe;

The type of frame that the base encoder produced, and whether or not it is flagged as a keyframe. Refer to the appropriate base encoder's standard for their meanings.

• qp

    int32_t         qp;

The Quantisation Parameter (QP) of the frame, as reported by the base encoder.

• pts

    int64_t         pts;

The presentation timestamp of the encoded frame. This is the input frame's pts, if it was specified; otherwise, it is the pts that the base encoder produced.

• dts

    int64_t         dts;

The decode timestamp of the encoded frame, as produced by the base encoder.

• config

    int8_t          config;

Whether the data in this output is the encoder's config. For example, if the base encoder is an H264 encoder and config is 1, then the output contains only the SPS and PPS NAL units. The config output is always presented before any frame data.

• data, data_length, lcevc, lcevc_length

    const uint8_t*  data;
    uint32_t        data_length;
    const uint8_t*  lcevc;
    uint32_t        lcevc_length;

The encoded bitstream for the frame and its length. If the separate_output config option is set, then the LCEVC data is on the lcevc field; otherwise, it is contained in the data field, along with the base bitstream.

• user_data

    void*           user_data;

The user data that was set on the picture that produced this frame.

• reserved

    const void*     reserved;

Internally used field. Do not modify.

EIL_QueryPropertyGroups, EIL_QueryProperty

EIL_QueryPropertyGroups, EIL_QueryProperty Overview

VN_EIL_PublicAPI EILReturnCode EIL_QueryPropertyGroups(EILContext context, EILPropertyGroups* output);
VN_EIL_PublicAPI EILReturnCode EIL_QueryProperty(EILContext context, const char* name, EILProperty* output);

This group of methods is used to query the options available through the EIL. These are usually controlled by the user.

• EIL_QueryPropertyGroups returns a collection of all properties in their logical groupings.

• EIL_QueryProperty obtains an individual property by its name.

The properties and their values are passed to the EIL in JSON format on the EILInitSettings structure through the properties_json field.

EILPropertyGroups

typedef struct EILPropertyGroups
{
    EILPropertyGroup*   group;
    uint32_t            group_count;
} EILPropertyGroups;

This structure contains an array of all property groups.

typedef struct EILPropertyGroup
{
    const char*     name;
    const char*     description;
    EILProperty*    properties;
    uint32_t        property_count;
} EILPropertyGroup;

The property group contains a logical grouping of all properties available through the EIL and base encoders.

EILProperty

EILProperty Overview

typedef struct EILProperty
{
    const char*     name;
    const char*     description;
    EILPropertyType type;

    union
    {
        int32_t     i32;
        int64_t     i64;
        uint32_t    u32;
        uint64_t    u64;
        float       f;
        double      d;
        const char* str;
    } value;

    const uint8_t*  blob;
    uint32_t        blob_size;
} EILProperty;

This structure is used to describe config properties or metadata of the EIL and the base encoder.

name, description

    const char*     name;
    const char*     description;

A short name and description of the property that was queried.

type

    EILPropertyType type;

The type of value contained in this property. This value should be used to determine which field of the value union is set:

• EIL_PT_Int32: value.i32

• EIL_PT_Int64: value.i64

• EIL_PT_Uint32: value.u32

• EIL_PT_Uint64: value.u64

• EIL_PT_Float: value.f

• EIL_PT_Double: value.d

• EIL_PT_String: value.str

• EIL_PT_Boolean: value.i32

blob, blob_size

    const uint8_t*  blob;
    uint32_t        blob_size;

blob and blob_size are used when the property type is EIL_PT_Blob. This is only ever the case when querying metadata.