Contents¶

Contents

How-To Guide for Writing UDF¶

This document describes stepwise instruction for writing a User defined Function (UDF) in C++/Python to make it deployable in the Open Edge Insights for Industrial (Open EII) environment. It explains APIs and configurational changes required in order to write an UDF for Open EII.

Note

In this document, you will find labels of ‘Edge Insights for Industrial (EII)’ for file names, paths, code snippets, and so on. Consider the references of EII as Open EII. This is due to the product name change of EII as Open EII.

Introduction¶

UDFs(User Defined Function) are one of the cardinal feature of Open EII framework. It enables users to adjoin any pre-processing or post-processing logic in data pipeline defined by Open EII configuration. As of Open EII 2.1 release, it supports UDFs to be implemented using following languages.

C++ (It is also called Native UDF as Open EII core components are implemented in C++)
Python

The order in which the UDFs are defined in Open EII configuration file is the order in which data will flow across them. Currently there is no support for demux/mux the data flow to/fro the UDFs.

All configs related to UDFs are to be in config.json of apps like VideoIngestion and VideoAnalytics.The UDF schema and description about keys/values in it presented in detail in the UDF README file.

Steps for Writing Native (C++) UDFs¶

Every UDF writing has two major part to it.

Writing actual pre/post processing logic using Open EII exposed APIs.
Adding Open EII infra specific configuration component for deploying it

Open EII APIs for Writing Native UDFs (C++)¶

There are three APIs defined semantically to add the pre/post processing logic. These APIs must be implemented as method of a user defined class inherited from the Udf class named **BaseUdf**.

#### Initialization and DeInitialization
```
class DummyUdf : public BaseUdf {
        public:
            DummyUdf(config_t* config) : BaseUdf(config) {
                //initialization Code can be added here
            };

            ~DummyUdf() {
                // Any de-initialization logic to be added here
            };
};
```
The DummyUdf in the above code snippet is the user defined class and the constructor of the class initialize the UDF’s specific data-structure. The only argument passed to this function is config which depicts configuration details mentioned in the config.json file of apps liks VideoIngestion and VideoAnalytics which process UDFs. The API to consume config is defined in Util README
#### Processing the Actual Data

The API to utilize the ingested input looks as below:
```
UdfRetCode
process(cv::Mat& frame, cv::Mat& outputFrame, msg_envelope_t* meta) override {
    // Logic for processing the frame & returning the inference result.
}
```
This function is a override method which user need to define in its UDF file.

The argument details are described as below:
- Argument 1(cv::Mat &frame): It represents the input frame for inference.
- Argument 2(cv::Mat &outputFrame): It represents the modified frame by the user. This can be used if user need to pass a modified frame forward.
- Argument 3(msg_envelope_t* meta): It represents the inference result returned by UDF. The user need to fill the msg_envelope_t structure as described in following **EIIMsgEnv README**. There are sample code suggested in the README which explains the API usage in detail.
The return code details are described as below:
- UdfRetCode: User need to return appropriate macro as mentioned below:
- UDF_OK - UDF has processed the frame gracefully.
- UDF_DROP_FRAME - The frame passed to process function need to be dropped.
- UDF_ERROR - it should be returned for any kind of error in UDF.
#### Linking Udfloader and Custom UDFs

The initialize_udf() function need to defined as follows to create a link between UdfLoader module and respective UDF. This function ensure UdfLoader to call proper constructor and process() function of respective UDF.
```
extern "C" {
    void *initialize_udf(config_t *config) {
        DummyUdf *udf = new DummyUdf(config);
        return (void *)udf;
    }
}
```
The “DummyUdf” is the class name of the user defined custom UDF.

Open EII APIs for Writing Raw Native UDFs(c++) for Multi Frame Support¶

There are three APIs defined semantically to add the pre/post processing logic. These APIs must be implemented as method of a user defined class inherited from the Udf class named **RawUdf**.

#### Initialization and Deinitialization
```
class RealSenseUdf : public RawBaseUdf {
        public:
            RealSenseUdf(config_t* config) : RawBaseUdf(config) {
                //initialization Code can be added here
            };

            ~RealSenseUdf() {
                // Any de-initialization logic to be added here
            };
};
```
The RealSenseUdf in the above code snippet is the user defined class and the constructor of the class initialize the UDF’s specific data-structure. The only argument passed to this function is config which depicts configuration details mentioned in the config.json file of apps liks VideoIngestion and VideoAnalytics which process UDFs. The API to consume config is defined in Util README
#### Processing Actual Data

The API to utilize the ingested input looks as below:
```
UdfRetCode
process(Frame* frame) override {
    // Logic for processing the frame & returning the inference result.
}
```
This function is a override method which user need to define in its UDF file.

The argument details are described as below:
Argument 1(Frame* frame): It represents the frame object.

The return code details are described as below:
- UdfRetCode: User need to return appropriate macro as mentioned below:
- UDF_OK - UDF has processed the frame gracefully.
- UDF_DROP_FRAME - The frame passed to process function need to be dropped.
- UDF_ERROR - it should be returned for any kind of error in UDF.
#### Linking Udfloader and Custom-UDFs

The initialize_udf() function need to defined as follows to create a link between UdfLoader module and respective UDF. This function ensure UdfLoader to call proper constructor and process() function of respective UDF.
```
extern "C" {
    void *initialize_udf(config_t *config) {
        RealSenseUdf *udf = new RealSenseUdf(config);
        return (void *)udf;
    }
}
```
The “RealSenseUdf” is the class name of the user defined custom UDF.

Steps for Writing Python UDFs¶

This section describes the process of writing a Python UDF. As discussed in the aforementioned scenario, it also has two aspects to it.

Writing the actual UDF. It needs knowledge Open EII exposed python APIs
Adding the UDF to Open EII framework by altering different configs.

Python APIs for writing UDF for Open EII¶

Initialization

In case of Python the initialization callback is also same as the native case. User must create a **Udf class** and the **__init__()** function defined in class act as a initialization routine for custom UDF. A dummy constructor code is pasted below:
```
class Udf:
"""Example UDF
"""
def __init__(self):
    """Constructor
    """
    # Add the initialization code in this method.
```
Process Actual Data

The API used to process the actual frame looks as below.
```
process(self, frame, metadata):
    # Process the frame in this method
    # metadata can be used to return inference result
```
Argument:

frame: Image frame in numpy’s ndarray format

metadata: An empty dictionary. Inference results can be inserted in this data structure.

Return value:

This function returns three values.

1st Value : Represents if the frame Need to be dropped or Not. It is boolean in nature. In case of failure user can return True in this positional return value.

2nd Value : It represents the actual modified frame if at all it has been modified. Hence the type is numpy’s ndarray. If the frame is not modified user can return a None in this place.

3rd Value : Metadata is returned in this place. Hence the type is dict. In general user can return the passed argument as part of this function.

For reference user can find example UDFs code in below mentioned links

PCB_FILTER([WORK_DIR]/IEdgeInsights/common/video/udfs/python/pcb/pcb_filter.py)
PCB_CLASSIFIER([WORK_DIR]/IEdgeInsights/common/video/udfs/python/pcb/pcb_classifier.py)
Dummy UDF([WORK_DIR]/IEdgeInsights/common/video/udfs/python/dummy.py)

Open EII Infrastructure Changes¶

For any UDF to be utilized by the Open EII infrastructure, user must follow the below steps.

Corresponding UDF entry must be added to the config.json file of apps like VideoIngestion and VideoAnalytics. The UDF entry syntax is explained in detail in the following document UDF README
All the python UDFs must be kept under python udf directory([WORK_DIR]/IEdgeInsights/common/video/udfs/python). Additionally the entry name key must have the file hierarchy till the file name as the name of the udf. For example: A file present in this path ./python/pcb/pcb_filter.py must have the name field as pcb.pcb_filter.

This syntax is described in detail in the UDF README.

Conclusion¶

The UDFs currently supported using python and C++. UDF uses in-memory message passing instead of sockets for the communication between the pipeline and UDFs making it faster in comparison. OEI has many sample UDFs can be found in following paths for C++([WORK_DIR]/IEdgeInsights/common/video/udfs/native) & for python([WORK_DIR]/IEdgeInsights/common/video/udfs/python)