Contents

• Contents

  • EII Video Profiler

    • EII Video Profiler Prerequisites

    • EII Video Profiler Modes

    • EII Video Profiler Configuration

    • Run Video Profiler

    • Run VideoProfiler in Helm Use Case

    • Optimize EII Video Pipeline by Analysing Video Profiler Results

    • Benchmarking with Multi-instance Config

EII Video Profiler

This tool can be used to determine the complete metrics involved in the entire video pipeline by measuring the time difference between every component of the pipeline and checking for queue blockages at every component, thereby determining the fast or slow components of the whole pipeline. It can also be used to calculate the FPS of any EII modules based on the stream published by that respective module.

EII Video Profiler Prerequisites

1. VideoProfiler expects a set of config, interfaces, and public private keys to be present in ETCD as a prerequisite.

   To achieve this, ensure an entry for VideoProfiler with its relative path from IEdgeInsights([WORK_DIR]/IEdgeInsights/) directory is set in any of the.yml files present in build/usecases([WORK_DIR]/IEdgeInsights/build/usecases) directory. Following is an example:

   AppContexts:
   - VideoIngestion
   - VideoAnalytics
   - tools/VideoProfiler
   

2. With the previous prerequisite done, run the following command:

   python3 builder.py -f usecases/video-streaming.yml
   

EII Video Profiler Modes

By default, the EII Video Profiler supports the FPS and the Monitor modes. The following are details for these modes:

• FPS mode Enabled by setting the’mode’ key in config([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/config.json) to ‘fps’, this mode calculates the frames per second of any EII module by subscribing to that module’s respective stream.

  "mode": "fps"
  

  Note: For running the Video Profiler in the FPS mode, it is recommended to keep PROFILING_MODE set to false in .env([WORK_DIR]/IEdgeInsights/build/.env) for better performance.

• Monitor mode Enabled by setting the ‘mode’ key in config([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/config.json) to ‘monitor’, this mode calculates average and per-frame stats for every frame while identifying if the frame was blocked at any queue of any module across the video pipeline, thereby determining the fastest and slowest components in the pipeline. To be performant in profiling scenarios, VideoProfiler is enabled to work when subscribing only to a single topic in monitor mode.

 The user must ensure that the ingestion_appname and analytics_appname fields of the monitor_mode_settings need to be set accordingly for monitor mode.


Refer to the following example configuration where VideoProfiler is used in monitor mode to subscribe to PySafetyGearAnalytics CustomUDF results.

    "config": {
    "mode": "monitor",
    "monitor_mode_settings": {
                                "ingestion_appname": "PySafetyGearIngestion",
                                "analytics_appname": "PySafetyGearAnalytics",
                                "display_metadata": false,
                                "per_frame_stats":false,
                                "avg_stats": true
                            },
    "total_number_of_frames" : 5,
    "export_to_csv" : false
}

"mode": "monitor"

The stats to be displayed by the tool in monitor_mode can be set in the monitor_mode_settings key of config.json([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/config.json).

• ‘display_metadata’: It displays the raw meta-data with timestamps associated with every frame.

• ‘per_frame_stats’: It continuously displays the per-frame metrics of every frame.

• ‘avg_stats’: It continuously displays the average metrics of every frame.

Note

• As a prerequisite for running in profiling or monitor mode, VI/VA should be running with the PROFILING_MODE set to true in .env([WORK_DIR]/IEdgeInsights/build/.env)

• It is mandatory to have a UDF to run in monitor mode. For instance, GVASafetyGearIngestion does not have any UDF (since it uses GVA elements), so it will not be supported in monitor mode. The workaround to use GVASafetyGearIngestion in the monitor mode is to add dummy-udf in GVASafetyGearIngestion-config([WORK_DIR]/IEdgeInsights/CustomUdfs/GVASafetyGearIngestion/config.json).

EII Video Profiler Configuration

Following are the EII Video Profiler configurations:

1. dev_mode

   Setting this to false enables secure communication with the EII stack. The user must ensure this switch is in sync with DEV_MODE in .env([WORK_DIR]/IEdgeInsights/build/.env) with PROD mode enabled, the path for the certs mentioned in config([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/config.json) can be changed by the user to point to the required certs.

2. total_number_of_frames

   If mode is set to ‘fps’, the average FPS is calculated for the number of frames set by this variable. If mode is set to “monitor”, the average stats are calculated for the number of frames set by this variable. Setting it to (-1) will run the profiler forever unless terminated by signal interrupts(‘Ctrl+C’).total_number_of_frames should never be set as (-1) for ‘fps’ mode.

3. export_to_csv

   Setting this switch to true exports csv files for the results obtained in VideoProfiler. For monitor_mode, runtime stats printed in the csv are based on the following precdence: avg_stats, per_frame_stats, display_metadata.

Run Video Profiler

Following are the steps to run the video profiler:

1. Set the environment variables accordingly in config.json([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/config.json).

2. Set the required output stream or streams and the appropriate stream config in the config.json([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/config.json) file.

3. If VideoProfiler is subscribing to multiple streams, ensure the AppName of VideoProfiler is added to the Clients list of all the publishers.

4. If using Video Profiler in IPC mode, ensure to set the required permissions to socket file created in SOCKET_DIR in build/.env([WORK_DIR]/IEdgeInsights/build/.env).

   sudo chmod -R 777 /opt/intel/eii/sockets
   

   Note:

   • This step is required every time the publisher is restarted in IPC mode.

   • Caution: This step will make the streams insecure. Do not do it on a production machine.

   • Refer the following VideoProfiler interface example to subscribe to PyMultiClassificationIngestion CustomUDF results in the FPS mode:

   "/VideoProfiler/interfaces": {
          "Subscribers": [
              {
                  "EndPoint": "/EII/sockets",
                  "Name": "default",
                  "PublisherAppName": "PyMultiClassificationIngestion",
                  "Topics": [
                      "py_multi_classification_results_stream"
                  ],
                  "Type": "zmq_ipc"
              }
          ]
      },
   

5. If you’re using VideoProfiler with the Helm usecase or trying to subscribe to any external publishers outside the EII network, ensure the correct IP of the publisher is specified in the interfaces section of [config] (config.json) and the correct ETCD host and port are specified in the environment for ETCD_ENDPOINT and ETCD_HOST.

   • For example, for the helm use case, since the ETCD_HOST and ETCD_PORT are different, run the following commands with the required HOST IP:

     export ETCD_HOST="<HOST IP>"
     export ETCD_ENDPOINT="<HOST IP>:32379"
     

6. To provision, build, and run the tool along with the EII video-streaming recipe or stack, see provision/README.md.

7. Ensure the containers VideoProfiler is subscribing to are up and running before trying to bring up VideoProfiler.

   • For example, if VideoProfiler is subcribed to VideoAnalytics, use these commands to bring up the entire EII stack and then restart VideoProfiler once the publishing containers are up and running:

     docker-compose up -d
     # Restart VideoProfiler after the publisher is up
     # by checking the logs
     docker restart ia_video_profiler
     

8. Run the following command to see the logs:

   docker logs -f ia_video_profiler
   

9. The runtime stats of Video Profiler, if enabled with the export_to_csv switch, can be found at [video_profiler_runtime_stats]. (video_profiler_runtime_stats.csv)

   Note:

   • poll_interval option in the VideoIngestion config([WORK_DIR]/IEdgeInsights/VideoIngestion/config.json) sets the delay(in seconds) to be induced after every consecutive frame is read by the opencv ingestor. Not setting it will ingest frames without any delay.

   • videorate element in the VideoIngestion config([WORK_DIR]/IEdgeInsights/VideoIngestion/config.json) can be used to modify the ingestion rate for gstreamer ingestor. For more information refer to README.

   • ZMQ_RECV_HWM option specifies the maximum number of inbound messages on the subscriber socket. The high water is a hard limit on the maximum number of outstanding messages ZeroMQ shall queue in memory for any single peer that the specified socket is communicating. If this limit has been reached, the socket shall enter an exeptional state and will reject any incoming messages.

   • If we are running Video Profiler for the GVA use case, we do not display the stats of the algorithm running with GVA since no UDFs are used.

   • The rate at which the UDFs process the frames can be measured using the FPS UDF and the ingestion rate can be monitored accordingly. If multiple UDFs are used, the FPS UDF is required to be added as the last UDF.

   • If running this tool with VI and VA in two different nodes, the same time needs to be set in both the nodes.

Run VideoProfiler with EdgeVideoAnalyticsMicroservice

For running VideoProfiler with EdgeVideoAnalyticsMicroservice as a publisher, the config can be updated to subscribe to the EndPoint and topic of EdgeVideoAnalyticsMicroservice in the following way:

{
    "interfaces": {
        "Subscribers": [
            {
                "Name": "default",
                "Type": "zmq_tcp",
                "EndPoint": "ia_edge_video_analytics_microservice:65114",
                "PublisherAppName": "EdgeVideoAnalyticsMicroservice",
                "Topics": [
                    "edge_video_analytics_results"
                ]
            }
        ]
    }
}

Run VideoProfiler in Helm Use Case

For running VideoProfiler in the helm use case to subscribe to either VideoIngestion or VideoAnalytics or any other EII service, the etcd endpoint, volume mount for helm certs, and service endpoints are to be updated.

For connecting to the ETCD server running in the helm environment, the endpoint and required volume mounts should be modified in the following manner in the environment and volumes section of docker-compose.yml([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/docker-compose.yml):

ia_video_profiler:
  ...
  environment:
  ...
    ETCD_HOST: ${ETCD_HOST}
    ETCD_CLIENT_PORT: ${ETCD_CLIENT_PORT}
    # Update this variable referring
    # for helm use case
    ETCD_ENDPOINT: <HOST_IP>:32379
    CONFIGMGR_CERT: "/run/secrets/VideoProfiler/VideoProfiler_client_certificate.pem"
    CONFIGMGR_KEY: "/run/secrets/VideoProfiler/VideoProfiler_client_key.pem"
    CONFIGMGR_CACERT: "/run/secrets/rootca/cacert.pem"
  ...
  volumes:
    - "${EII_INSTALL_PATH}/tools_output:/app/out"
    - "${EII_INSTALL_PATH}/sockets:${SOCKET_DIR}"
    - ./helm-eii/eii-deploy/Certificates/rootca:/run/secrets/rootca
    - ./helm-eii/eii-deploy/Certificates/VideoProfiler:/run/secrets/VideoProfiler

For connecting to any service running in the helm usecase, the container IP associated with the specific service should be updated in the Endpoint section in VideoProfiler config([WORK_DIR]/IEdgeInsights/tools/VideoProfiler/config.json).

The IP associated with the service container can be obtained by checking the container pod IP using docker inspect. Assuming we are connecting to the VideoAnalytics service, execute the following command:

docker inspect <VIDEOANALYTICS CONTAINER ID> | grep VIDEOANALYTICS

The output of the previous command consists of the IP of the VideoAnalytics container that can be updated in the VideoProfiler config using EtcdUI:

"VIDEOANALYTICS_SERVICE_HOST=10.99.204.80"

The config can be updated with the obtained container IP in the following way:

{
    "interfaces": {
        "Subscribers": [
            {
                "Name": "default",
                "Type": "zmq_tcp",
                "EndPoint": "10.99.204.80:65013",
                "PublisherAppName": "VideoAnalytics",
                "Topics": [
                    "camera1_stream_results"
                ]
            }
        ]
    }
}

Optimize EII Video Pipeline by Analysing Video Profiler Results

1. If the VI ingestor/UDF input queue is blocked, consider reducing the ingestion rate.

   If this log is displayed by the Video Profiler tool, it indicates that the ingestion rate is too high or that the VideoIngestion UDFs are slow and causing latency throughout the pipeline. As per the log suggests, the user can increase the poll_interval to an optimum value to reduce the blockage of the VideoIngestion ingestor queue, thereby optimizing the video pipeline in the case of using the openCV ingestor. If Gstreamer ingestor is used, the videorate option can be optimized by following the README.

2. If the VA subs or UDF input queue is blocked, reduce the ZMQ_RECV_HWM value or the ingestion rate.

   If this log is displayed by the Video Profiler tool, it indicates that the VideoAnalytics UDFs are slow and causing latency throughout the pipeline. As per the log suggests, the user can consider reducing ZMQ_RECV_HWM to an optimum value to free the VideoAnalytics UDF input or subscriber queue by dropping incoming frames or reducing the ingestion rate to a required value.

3. If the UDF VI output queue is blocked,

   If this log is displayed by the Video Profiler tool, it indicates that the VI to VA message bus transfer is delayed.

   1. Users can consider reducing the ingestion rate to a required value.

   2. The user can increase ZMQ_RECV_HWM to an optimum value so as to not drop the frames when the queue is full or switching to the IPC mode of communication.

4. If the UDF VA output queue is blocked,

   If this log is displayed by the Video Profiler tool, it indicates that the VA to VideoProfiler message bus transfer is delayed.

   1. User can consider reducing the ingestion rate to a required value.

   2. The user can increase ZMQ_RECV_HWM to an optimum value so as to not drop the frames when the queue is full or switching to the IPC mode of communication.

Benchmarking with Multi-instance Config

1. EII supports multi-instance config generation for benchmarking purposes. This is accomplished by running the builder.py([WORK_DIR]/IEdgeInsights/build/builder.py) with specific parameters, refer to the Multi-instance Config Generation section of EII Pre-requisites in the README for more details.

2. For running VideoProfiler for multiple streams, run the builder with the -v flag provided the pre-requisites mentioned previously are done. The following is an example for generating 6 streams config:

   python3 builder.py -f usecases/video-streaming.yml -v 6
   

Note

• For multi-instance monitor mode use case, ensure only VideoIngestion and VideoAnalytics are used as AppName for the Publishers.

• Running VideoProfiler with CustomUDFs for monitor mode is supported for single stream only. If multiple streams are required, ensure VideoIngestion & VideoAnalytics are used as AppName.

• In IPC mode, for accelerators: MYRIAD, GPU, and USB 3.0 Vision cameras, add user: root in VideoProfiler-docker-compose.yml([WORK_DIR]/IEdgeInsights/docker-compose.yml) as the subscriber needs to run as root if the publisher is running as root.