1 of 3

Enable GPU Acceleration

OvenMediaEngine supports GPU-based hardware decoding and encoding. Currently supported GPU acceleration devices are Intel's QuickSync and NVIDIA. This article explains how to install the drivers for OvenMediaEngine and set up the configuration to use your GPU.

1. Install Drivers

1. Install NVIDIA GPU Driver

If you are using an NVIDIA graphics card, please refer to the following guide to install the driver. The OS that supports installation with the provided script are CentOS 7/8 and Ubuntu 18/20 versions. If you want to install the driver in another OS, please refer to the manual installation guide document.

CentOS environment requires the process of uninstalling the nouveau driver. After uninstalling the driver, the first reboot is required, and a new NVIDIA driver must be installed and rebooted. Therefore, two install scripts must be executed.

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz)
OvenMediaEngine-master/misc/install_nvidia_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver is operating normally with the nvidia-smi command.

$ nvidia-smi

Thu Jun 17 10:20:23 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 20%   35C    P8    N/A /  75W |    156MiB /  1997MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

2 . Prerequisites

If you have finished installing the driver to use the GPU, you need to reinstall the open source library using Prerequisites.sh . The purpose is to allow external libraries to use the installed graphics driver.

OvenMediaEngine-master/misc/prerequisites.sh --enable-nvc

1. Install NVIDIA GPU Driver

Please refer to the NVIDIA Driver installation guide written previously.

2. Install NVIDIA Container Toolkit

To use GPU acceleration in Docker, you need to install NVIDIA drivers on your host OS and install the NVIDIA Container Toolkit. This toolkit includes container runtime libraries and utilities for using NVIDIA GPUs in Docker containers.

OvenMediaEngine-master/misc/install_nvidia_docker_container.sh

3 . Build Image

A Docker Image build script that supports NVIDIA GPU is provided separately. Please refer to the previous guide for how to build

OvenMediaEngine-master/Dockerfile.cuda
OvenMediaEngine-master/Dockerfile.cuda.local

1. Install Intel QuickSync Driver

If you are using an Intel CPU that supports QuickSync, please refer to the following guide to install the driver. The OSes that support installation using the provided scripts are CentOS 7/8 and Ubuntu 18/20 versions. If you want to install the driver on a different OS, please refer to the Manual Installation Guide document.

When the Intel QuickSync driver installation is complete, the OS must be rebooted for normal operation.

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz) 
OvenMediaEngine-master/misc/install_intel_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver operates normally with the Matrix Monitor program.

# Use the samples provided in the Intel Media SDK
# Check the list of codecs supported by iGPU
/MediaSDK-intel-mediasdk-21.1.2/build/__bin/release/simple_7_codec

2. Prerequisites

Using Intel QuickSync GPU

OvenMediaEngine-master/misc/prerequisites.sh --enable-qsv

1. Install XCODER

Please refer to the Netint documentation to install XCODER.

How to check driver installation

After the driver installation is complete, check if the libxcoder exist: the CLI must return something like libxcoder_logan.so (libc6,x86-64) => /usr/local/lib/libxcoder_logan.so

ldconfig -p | grep libxcoder_logan.so

2. Prerequisites

If you have finished installing the driver to use the VPU, you need to reinstall the open source library using Prerequisites.sh . The purpose is to allow external libraries to use the installed graphics driver. You also have to unzip the ffmpeg patch provide by netint in a specfic path

Using Netint VPU

./prerequisites.sh --enable-nilogan --nilogan-path=/root/T4xx/release/FFmpeg-n5.0_t4xx_patch

2. Build & Run

Please refer to the link for how to build and run.

Intructions on running Docker

you must include the --gpus all option when running Docker

docker run -d ... --gpus all airensoft/ovenmediaengine:dev

3. Configuration

To use hardware acceleration, set the HardwareAcceleration option to true under OutputProfiles. If this option is enabled, a hardware codec is automatically used when creating a stream, and if it is unavailable due to insufficient hardware resources, it is replaced with a software codec.

<VirtualHosts>
   <VirtualHost>
      <Name>default</Name>
      ...
      <!-- Settings for applications -->
      <Applications>
         <Application>
            <Name>app</Name>
            <Type>live</Type>
            <OutputProfiles>
               <!-- Settings to use hardware codecs -->
               <HardwareAcceleration>true</HardwareAcceleration>
               <OutputProfile>
                  ...
               </OutputProfile>
               <OutputProfile>
                  ...
               </OutputProfile>
            </OutputProfiles>
            <Providers>
               ...
            </Providers>
            <Publishers>
               ...
            </Publishers>
         </Application>
      </Applications>
   </VirtualHost>
</VirtualHosts>

Appendix. Support Format

The codecs available using hardware accelerators in OvenMediaEngine are as shown in the table below. Different GPUs support different codecs. If the hardware codec is not available, you should check if your GPU device supports the codec.

D : Decoding, E : Encoding

Reference

Quick Sync Video Format : https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
NVIDIA NVDEC Video Format : https://en.wikipedia.org/wiki/Nvidia_NVDEC
NVIDIA NVENV Video Format : https://en.wikipedia.org/wiki/Nvidia_NVENC
CUDA Toolkit Installation Guide : https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#introduction
NVIDIA Container Toolkit : https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/arch-overview.html#arch-overview
Quick Sync Video format support: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video

TranscodeWebhook

TranscodeWebhook allows OvenMediaEngine to use OutputProfiles from the Control Server's response instead of the OutputProfiles in the local configuration (Server.xml). OvenMediaEngine requests OutputProfiles from the Control Server when streams are created, enabling the specification of different profiles for each individual stream.

Configuration

<Applications>
	<Application>
		<Name>app</Name>
		<!-- Application type (live/vod) -->
		<Type>live</Type>

		<TranscodeWebhook>
			<Enable>true</Enable>
			<ControlServerUrl>http://example.com/webhook</ControlServerUrl>
			<SecretKey>abc123!@#</SecretKey>
			<Timeout>1500</Timeout>
			<UseLocalProfilesOnConnectionFailure>true</UseLocalProfilesOnConnectionFailure>
                        <UseLocalProfilesOnServerDisallow>false</UseLocalProfilesOnServerDisallow>
                        <UseLocalProfilesOnErrorResponse>false</UseLocalProfilesOnErrorResponse>
		</TranscodeWebhook>

Enable (required) You can enable or disable TranscodeWebhook settings.

ControlServerUrl (required) It's the URL of the Control Server, and it supports both HTTP and HTTPS.

SecretKey (optional) This is the Secret Key used to pass authentication for the Control Server. To pass security authentication, an HMAC-SHA1 encrypted value of the HTTP Payload is added to the HTTP Header's X-OME-Signature. This Key is used for generating this value.

Timeout (optional, default: 1500) This is the Timeout value used when connecting to the Control Server.

UseLocalProfilesOnConnectionFailure(optional, default: true) This determines whether to use the OutputProfiles from Local settings in case of communication failure with the Control Server. If it's set to "false," a communication failure with the Control Server will result in a failure to create the Output stream.

UseLocalProfilesOnServerDisallow (optional, default: false) When the Control Server responds with a 200 OK, but "allowed" is set to "false," this policy is followed.

UseLocalProfilesOnErrorResponse (optional, default: false) When the Control Server responds with error status codes such as 400 Bad Request, 404 Not Found, 500 Internal Error, OvenMediaEngine follows this policy.

Protocol

Request (OME → Control Server)

OvenMediaEngine sends requests to the Control Server in the following format.

POST /configured/target/url/ HTTP/1.1
Content-Length: 944
Content-Type: application/json
Accept: application/json
X-OME-Signature: f871jd991jj1929jsjd91pqa0amm1
{
	"source" : "TCP://192.168.0.200:8843",
	"stream" : 
	{
    	"name" : "stream",
		"virtualHost" : "default",
		"application" : "app",
		"sourceType" : "Rtmp",
		"createdTime" : "2023-10-24T18:10:51.120+09:00",
		"sourceUrl" : "TCP://192.168.0.200:8843",
		"tracks" : 
		[
			{
				"id" : 0,
				"name" : "Video",
				"type" : "Video",
				"video" : 
				{
					"bitrate" : "2500000",
					"bypass" : false,
					"codec" : "H264",
					"framerate" : 30.0,
					"hasBframes" : false,
					"height" : 720,
					"keyFrameInterval" : 0,
					"width" : 1280
				}
			},
			{
				"audio" : 
				{
					"bitrate" : "160000",
					"bypass" : false,
					"channel" : 2,
					"codec" : "AAC",
					"samplerate" : 44100
				},
				"id" : 1,
				"name" : "Audio",
				"type" : "Audio"
			},
			{
				"id" : 2,
				"name" : "Data",
				"type" : "Data"
			}
		]
	}
}

Response (Control Server → OME)

The Control Server responds in the following format to specify OutputProfiles for the respective stream.

HTTP/1.1 200 OK
Content-Length: 102
Content-Type: application/json
Connection: Closed
{
  "allowed": true,
  "reason": "it will be output to the log file when "allowed" is false",
  "outputProfiles" : {
    "hardwareAcceleration": true,
    "outputProfile":[
      {
        "name": "bypass",
        "outputStreamName": "${OriginStreamName}",
        "playlists": [
          {
            "name": "default",
            "fileName": "default",
            "renditions":[
              {
                "name": "bypass",
                "video": "bypass_video",
                "audio": "bypass_audio"
              }
            ]
          }
        ],
        "encodes": {
          "videos": [
            {
              "name": "bypass_video",
              "bypass": true
            }
          ],
          "audios": [
            {
              "name": "bypass_audio",
              "bypass": true
            }
          ]
        }
      }
    ]
  }
}

The "outputProfiles" in the JSON is identical to the configuration in Server.xml, and the format is as follows.

"outputProfiles": {
    "hardwareAcceleration": true,
    "outputProfile": [
        {
            "name": "bypass",
            "outputStreamName": "${OriginStreamName}",
            "encodes": {
                "audios": [
                    {
                        "name": "bypass_audio",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "bitrate": "",
                        "samplerate": 0,
                        "channel": 0,
                        "bypassIfMatch": {
                            "codec": "",
                            "bitrate": "",
                            "framerate": "",
                            "width": "",
                            "height": "",
                            "sar": "",
                            "samplerate": "",
                            "channel": ""
                        }
                    }
                ],
                "videos": [
                    {
                        "name": "bypass_video",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "profile": "",
                        "bitrate": "",
                        "width": 0,
                        "height": 0,
                        "framerate": 0.000000,
                        "preset": "",
                        "threadCount": -1,
                        "keyFrameInterval": 0,
                        "bFrames": 0,
                        "bypassIfMatch": {
                            "codec": "",
                            "bitrate": "",
                            "framerate": "",
                            "width": "",
                            "height": "",
                            "sar": "",
                            "samplerate": "",
                            "channel": ""
                        }
                    }
                ],
                "images": [
                    {
                        "name": "bypass_video",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "width": 0,
                        "height": 0,
                        "framerate": 0.000000
                    }
                ]
            },
            "playlists": [
                {
                    "name": "default",
                    "fileName": "default",
                    "options": {
                        "webRtcAutoAbr": true,
                        "hlsChunklistPathDepth": -1
                    },
                    "rendition/renditions": [
                        {
                            "name/name": "bypass",
                            "video/video": "bypass_video",
                            "audio/audio": "bypass_audio"
                        }
                    ]
                }
            ]
        }
    ]
}

Enable GPU Acceleration

1. Install Drivers

1. Install NVIDIA GPU Driver

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz)
OvenMediaEngine-master/misc/install_nvidia_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver is operating normally with the nvidia-smi command.

$ nvidia-smi

Thu Jun 17 10:20:23 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 20%   35C    P8    N/A /  75W |    156MiB /  1997MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

2 . Prerequisites

OvenMediaEngine-master/misc/prerequisites.sh --enable-nvc

1. Install NVIDIA GPU Driver

Please refer to the NVIDIA Driver installation guide written previously.

2. Install NVIDIA Container Toolkit

OvenMediaEngine-master/misc/install_nvidia_docker_container.sh

3 . Build Image

A Docker Image build script that supports NVIDIA GPU is provided separately. Please refer to the previous guide for how to build

OvenMediaEngine-master/Dockerfile.cuda
OvenMediaEngine-master/Dockerfile.cuda.local

1. Install Intel QuickSync Driver

When the Intel QuickSync driver installation is complete, the OS must be rebooted for normal operation.

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz) 
OvenMediaEngine-master/misc/install_intel_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver operates normally with the Matrix Monitor program.

# Use the samples provided in the Intel Media SDK
# Check the list of codecs supported by iGPU
/MediaSDK-intel-mediasdk-21.1.2/build/__bin/release/simple_7_codec

2. Prerequisites

Using Intel QuickSync GPU

OvenMediaEngine-master/misc/prerequisites.sh --enable-qsv

1. Install XCODER

Please refer to the Netint documentation to install XCODER.

How to check driver installation

After the driver installation is complete, check if the libxcoder exist: the CLI must return something like libxcoder_logan.so (libc6,x86-64) => /usr/local/lib/libxcoder_logan.so

ldconfig -p | grep libxcoder_logan.so

2. Prerequisites

Using Netint VPU

./prerequisites.sh --enable-nilogan --nilogan-path=/root/T4xx/release/FFmpeg-n5.0_t4xx_patch

2. Build & Run

Please refer to the link for how to build and run.

Getting Started

Intructions on running Docker

you must include the --gpus all option when running Docker

docker run -d ... --gpus all airensoft/ovenmediaengine:dev

3. Configuration

<VirtualHosts>
   <VirtualHost>
      <Name>default</Name>
      ...
      <!-- Settings for applications -->
      <Applications>
         <Application>
            <Name>app</Name>
            <Type>live</Type>
            <OutputProfiles>
               <!-- Settings to use hardware codecs -->
               <HardwareAcceleration>true</HardwareAcceleration>
               <OutputProfile>
                  ...
               </OutputProfile>
               <OutputProfile>
                  ...
               </OutputProfile>
            </OutputProfiles>
            <Providers>
               ...
            </Providers>
            <Publishers>
               ...
            </Publishers>
         </Application>
      </Applications>
   </VirtualHost>
</VirtualHosts>

Configuration

Appendix. Support Format

Device

H264

H265

VP8

VP9

D : Decoding, E : Encoding

Reference

Quick Sync Video Format : https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
NVIDIA NVDEC Video Format : https://en.wikipedia.org/wiki/Nvidia_NVDEC
NVIDIA NVENV Video Format : https://en.wikipedia.org/wiki/Nvidia_NVENC
CUDA Toolkit Installation Guide : https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#introduction
NVIDIA Container Toolkit : https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/arch-overview.html#arch-overview
Quick Sync Video format support: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video

TranscodeWebhook

Configuration

<Applications>
	<Application>
		<Name>app</Name>
		<!-- Application type (live/vod) -->
		<Type>live</Type>

		<TranscodeWebhook>
			<Enable>true</Enable>
			<ControlServerUrl>http://example.com/webhook</ControlServerUrl>
			<SecretKey>abc123!@#</SecretKey>
			<Timeout>1500</Timeout>
			<UseLocalProfilesOnConnectionFailure>true</UseLocalProfilesOnConnectionFailure>
                        <UseLocalProfilesOnServerDisallow>false</UseLocalProfilesOnServerDisallow>
                        <UseLocalProfilesOnErrorResponse>false</UseLocalProfilesOnErrorResponse>
		</TranscodeWebhook>

Enable (required) You can enable or disable TranscodeWebhook settings.

ControlServerUrl (required) It's the URL of the Control Server, and it supports both HTTP and HTTPS.

Timeout (optional, default: 1500) This is the Timeout value used when connecting to the Control Server.

UseLocalProfilesOnServerDisallow (optional, default: false) When the Control Server responds with a 200 OK, but "allowed" is set to "false," this policy is followed.

Protocol

Request (OME → Control Server)

OvenMediaEngine sends requests to the Control Server in the following format.

POST /configured/target/url/ HTTP/1.1
Content-Length: 944
Content-Type: application/json
Accept: application/json
X-OME-Signature: f871jd991jj1929jsjd91pqa0amm1
{
	"source" : "TCP://192.168.0.200:8843",
	"stream" : 
	{
    	"name" : "stream",
		"virtualHost" : "default",
		"application" : "app",
		"sourceType" : "Rtmp",
		"createdTime" : "2023-10-24T18:10:51.120+09:00",
		"sourceUrl" : "TCP://192.168.0.200:8843",
		"tracks" : 
		[
			{
				"id" : 0,
				"name" : "Video",
				"type" : "Video",
				"video" : 
				{
					"bitrate" : "2500000",
					"bypass" : false,
					"codec" : "H264",
					"framerate" : 30.0,
					"hasBframes" : false,
					"height" : 720,
					"keyFrameInterval" : 0,
					"width" : 1280
				}
			},
			{
				"audio" : 
				{
					"bitrate" : "160000",
					"bypass" : false,
					"channel" : 2,
					"codec" : "AAC",
					"samplerate" : 44100
				},
				"id" : 1,
				"name" : "Audio",
				"type" : "Audio"
			},
			{
				"id" : 2,
				"name" : "Data",
				"type" : "Data"
			}
		]
	}
}

Response (Control Server → OME)

The Control Server responds in the following format to specify OutputProfiles for the respective stream.

HTTP/1.1 200 OK
Content-Length: 102
Content-Type: application/json
Connection: Closed
{
  "allowed": true,
  "reason": "it will be output to the log file when "allowed" is false",
  "outputProfiles" : {
    "hardwareAcceleration": true,
    "outputProfile":[
      {
        "name": "bypass",
        "outputStreamName": "${OriginStreamName}",
        "playlists": [
          {
            "name": "default",
            "fileName": "default",
            "renditions":[
              {
                "name": "bypass",
                "video": "bypass_video",
                "audio": "bypass_audio"
              }
            ]
          }
        ],
        "encodes": {
          "videos": [
            {
              "name": "bypass_video",
              "bypass": true
            }
          ],
          "audios": [
            {
              "name": "bypass_audio",
              "bypass": true
            }
          ]
        }
      }
    ]
  }
}

The "outputProfiles" in the JSON is identical to the configuration in Server.xml, and the format is as follows.

"outputProfiles": {
    "hardwareAcceleration": true,
    "outputProfile": [
        {
            "name": "bypass",
            "outputStreamName": "${OriginStreamName}",
            "encodes": {
                "audios": [
                    {
                        "name": "bypass_audio",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "bitrate": "",
                        "samplerate": 0,
                        "channel": 0,
                        "bypassIfMatch": {
                            "codec": "",
                            "bitrate": "",
                            "framerate": "",
                            "width": "",
                            "height": "",
                            "sar": "",
                            "samplerate": "",
                            "channel": ""
                        }
                    }
                ],
                "videos": [
                    {
                        "name": "bypass_video",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "profile": "",
                        "bitrate": "",
                        "width": 0,
                        "height": 0,
                        "framerate": 0.000000,
                        "preset": "",
                        "threadCount": -1,
                        "keyFrameInterval": 0,
                        "bFrames": 0,
                        "bypassIfMatch": {
                            "codec": "",
                            "bitrate": "",
                            "framerate": "",
                            "width": "",
                            "height": "",
                            "sar": "",
                            "samplerate": "",
                            "channel": ""
                        }
                    }
                ],
                "images": [
                    {
                        "name": "bypass_video",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "width": 0,
                        "height": 0,
                        "framerate": 0.000000
                    }
                ]
            },
            "playlists": [
                {
                    "name": "default",
                    "fileName": "default",
                    "options": {
                        "webRtcAutoAbr": true,
                        "hlsChunklistPathDepth": -1
                    },
                    "rendition/renditions": [
                        {
                            "name/name": "bypass",
                            "video/video": "bypass_video",
                            "audio/audio": "bypass_audio"
                        }
                    ]
                }
            ]
        }
    ]
}