1 of 3

ABR and Transcoding

OvenMediaEngine has a built-in live transcoder. The live transcoder can decode the incoming live source and re-encode it with the set codec or adjust the quality to encode at multiple bitrates.

Supported Video, Audio and Image Codecs

Decoders

Type

Codec

Video

VP8, H.264, H.265

Audio

AAC, Opus

Encoders

Type

Codec

Codec of Configuration

Video

VP8

vp8

H.264

h264 (Automatic Codec Selection)

Open H264

h264_openh264

Nvidia Hardware

h264_nvenc

Intel Hardware

h264_qsv

Xilinx Hardware

h264_XMA

NetInt Hardware

h264_NILOGAN

H.265

h265 (Automatic Codec Selection)

Nvidia Hardware

h265_nvenc

Intel Hardware

h265_qsv

Xilinx Hardware

h265_xma

NetInt Hardware

h265_nilogan

Audio

AAC

aac

Opus

opus

Image

JPEG

jpeg

PNG

png

OutputProfiles

OutputProfile

The <OutputProfile> setting allows incoming streams to be re-encoded via the <Encodes> setting to create a new output stream. The name of the new output stream is determined by the rules set in <OutputStreamName>, and the newly created stream can be used according to the streaming URL format.

<OutputProfiles>
    <OutputProfile>
        <Name>bypass_stream</Name>
        <OutputStreamName>${OriginStreamName}_bypass</OutputStreamName>
        <Encodes>
            <Video>
                <Bypass>true</Bypass>
            </Video>
            <Audio>
                <Bypass>true</Bypass>
            </Audio>            
            <Audio>
                <Codec>opus</Codec>
                <Bitrate>128000</Bitrate>
                <Samplerate>48000</Samplerate>
                <Channel>2</Channel>
            </Audio>
        </Encodes>
    </OutputProfile>
</OutputProfiles>

According to the above setting, if the incoming stream name is stream, the output stream becomes stream_bypassand the stream URL can be used as follows.

WebRTC ws://192.168.0.1:3333/app/stream_bypass
LLHLS http://192.168.0.1:8080/app/stream_bypass/llhls.m3u8
HLS http://192.168.0.1:8080/app/stream_bypass/ts:playlist.m3u8

Encodes

Video

You can set the video profile as below:

<Encodes>
    <Video>
        <Name>h264_hd</Name>
        <Codec>h264</Codec>
        <Width>1280</Width>
        <Height>720</Height>
        <Bitrate>2000000</Bitrate>
        <Framerate>30.0</Framerate>
        <KeyFrameInterval>30</KeyFrameInterval>
        <BFrames>0</BFrames>
        <Preset>fast</Preset>
        <ThreadCount>4</ThreadCount>
    </Video>
</Encodes>

The meaning of each property is as follows:

Property

Description

Codec*

Specifies the vp8 or h264 codec to use

Bitrate*

Bit per second

Name

Encode name for Renditions

Width

Width of resolution

Height

Height of resolution

Framerate

Frames per second

KeyFrameInterval

Number of frames between two keyframes (0~600) default is framerate (i.e. 1 second)

BFrames

Number of B-frame (0~16) default is 0

Profile

H264 only encoding profile (baseline, main, high)

Preset

Presets of encoding quality and performance

ThreadCount

Number of threads in encoding

* required

Table of presets

A table in which presets provided for each codec library are mapped to OvenMediaEngine presets. Slow presets are of good quality and use a lot of resources, whereas Fast presets have lower quality and better performance. It can be set according to your own system environment and service purpose.

Presets

openh264

h264_nvenc

h264_qsv

vp8

slower

QP( 10-39)

No Support

best

slow

QP (16-45)

No Support

best

medium

QP (24-51)

No Support

good

fast

QP (32-51)

No Support

realtime

faster

QP (40-51)

No Support

realtime

References

https://trac.ffmpeg.org/wiki/Encode/VP8
https://docs.nvidia.com/video-technologies/video-codec-sdk/nvenc-preset-migration-guide/

Audio

You can set the audio profile as below:

<Encodes>
    <Audio>
        <Name>opus_128</Name>
        <Codec>opus</Codec>
        <Bitrate>128000</Bitrate>
        <Samplerate>48000</Samplerate>
        <Channel>2</Channel>
    </Audio>
</Encodes>

The meaning of each property is as follows:

Property

Description

Codec*

Specifies the opus or aac codec to use

Bitrate*

Bits per second

Name

Encode name for Renditions

Samplerate

Samples per second

Channel

The number of audio channels

* required

It is possible to have an audio only output profile by specifying the Audio profile and omitting a Video one.

Image

You can set the Image profile as below:

<Encodes>
    <Image>
        <Codec>jpeg</Codec>
        <Width>1280</Width>
        <Height>720</Height>
        <Framerate>1</Framerate>
    </Image>
</Encodes>

The meaning of each property is as follows:

Property

Description

Codec

Specifies the jpeg or png codec to use

Width

Width of resolution

Height

Height of resolution

Framerate

Frames per second

The image encoding profile is only used by thumbnail publishers. and, bypass option is not supported.

Bypass without transcoding

You can configure Video and Audio to bypass transcoding as follows:

<Video>
    <Bypass>true</Bypass>
</Video>
<Audio>
    <Bypass>true</Bypass>
</Audio>

You need to consider codec compatibility with some browsers. For example, chrome only supports OPUS codec for audio to play WebRTC stream. If you set to bypass incoming audio, it can't play on chrome.

WebRTC doesn't support AAC, so if video bypasses transcoding, audio must be encoded in OPUS.

<Encodes>
    <Video>
        <Bypass>true</Bypass>
    </Video>
    <Audio>
        <Codec>opus</Codec>
        <Bitrate>128000</Bitrate>
        <Samplerate>48000</Samplerate>
        <Channel>2</Channel>
    </Audio>
</Encodes>

Conditional transcoding

If the codec or quality of the input stream is the same as the profile to be encoded into the output stream. there is no need to perform re-transcoding while unnecessarily consuming a lot of system resources. If the quality of the input track matches all the conditions of BypassIfMatch, it will be Pass-through without encoding

Matching elements in video

Elements

Condition

Description

Codec (Optional)

Compare video codecs

Width (Optional)

eq, lte, gte

Compare horizontal pixel of video resolution

Height (Optional)

eq, lte, gte

Compare vertical pixel of video resolution

SAR (Optional)

Compare ratio of video resolution

* eq: equal to / lte: less than or equal to / gte: greater than or equal to

Matching elements in audio

Elements

Condition

Description

Codec (Optional)

Compare audio codecs

Samplerate (Optional)

eq, lte, gte

Compare sampling rate of audio

Channel (Optional)

eq, lte, gte

Compare number of channels in audio

* eq: equal to / lte: less than or equal to / gte: greater than or equal to

To support WebRTC and LLHLS, AAC and Opus codecs must be supported at the same time. Use the settings below to reduce unnecessary audio encoding.

<Encodes>
	<Video>
                <Bypass>true</Bypass>	
	</Video>
	<Audio>
		<Name>cond_audio_aac</Name>
		<Codec>aac</Codec>
		<Bitrate>128000</Bitrate>
		<Samplerate>48000</Samplerate>
		<Channel>2</Channel>
		<BypassIfMatch>
			<Codec>eq</Codec>
			<Samplerate>lte</Samplerate>
			<Channel>eq</Channel>			
		</BypassIfMatch>
	</Audio>		
	<Audio>
		<Name>cond_audio_opus</Name>
		<Codec>opus</Codec>
		<Bitrate>128000</Bitrate>
		<Samplerate>48000</Samplerate>
		<Channel>2</Channel>
		<BypassIfMatch>
			<Codec>eq</Codec>
			<Samplerate>lte</Samplerate>
			<Channel>eq</Channel>	
		</BypassIfMatch>	
	</Audio>
</Encodes>

If a video track with a lower quality than the encoding option is input, unnecessary upscaling can be prevented. SAR (Storage Aspect Ratio) is the ratio of original pixels. In the example below, even if the width and height of the original video are smaller than or equal to the width and height set in the encoding option, if the ratio is different, it means that encoding is performed without bypassing.

<Encodes>
	<Video>                                                                                                    
		<Name>prevent_upscaling_video</Name>
		<Codec>h264</Codec>
		<Bitrate>2048000</Bitrate>
		<Width>1280</Width>
		<Height>720</Height>
		<Framerate>30</Framerate>
		<BypassIfMatch>
			<Codec>eq</Codec>
			<Width>lte</Width>
			<Height>lte</Height>
			<SAR>eq</SAR>
		</BypassIfMatch>
	</Video>
</Encodes>

Keep the original with transcoding

If you want to transcode with the same quality as the original. See the sample below for possible parameters that OME supports to keep original. If you remove the Width, Height, Framerate, Samplerate, and Channel parameters. then, It is transcoded with the same options as the original.

<Encodes>
    <Video>
        <Codec>vp8</Codec>
        <Bitrate>2000000</Bitrate>
    </Video>
     <Audio>
        <Codec>opus</Codec>
        <Bitrate>128000</Bitrate>
    </Audio>  
    <Image>
        <Codec>jpeg</Codec>
    </Image>
</Encodes>

Rescaling while keep the aspect ratio

To change the video resolution when transcoding, use the values of width and height in the Video encode option. If you don't know the resolution of the original, it will be difficult to keep the aspect ratio after transcoding. Please use the following methods to solve these problems. For example, if you input only the Width value in the Video encoding option, the Height value is automatically generated according to the ratio of the original video.

<Encodes>
    <Video>
        <Codec>h264</Codec>
        <Bitrate>2000000</Bitrate>
        <Width>1280</Width>
        <!-- Height is automatically calculated as the original video ratio -->
        <Framerate>30.0</Framerate>
    </Video>
    <Video>
        <Codec>h264</Codec>
        <Bitrate>2000000</Bitrate>
        <!-- Width is automatically calculated as the original video ratio -->
        <Height>720</Height>
        <Framerate>30.0</Framerate>
    </Video>    
</Encodes>

Adaptive Bitrate Streaming (ABR)

From version 0.14.0, OvenMediaEngine can encode same source with multiple bitrates renditions and deliver it to the player.

As shown in the example configuration below, you can provide ABR by adding <Playlists> to <OutputProfile>. There can be multiple playlists, and each playlist can be accessed with <FileName>.

The method to access the playlist set through LLHLS is as follows.

http[s]://<domain>[:port]/<app>/<stream>/<FileName>.m3u8

The method to access the playlist set through HLS is as follows.

http[s]://<domain>[:port]/<app>/<stream>/<FileName>.m3u8?format=ts

The method to access the Playlist set through WebRTC is as follows.

ws[s]://<domain>[:port]/<app>/<stream>/<FileName>

Note that <FileName> must never contain the playlist and chunklist keywords. This is a reserved word used inside the system.

To set up <Rendition>, you need to add <Name> to the elements of <Encodes>. Connect the set <Name> into <Rendition><Video> or <Rendition><Audio>.

In the example below, three quality renditions are provided and the URL to play the abr playlist as LLHLS is https://domain:port/app/stream/abr.m3u8 and The WebRTC playback URL is wss://domain:port/app/stream/abr

TS files used in HLS must have A/V pre-muxed, so the EnableTsPackaging option must be set in the Playlist.

<OutputProfile>
	<Name>bypass_stream</Name>
	<OutputStreamName>${OriginStreamName}</OutputStreamName>
	<!--LLHLS URL : https://domain/app/stream/abr.m3u8 --> 
	<Playlist>
		<Name>For LLHLS</Name>
		<FileName>abr</FileName>
		<Options> <!-- Optinal -->
			<!-- 
			Automatically switch rendition in WebRTC ABR 
			[Default] : true
			-->
			<WebRtcAutoAbr>true</WebRtcAutoAbr> 
			<EnableTsPackaging>true</EnableTsPackaging>
		</Options>
		<Rendition>
			<Name>Bypass</Name>
			<Video>bypass_video</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
		<Rendition>
			<Name>FHD</Name>
			<Video>video_1280</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
		<Rendition>
			<Name>HD</Name>
			<Video>video_720</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
	</Playlist>
	<!--LLHLS URL : https://domain/app/stream/llhls.m3u8 --> 
	<Playlist>
		<Name>Change Default</Name>
		<FileName>llhls</FileName>
		<Rendition>
			<Name>HD</Name>
			<Video>video_720</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
	</Playlist> 
	<Encodes>
		<Audio>
			<Name>bypass_audio</Name>
			<Bypass>true</Bypass>
		</Audio>
		<Video>
			<Name>bypass_video</Name>
			<Bypass>true</Bypass>
		</Video>
		<Audio>
			<Codec>opus</Codec>
			<Bitrate>128000</Bitrate>
			<Samplerate>48000</Samplerate>
			<Channel>2</Channel>
		</Audio>
		<Video>
			<Name>video_1280</Name>
			<Codec>h264</Codec>
			<Bitrate>5024000</Bitrate>
			<Framerate>30</Framerate>
			<Width>1920</Width>
			<Height>1280</Height>
			<Preset>faster</Preset>
		</Video>
		<Video>
			<Name>video_720</Name>
			<Codec>h264</Codec>
			<Bitrate>2024000</Bitrate>
			<Framerate>30</Framerate>
			<Width>1280</Width>
			<Height>720</Height>
			<Preset>faster</Preset>
		</Video>
	</Encodes>
</OutputProfile>

Supported codecs by streaming protocol

Even if you set up multiple codecs, there is a codec that matches each streaming protocol supported by OME, so it can automatically select and stream codecs that match the protocol. However, if you don't set a codec that matches the streaming protocol you want to use, it won't be streamed.

The following is a list of codecs that match each streaming protocol:

Protocol

Supported Codec

WebRTC

VP8, H.264, Opus

LLHLS

H.264, H.265, AAC

Therefore, you set it up as shown in the table. If you want to stream using LLHLS, you need to set up H.264, H.265 and AAC, and if you want to stream using WebRTC, you need to set up Opus.

Also, if you are going to use WebRTC on all platforms, you need to configure both VP8 and H.264. This is because different codecs are supported for each browser, for example, VP8 only, H264 only, or both.

However, don't worry. If you set the codecs correctly, OME automatically sends the stream of codecs requested by the browser.

Enable GPU Acceleration

OvenMediaEngine supports GPU-based hardware decoding and encoding. Currently supported GPU acceleration devices are Intel's QuickSync and NVIDIA. This article explains how to install the drivers for OvenMediaEngine and set up the configuration to use your GPU.

1. Install Drivers

1. Install NVIDIA GPU Driver

If you are using an NVIDIA graphics card, please refer to the following guide to install the driver. The OS that supports installation with the provided script are CentOS 7/8 and Ubuntu 18/20 versions. If you want to install the driver in another OS, please refer to the manual installation guide document.

CentOS environment requires the process of uninstalling the nouveau driver. After uninstalling the driver, the first reboot is required, and a new NVIDIA driver must be installed and rebooted. Therefore, two install scripts must be executed.

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz)
OvenMediaEngine-master/misc/install_nvidia_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver is operating normally with the nvidia-smi command.

$ nvidia-smi

Thu Jun 17 10:20:23 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 20%   35C    P8    N/A /  75W |    156MiB /  1997MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

2 . Prerequisites

If you have finished installing the driver to use the GPU, you need to reinstall the open source library using Prerequisites.sh . The purpose is to allow external libraries to use the installed graphics driver.

OvenMediaEngine-master/misc/prerequisites.sh --enable-nvc

1. Install NVIDIA GPU Driver

Please refer to the NVIDIA Driver installation guide written previously.

2. Install NVIDIA Container Toolkit

To use GPU acceleration in Docker, you need to install NVIDIA drivers on your host OS and install the NVIDIA Container Toolkit. This toolkit includes container runtime libraries and utilities for using NVIDIA GPUs in Docker containers.

OvenMediaEngine-master/misc/install_nvidia_docker_container.sh

3 . Build Image

A Docker Image build script that supports NVIDIA GPU is provided separately. Please refer to the previous guide for how to build

OvenMediaEngine-master/Dockerfile.cuda
OvenMediaEngine-master/Dockerfile.cuda.local

1. Install Intel QuickSync Driver

If you are using an Intel CPU that supports QuickSync, please refer to the following guide to install the driver. The OSes that support installation using the provided scripts are CentOS 7/8 and Ubuntu 18/20 versions. If you want to install the driver on a different OS, please refer to the Manual Installation Guide document.

When the Intel QuickSync driver installation is complete, the OS must be rebooted for normal operation.

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz) 
OvenMediaEngine-master/misc/install_intel_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver operates normally with the Matrix Monitor program.

# Use the samples provided in the Intel Media SDK
# Check the list of codecs supported by iGPU
/MediaSDK-intel-mediasdk-21.1.2/build/__bin/release/simple_7_codec

2. Prerequisites

Using Intel QuickSync GPU

OvenMediaEngine-master/misc/prerequisites.sh --enable-qsv

1. Install XCODER

Please refer to the Netint documentation to install XCODER.

How to check driver installation

After the driver installation is complete, check if the libxcoder exist: the CLI must return something like libxcoder_logan.so (libc6,x86-64) => /usr/local/lib/libxcoder_logan.so

ldconfig -p | grep libxcoder_logan.so

2. Prerequisites

If you have finished installing the driver to use the VPU, you need to reinstall the open source library using Prerequisites.sh . The purpose is to allow external libraries to use the installed graphics driver. You also have to unzip the ffmpeg patch provide by netint in a specfic path

Using Netint VPU

./prerequisites.sh --enable-nilogan --nilogan-path=/root/T4xx/release/FFmpeg-n5.0_t4xx_patch

2. Build & Run

Please refer to the link for how to build and run.

Intructions on running Docker

you must include the --gpus all option when running Docker

docker run -d ... --gpus all airensoft/ovenmediaengine:dev

3. Configuration

To use hardware acceleration, set the HardwareAcceleration option to true under OutputProfiles. If this option is enabled, a hardware codec is automatically used when creating a stream, and if it is unavailable due to insufficient hardware resources, it is replaced with a software codec.

<VirtualHosts>
   <VirtualHost>
      <Name>default</Name>
      ...
      <!-- Settings for applications -->
      <Applications>
         <Application>
            <Name>app</Name>
            <Type>live</Type>
            <OutputProfiles>
               <!-- Settings to use hardware codecs -->
               <HardwareAcceleration>true</HardwareAcceleration>
               <OutputProfile>
                  ...
               </OutputProfile>
               <OutputProfile>
                  ...
               </OutputProfile>
            </OutputProfiles>
            <Providers>
               ...
            </Providers>
            <Publishers>
               ...
            </Publishers>
         </Application>
      </Applications>
   </VirtualHost>
</VirtualHosts>

Appendix. Support Format

The codecs available using hardware accelerators in OvenMediaEngine are as shown in the table below. Different GPUs support different codecs. If the hardware codec is not available, you should check if your GPU device supports the codec.

Device

H264

H265

VP8

VP9

QuickSync

D / E

NVIDIA

D / E

Docker on NVIDIA Container Toolkit

D / E

D : Decoding, E : Encoding

Reference

Quick Sync Video Format : https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
NVIDIA NVDEC Video Format : https://en.wikipedia.org/wiki/Nvidia_NVDEC
NVIDIA NVENV Video Format : https://en.wikipedia.org/wiki/Nvidia_NVENC
CUDA Toolkit Installation Guide : https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#introduction
NVIDIA Container Toolkit : https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/arch-overview.html#arch-overview
Quick Sync Video format support: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video

TranscodeWebhook

TranscodeWebhook allows OvenMediaEngine to use OutputProfiles from the Control Server's response instead of the OutputProfiles in the local configuration (Server.xml). OvenMediaEngine requests OutputProfiles from the Control Server when streams are created, enabling the specification of different profiles for each individual stream.

Configuration

<Applications>
	<Application>
		<Name>app</Name>
		<!-- Application type (live/vod) -->
		<Type>live</Type>

		<TranscodeWebhook>
			<Enable>true</Enable>
			<ControlServerUrl>http://example.com/webhook</ControlServerUrl>
			<SecretKey>abc123!@#</SecretKey>
			<Timeout>1500</Timeout>
			<UseLocalProfilesOnConnectionFailure>true</UseLocalProfilesOnConnectionFailure>
                        <UseLocalProfilesOnServerDisallow>false</UseLocalProfilesOnServerDisallow>
                        <UseLocalProfilesOnErrorResponse>false</UseLocalProfilesOnErrorResponse>
		</TranscodeWebhook>

Enable (required) You can enable or disable TranscodeWebhook settings.

ControlServerUrl (required) It's the URL of the Control Server, and it supports both HTTP and HTTPS.

SecretKey (optional) This is the Secret Key used to pass authentication for the Control Server. To pass security authentication, an HMAC-SHA1 encrypted value of the HTTP Payload is added to the HTTP Header's X-OME-Signature. This Key is used for generating this value.

Timeout (optional, default: 1500) This is the Timeout value used when connecting to the Control Server.

UseLocalProfilesOnConnectionFailure(optional, default: true) This determines whether to use the OutputProfiles from Local settings in case of communication failure with the Control Server. If it's set to "false," a communication failure with the Control Server will result in a failure to create the Output stream.

UseLocalProfilesOnServerDisallow (optional, default: false) When the Control Server responds with a 200 OK, but "allowed" is set to "false," this policy is followed.

UseLocalProfilesOnErrorResponse (optional, default: false) When the Control Server responds with error status codes such as 400 Bad Request, 404 Not Found, 500 Internal Error, OvenMediaEngine follows this policy.

Protocol

Request (OME → Control Server)

OvenMediaEngine sends requests to the Control Server in the following format.

POST /configured/target/url/ HTTP/1.1
Content-Length: 944
Content-Type: application/json
Accept: application/json
X-OME-Signature: f871jd991jj1929jsjd91pqa0amm1
{
	"source" : "TCP://192.168.0.200:8843",
	"stream" : 
	{
    	"name" : "stream",
		"virtualHost" : "default",
		"application" : "app",
		"sourceType" : "Rtmp",
		"createdTime" : "2023-10-24T18:10:51.120+09:00",
		"sourceUrl" : "TCP://192.168.0.200:8843",
		"tracks" : 
		[
			{
				"id" : 0,
				"name" : "Video",
				"type" : "Video",
				"video" : 
				{
					"bitrate" : "2500000",
					"bypass" : false,
					"codec" : "H264",
					"framerate" : 30.0,
					"hasBframes" : false,
					"height" : 720,
					"keyFrameInterval" : 0,
					"width" : 1280
				}
			},
			{
				"audio" : 
				{
					"bitrate" : "160000",
					"bypass" : false,
					"channel" : 2,
					"codec" : "AAC",
					"samplerate" : 44100
				},
				"id" : 1,
				"name" : "Audio",
				"type" : "Audio"
			},
			{
				"id" : 2,
				"name" : "Data",
				"type" : "Data"
			}
		]
	}
}

Response (Control Server → OME)

The Control Server responds in the following format to specify OutputProfiles for the respective stream.

HTTP/1.1 200 OK
Content-Length: 102
Content-Type: application/json
Connection: Closed
{
  "allowed": true,
  "reason": "it will be output to the log file when "allowed" is false",
  "outputProfiles" : {
    "hardwareAcceleration": true,
    "outputProfile":[
      {
        "name": "bypass",
        "outputStreamName": "${OriginStreamName}",
        "playlists": [
          {
            "name": "default",
            "fileName": "default",
            "renditions":[
              {
                "name": "bypass",
                "video": "bypass_video",
                "audio": "bypass_audio"
              }
            ]
          }
        ],
        "encodes": {
          "videos": [
            {
              "name": "bypass_video",
              "bypass": true
            }
          ],
          "audios": [
            {
              "name": "bypass_audio",
              "bypass": true
            }
          ]
        }
      }
    ]
  }
}

The "outputProfiles" in the JSON is identical to the configuration in Server.xml, and the format is as follows.

"outputProfiles": {
    "hardwareAcceleration": true,
    "outputProfile": [
        {
            "name": "bypass",
            "outputStreamName": "${OriginStreamName}",
            "encodes": {
                "audios": [
                    {
                        "name": "bypass_audio",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "bitrate": "",
                        "samplerate": 0,
                        "channel": 0,
                        "bypassIfMatch": {
                            "codec": "",
                            "bitrate": "",
                            "framerate": "",
                            "width": "",
                            "height": "",
                            "sar": "",
                            "samplerate": "",
                            "channel": ""
                        }
                    }
                ],
                "videos": [
                    {
                        "name": "bypass_video",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "profile": "",
                        "bitrate": "",
                        "width": 0,
                        "height": 0,
                        "framerate": 0.000000,
                        "preset": "",
                        "threadCount": -1,
                        "keyFrameInterval": 0,
                        "bFrames": 0,
                        "bypassIfMatch": {
                            "codec": "",
                            "bitrate": "",
                            "framerate": "",
                            "width": "",
                            "height": "",
                            "sar": "",
                            "samplerate": "",
                            "channel": ""
                        }
                    }
                ],
                "images": [
                    {
                        "name": "bypass_video",
                        "bypass": true,
                        "active": true,
                        "codec": "",
                        "width": 0,
                        "height": 0,
                        "framerate": 0.000000
                    }
                ]
            },
            "playlists": [
                {
                    "name": "default",
                    "fileName": "default",
                    "options": {
                        "webRtcAutoAbr": true,
                        "hlsChunklistPathDepth": -1
                    },
                    "rendition/renditions": [
                        {
                            "name/name": "bypass",
                            "video/video": "bypass_video",
                            "audio/audio": "bypass_audio"
                        }
                    ]
                }
            ]
        }
    ]
}

Enable GPU Acceleration

1. Install Drivers

1. Install NVIDIA GPU Driver

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz)
OvenMediaEngine-master/misc/install_nvidia_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver is operating normally with the nvidia-smi command.

$ nvidia-smi

Thu Jun 17 10:20:23 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:01:00.0 Off |                  N/A |
| 20%   35C    P8    N/A /  75W |    156MiB /  1997MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

2 . Prerequisites

OvenMediaEngine-master/misc/prerequisites.sh --enable-nvc

1. Install NVIDIA GPU Driver

Please refer to the NVIDIA Driver installation guide written previously.

2. Install NVIDIA Container Toolkit

OvenMediaEngine-master/misc/install_nvidia_docker_container.sh

3 . Build Image

A Docker Image build script that supports NVIDIA GPU is provided separately. Please refer to the previous guide for how to build

OvenMediaEngine-master/Dockerfile.cuda
OvenMediaEngine-master/Dockerfile.cuda.local

1. Install Intel QuickSync Driver

When the Intel QuickSync driver installation is complete, the OS must be rebooted for normal operation.

(curl -LOJ https://github.com/AirenSoft/OvenMediaEngine/archive/master.tar.gz && tar xvfz OvenMediaEngine-master.tar.gz) 
OvenMediaEngine-master/misc/install_intel_driver.sh

How to check driver installation

After the driver installation is complete, check whether the driver operates normally with the Matrix Monitor program.

# Use the samples provided in the Intel Media SDK
# Check the list of codecs supported by iGPU
/MediaSDK-intel-mediasdk-21.1.2/build/__bin/release/simple_7_codec

2. Prerequisites

Using Intel QuickSync GPU

OvenMediaEngine-master/misc/prerequisites.sh --enable-qsv

1. Install XCODER

Please refer to the Netint documentation to install XCODER.

How to check driver installation

After the driver installation is complete, check if the libxcoder exist: the CLI must return something like libxcoder_logan.so (libc6,x86-64) => /usr/local/lib/libxcoder_logan.so

ldconfig -p | grep libxcoder_logan.so

2. Prerequisites

Using Netint VPU

./prerequisites.sh --enable-nilogan --nilogan-path=/root/T4xx/release/FFmpeg-n5.0_t4xx_patch

2. Build & Run

Please refer to the link for how to build and run.

Getting Started

Intructions on running Docker

you must include the --gpus all option when running Docker

docker run -d ... --gpus all airensoft/ovenmediaengine:dev

3. Configuration

<VirtualHosts>
   <VirtualHost>
      <Name>default</Name>
      ...
      <!-- Settings for applications -->
      <Applications>
         <Application>
            <Name>app</Name>
            <Type>live</Type>
            <OutputProfiles>
               <!-- Settings to use hardware codecs -->
               <HardwareAcceleration>true</HardwareAcceleration>
               <OutputProfile>
                  ...
               </OutputProfile>
               <OutputProfile>
                  ...
               </OutputProfile>
            </OutputProfiles>
            <Providers>
               ...
            </Providers>
            <Publishers>
               ...
            </Publishers>
         </Application>
      </Applications>
   </VirtualHost>
</VirtualHosts>

Configuration

Appendix. Support Format

Device

H264

H265

VP8

VP9

QuickSync

D / E

NVIDIA

D / E

Docker on NVIDIA Container Toolkit

D / E

D : Decoding, E : Encoding

Reference

Quick Sync Video Format : https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video
NVIDIA NVDEC Video Format : https://en.wikipedia.org/wiki/Nvidia_NVDEC
NVIDIA NVENV Video Format : https://en.wikipedia.org/wiki/Nvidia_NVENC
CUDA Toolkit Installation Guide : https://docs.nvidia.com/cuda/cuda-installation-guide-linux/index.html#introduction
NVIDIA Container Toolkit : https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/arch-overview.html#arch-overview
Quick Sync Video format support: https://en.wikipedia.org/wiki/Intel_Quick_Sync_Video

ABR and Transcoding

OvenMediaEngine has a built-in live transcoder. The live transcoder can decode the incoming live source and re-encode it with the set codec or adjust the quality to encode at multiple bitrates.

Supported Video, Audio and Image Codecs

Decoders

Type

Codec

Video

VP8, H.264, H.265

Audio

AAC, Opus

Encoders

Type

Codec

Codec of Configuration

Video

VP8

vp8

H.264

h264 (Automatic Codec Selection)

Open H264

h264_openh264

Nvidia Hardware

h264_nvenc

Intel Hardware

h264_qsv

Xilinx Hardware

h264_XMA

NetInt Hardware

h264_NILOGAN

H.265

h265 (Automatic Codec Selection)

Nvidia Hardware

h265_nvenc

Intel Hardware

h265_qsv

Xilinx Hardware

h265_xma

NetInt Hardware

h265_nilogan

Audio

AAC

aac

Opus

opus

Image

JPEG

jpeg

PNG

png

OutputProfiles

OutputProfile

<OutputProfiles>
    <OutputProfile>
        <Name>bypass_stream</Name>
        <OutputStreamName>${OriginStreamName}_bypass</OutputStreamName>
        <Encodes>
            <Video>
                <Bypass>true</Bypass>
            </Video>
            <Audio>
                <Bypass>true</Bypass>
            </Audio>            
            <Audio>
                <Codec>opus</Codec>
                <Bitrate>128000</Bitrate>
                <Samplerate>48000</Samplerate>
                <Channel>2</Channel>
            </Audio>
        </Encodes>
    </OutputProfile>
</OutputProfiles>

According to the above setting, if the incoming stream name is stream, the output stream becomes stream_bypassand the stream URL can be used as follows.

WebRTC ws://192.168.0.1:3333/app/stream_bypass
LLHLS http://192.168.0.1:8080/app/stream_bypass/llhls.m3u8
HLS http://192.168.0.1:8080/app/stream_bypass/ts:playlist.m3u8

Encodes

Video

You can set the video profile as below:

<Encodes>
    <Video>
        <Name>h264_hd</Name>
        <Codec>h264</Codec>
        <Width>1280</Width>
        <Height>720</Height>
        <Bitrate>2000000</Bitrate>
        <Framerate>30.0</Framerate>
        <KeyFrameInterval>30</KeyFrameInterval>
        <BFrames>0</BFrames>
        <Preset>fast</Preset>
        <ThreadCount>4</ThreadCount>
    </Video>
</Encodes>

The meaning of each property is as follows:

Property

Description

Codec*

Specifies the vp8 or h264 codec to use

Bitrate*

Bit per second

Name

Encode name for Renditions

Width

Width of resolution

Height

Height of resolution

Framerate

Frames per second

KeyFrameInterval

Number of frames between two keyframes (0~600) default is framerate (i.e. 1 second)

BFrames

Number of B-frame (0~16) default is 0

Profile

H264 only encoding profile (baseline, main, high)

Preset

Presets of encoding quality and performance

ThreadCount

Number of threads in encoding

* required

Table of presets

Presets

openh264

h264_nvenc

h264_qsv

vp8

slower

QP( 10-39)

No Support

best

slow

QP (16-45)

No Support

best

medium

QP (24-51)

No Support

good

fast

QP (32-51)

No Support

realtime

faster

QP (40-51)

No Support

realtime

References

https://trac.ffmpeg.org/wiki/Encode/VP8
https://docs.nvidia.com/video-technologies/video-codec-sdk/nvenc-preset-migration-guide/

Audio

You can set the audio profile as below:

<Encodes>
    <Audio>
        <Name>opus_128</Name>
        <Codec>opus</Codec>
        <Bitrate>128000</Bitrate>
        <Samplerate>48000</Samplerate>
        <Channel>2</Channel>
    </Audio>
</Encodes>

The meaning of each property is as follows:

Property

Description

Codec*

Specifies the opus or aac codec to use

Bitrate*

Bits per second

Name

Encode name for Renditions

Samplerate

Samples per second

Channel

The number of audio channels

* required

It is possible to have an audio only output profile by specifying the Audio profile and omitting a Video one.

Image

You can set the Image profile as below:

<Encodes>
    <Image>
        <Codec>jpeg</Codec>
        <Width>1280</Width>
        <Height>720</Height>
        <Framerate>1</Framerate>
    </Image>
</Encodes>

The meaning of each property is as follows:

Property

Description

Codec

Specifies the jpeg or png codec to use

Width

Width of resolution

Height

Height of resolution

Framerate

Frames per second

The image encoding profile is only used by thumbnail publishers. and, bypass option is not supported.

Bypass without transcoding

You can configure Video and Audio to bypass transcoding as follows:

<Video>
    <Bypass>true</Bypass>
</Video>
<Audio>
    <Bypass>true</Bypass>
</Audio>

WebRTC doesn't support AAC, so if video bypasses transcoding, audio must be encoded in OPUS.

<Encodes>
    <Video>
        <Bypass>true</Bypass>
    </Video>
    <Audio>
        <Codec>opus</Codec>
        <Bitrate>128000</Bitrate>
        <Samplerate>48000</Samplerate>
        <Channel>2</Channel>
    </Audio>
</Encodes>

Conditional transcoding

Matching elements in video

Elements

Condition

Description

Codec (Optional)

Compare video codecs

Width (Optional)

eq, lte, gte

Compare horizontal pixel of video resolution

Height (Optional)

eq, lte, gte

Compare vertical pixel of video resolution

SAR (Optional)

Compare ratio of video resolution

* eq: equal to / lte: less than or equal to / gte: greater than or equal to

Matching elements in audio

Elements

Condition

Description

Codec (Optional)

Compare audio codecs

Samplerate (Optional)

eq, lte, gte

Compare sampling rate of audio

Channel (Optional)

eq, lte, gte

Compare number of channels in audio

* eq: equal to / lte: less than or equal to / gte: greater than or equal to

To support WebRTC and LLHLS, AAC and Opus codecs must be supported at the same time. Use the settings below to reduce unnecessary audio encoding.

<Encodes>
	<Video>
                <Bypass>true</Bypass>	
	</Video>
	<Audio>
		<Name>cond_audio_aac</Name>
		<Codec>aac</Codec>
		<Bitrate>128000</Bitrate>
		<Samplerate>48000</Samplerate>
		<Channel>2</Channel>
		<BypassIfMatch>
			<Codec>eq</Codec>
			<Samplerate>lte</Samplerate>
			<Channel>eq</Channel>			
		</BypassIfMatch>
	</Audio>		
	<Audio>
		<Name>cond_audio_opus</Name>
		<Codec>opus</Codec>
		<Bitrate>128000</Bitrate>
		<Samplerate>48000</Samplerate>
		<Channel>2</Channel>
		<BypassIfMatch>
			<Codec>eq</Codec>
			<Samplerate>lte</Samplerate>
			<Channel>eq</Channel>	
		</BypassIfMatch>	
	</Audio>
</Encodes>

<Encodes>
	<Video>                                                                                                    
		<Name>prevent_upscaling_video</Name>
		<Codec>h264</Codec>
		<Bitrate>2048000</Bitrate>
		<Width>1280</Width>
		<Height>720</Height>
		<Framerate>30</Framerate>
		<BypassIfMatch>
			<Codec>eq</Codec>
			<Width>lte</Width>
			<Height>lte</Height>
			<SAR>eq</SAR>
		</BypassIfMatch>
	</Video>
</Encodes>

Keep the original with transcoding

<Encodes>
    <Video>
        <Codec>vp8</Codec>
        <Bitrate>2000000</Bitrate>
    </Video>
     <Audio>
        <Codec>opus</Codec>
        <Bitrate>128000</Bitrate>
    </Audio>  
    <Image>
        <Codec>jpeg</Codec>
    </Image>
</Encodes>

Rescaling while keep the aspect ratio

<Encodes>
    <Video>
        <Codec>h264</Codec>
        <Bitrate>2000000</Bitrate>
        <Width>1280</Width>
        <!-- Height is automatically calculated as the original video ratio -->
        <Framerate>30.0</Framerate>
    </Video>
    <Video>
        <Codec>h264</Codec>
        <Bitrate>2000000</Bitrate>
        <!-- Width is automatically calculated as the original video ratio -->
        <Height>720</Height>
        <Framerate>30.0</Framerate>
    </Video>    
</Encodes>

Adaptive Bitrate Streaming (ABR)

From version 0.14.0, OvenMediaEngine can encode same source with multiple bitrates renditions and deliver it to the player.

As shown in the example configuration below, you can provide ABR by adding <Playlists> to <OutputProfile>. There can be multiple playlists, and each playlist can be accessed with <FileName>.

The method to access the playlist set through LLHLS is as follows.

http[s]://<domain>[:port]/<app>/<stream>/<FileName>.m3u8

The method to access the playlist set through HLS is as follows.

http[s]://<domain>[:port]/<app>/<stream>/<FileName>.m3u8?format=ts

The method to access the Playlist set through WebRTC is as follows.

ws[s]://<domain>[:port]/<app>/<stream>/<FileName>

Note that <FileName> must never contain the playlist and chunklist keywords. This is a reserved word used inside the system.

To set up <Rendition>, you need to add <Name> to the elements of <Encodes>. Connect the set <Name> into <Rendition><Video> or <Rendition><Audio>.

TS files used in HLS must have A/V pre-muxed, so the EnableTsPackaging option must be set in the Playlist.

<OutputProfile>
	<Name>bypass_stream</Name>
	<OutputStreamName>${OriginStreamName}</OutputStreamName>
	<!--LLHLS URL : https://domain/app/stream/abr.m3u8 --> 
	<Playlist>
		<Name>For LLHLS</Name>
		<FileName>abr</FileName>
		<Options> <!-- Optinal -->
			<!-- 
			Automatically switch rendition in WebRTC ABR 
			[Default] : true
			-->
			<WebRtcAutoAbr>true</WebRtcAutoAbr> 
			<EnableTsPackaging>true</EnableTsPackaging>
		</Options>
		<Rendition>
			<Name>Bypass</Name>
			<Video>bypass_video</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
		<Rendition>
			<Name>FHD</Name>
			<Video>video_1280</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
		<Rendition>
			<Name>HD</Name>
			<Video>video_720</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
	</Playlist>
	<!--LLHLS URL : https://domain/app/stream/llhls.m3u8 --> 
	<Playlist>
		<Name>Change Default</Name>
		<FileName>llhls</FileName>
		<Rendition>
			<Name>HD</Name>
			<Video>video_720</Video>
			<Audio>bypass_audio</Audio>
		</Rendition>
	</Playlist> 
	<Encodes>
		<Audio>
			<Name>bypass_audio</Name>
			<Bypass>true</Bypass>
		</Audio>
		<Video>
			<Name>bypass_video</Name>
			<Bypass>true</Bypass>
		</Video>
		<Audio>
			<Codec>opus</Codec>
			<Bitrate>128000</Bitrate>
			<Samplerate>48000</Samplerate>
			<Channel>2</Channel>
		</Audio>
		<Video>
			<Name>video_1280</Name>
			<Codec>h264</Codec>
			<Bitrate>5024000</Bitrate>
			<Framerate>30</Framerate>
			<Width>1920</Width>
			<Height>1280</Height>
			<Preset>faster</Preset>
		</Video>
		<Video>
			<Name>video_720</Name>
			<Codec>h264</Codec>
			<Bitrate>2024000</Bitrate>
			<Framerate>30</Framerate>
			<Width>1280</Width>
			<Height>720</Height>
			<Preset>faster</Preset>
		</Video>
	</Encodes>
</OutputProfile>

Supported codecs by streaming protocol

The following is a list of codecs that match each streaming protocol:

Protocol

Supported Codec

WebRTC

VP8, H.264, Opus

LLHLS

H.264, H.265, AAC

Therefore, you set it up as shown in the table. If you want to stream using LLHLS, you need to set up H.264, H.265 and AAC, and if you want to stream using WebRTC, you need to set up Opus.

However, don't worry. If you set the codecs correctly, OME automatically sends the stream of codecs requested by the browser.