Monday, September 3, 2018

Encode High Quality Audio with FFmpeg

Why we need to transcode audio?
If you want to implement streaming audio, to make it easier to stream the audio then split the file into small files. To prevent and avoid transcoding from a lossy format to the same or another lossy format when possible. Transcode to lossy from the lossless source, or just copy the lossy source audio track instead of transcoding.

Generation loss
Transcoding from a lossy format like MP3, AAC, Vorbis, Opus, WMA, etc. to the same or different lossy format might degrade the audio quality even if the bitrate stays the same (or higher). This quality degradation might not be audible to you but it might be audible to others.

Copying audio tracks
If the target container format supports the audio codec of the source file then consider just muxing it into the output file without re-encoding. Muxing is the process for combining two or more signals into one for example combining a video track, one or more audio tracks. MKV supports virtually any audio codec. This can be achieved by specifying 'copy' as the audio codec.

Example:
Transcoding a WebM file (with VP8 video/Vorbis audio) to a MKV file (with H.264 video/unaltered Vorbis audio):

           ffmpeg -i someFile.avi -c:a copy -c:v libx264 outFile.mkv

In some cases this might not be possible, because the target device/player doesn't support the codec or the target container format doesn't support the codec. Another reason to transcode might be that the source audio track is too big (it has a higher bitrate than what you want to use in the output file).


Audio encoders FFmpeg can use
 FFmpeg can encode to a wide variety of lossy audio formats.

Here are some popular lossy formats with encoders listed that FFmpeg can use:

Dolby Digital: ac3
Dolby Digital Plus: eac3
MP2: libtwolame, mp2
Windows Media Audio 1: wmav1
Windows Media Audio 2: wmav2
AAC LC: libfdk_aac, aac
HE-AAC: libfdk_aac
Vorbis: libvorbis, vorbis
MP3: libmp3lame, libshine
Opus: libopus

Based on quality produced from high to low:
libopus > libvorbis >= libfdk_aac > aac > libmp3lame >= eac3/ac3 > libtwolame > vorbis > mp2 > wmav2/wmav1

The >= sign means greater or the same quality.
This list is just a general guide and there may be cases where a codec listed to the right will perform better than one listed to the left at certain bitrates.
The highest quality internal/native encoder available in FFmpeg without any external libraries is aac.

Please note it is not recommended to use the experimental vorbis for Vorbis encoding; use libvorbis instead.
Please note that wmav1 and wmav2 don't seem to be able to reach transparency at any given bitrate.

Recommended minimum bitrates to use

The bitrates listed here assume 2-channel stereo and a sample rate of 44.1kHz or 48kHz. Mono, speech, and quiet audio may require fewer bits.

    libopus – usable range ≥ 32Kbps. Recommended range ≥ 64Kbps
    libfdk_aac default AAC LC profile – recommended range ≥ 128Kbps; see AAC Encoding Guide.
    libfdk_aac -profile:a aac_he_v2 – usable range ≤ 48Kbps CBR. Transparency: Does not reach transparency. Use AAC LC instead to achieve transparency
    libfdk_aac -profile:a aac_he – usable range ≥ 48Kbps and ≤ 80Kbps CBR. Transparency: Does not reach transparency. Use AAC LC instead to achieve transparency
    libvorbis – usable range ≥ 96Kbps. Recommended range -aq 4 (≥ 128Kbps)
    libmp3lame – usable range ≥ 128Kbps. Recommended range -aq 2 (≥ 192Kbps)
    ac3 or eac3 – usable range ≥ 160Kbps. Recommended range ≥ 160Kbps

    Example of usage:

    ffmpeg -i input.wav -c:a libfaac -q:a 330 -cutoff 15000 output.m4a

    aac – usable range ≥ 32Kbps (depending on profile and audio). Recommended range ≥ 128Kbps
    Example of usage:

    ffmpeg -i input.wav output.m4a

    libtwolame – usable range ≥ 192Kbps. Recommended range ≥ 256Kbps
    mp2 – usable range ≥ 320Kbps. Recommended range ≥ 320Kbps

The vorbis and wmav1/wmav2 encoders are not worth using.
The wmav1/wmav2 encoder does not reach transparency at any bitrate.
The vorbis encoder does not use the bitrate specified in FFmpeg. On some samples it does sound reasonable, but the bitrate is very high.

To calculate the bitrate to use for multi-channel audio: (bitrate for stereo) x (channels / 2).
Example for 5.1(6 channels) Vorbis audio: 128Kbps x (6 / 2) = 384Kbps

When compatibility with hardware players doesn't matter then use libvorbis in a MKV container when libfdk_aac isn't available.
-Note - libopus will likely give higher quality
When compatibility with hardware players does matter then use libmp3lame or ac3 in a MP4/MKV container when libfdk_aac isn't available.
Transparency means the encoded audio sounds indistinguishable from the audio in the source file.
Some codecs have a more efficient variable bitrate (VBR) mode which optimizes to a given, constant quality level rather than having variable quality at a given, constant bitrate (CBR). The info above is for CBR. VBR is more efficient than CBR but may not be as hardware-compatible.


             

Other Topics:
               
               

Thursday, March 8, 2018

NGINX and PHP-FPM

Scenario

Server with the following technical specs:
  • CPU: 4 @ 2.7GHZ
  • RAM: 8GB

Checking Nginx Parameter

Key areas to inspect when looking at Nginx for a PHP-FPM environment consist of worker_connections and worker_processes

Worker Connections

Sets the maximum number of simultaneous connections that can be opened by a worker process. This is often set to the number of CPU cores.

Worker Processes

Sets the maximum number of simultaneous connections that can be opened by a worker process.

PHP-FPM

MaxChildren

To optimize PHP-FPM it is largely depends on the application itself. However a standard rule that would satisfy a broad subset of use cases would be:

pm.max_children = (Total RAM - Memory used for Linux, DB, etc.) / Average php process size

First, we need to install ps_mem to check the memory usage.

CentOS 
yum install ps_mem -y

Ubuntu
wget http://www.pixelbeat.org/scripts/ps_mem.py mv ps_mem.py /usr/local/sbin/ chmod 755 /usr/local/sbin/ps_mem.py
then run command
ps_mem.py


Here is the result:


the output should give you something like this (refer to the picture shown as above):
 776.6 MiB + 49.4 MiB =   826.1 MiB    php-fpm (105)

This means php-fpm is using in total 826.1 MiB with 105 Processes. After converting the value in Megabyte, we will divide it by the number of processes. This will tell us: How much each child process (105) is consuming.

826.1 MiB/105 Processes =  7.86 MiB

7.86MB memory / Processes

We can now calculate the number of process php-fpm can calculate via this simple formula:

max_children = (Total Number of Memory - 1000MB) / FPM Memory per Process

We reserved 1000MB for other process like mysql, nginx, etc.

max_children = (4000 MB - 1000 MB) / 7.86

max_children = 381.67


Quick Setup

At a most base level this seems to be the ideal setup based on peer review:
  • nginx (<=1.9.0)
  • zendopcache > apc (deprecated)
  • php-fpm (fastcgi)
Note: It is also important to highlight that single core php-fpm instances will not gain much of a performance improvement. PHP-FPM benefits proportionately to the number of cpu cores available. While a dual core would yield a performance gain. A recommended number would be at least 4 cores.


Detailed Setup

In the /etc/php5/fpm/pool.d/www.conf file:

[www]
user = www-data
group = www-data
chdir = /
listen = 127.0.0.1:9000
pm = dynamic            (ondemand might be better for memory usage)
pm.max_children         (total RAM - (DB etc) / process size)
pm.start_servers        (1/4 of cpu cores)
pm.min_spare_servers    (1/2 of cpu cores)
pm.max_spare_servers    (total of cpur cores)
pm.max_requests         (high to prevent server respwan)
                        (tip: set really low to find memory leaks)
In the /etc/php5/fpm/conf.d/local.ini file:

memory_limit = 324M (WxT needs at minimum 256MB)
 
According to our scenario sample then it will give result as below
pm = ondemand            
pm.max_children         (382)
pm.start_servers        (1)
pm.min_spare_servers    (2)
pm.max_spare_servers    (4)
pm.max_requests         (1000)
                       
  
Then just execute the following commands:
nginx -s reload




Other Topics: