Quality metrics are the gathering and/or use of values that indicate how accurately a system or service can reproduce media or perform actions within desired levels.

Objective Quality

Objective quality is the determination of accuracy or the ability of a system to provide desired results using evaluation criteria and sources that are repeatable. Objective quality of video signals can be calculated by comparing the pixel locations or signal levels in a source (reference) image to the pixel location or signal levels in a received image. These calculations can be in the form of average or peak error between the images or signals.

Mean Square Error (MSE)

Mean square error is the sum of the average error amount squared adjusted for by offset bias squared. MSE indicates the expected amount of error for a sample set of values.

Peak Signal to Noise Ratio (PSNR)

Peak signal to noise ratio is a measure of the difference between the maximum received signal level and the level of interfering signals (noise). The larger the PSNR level, the better the quality. For video signals, a PSNR that has less than 20 dB (100x) is considered unwatchable.

Subjective Quality

Subjective quality is determination of accuracy or the ability of a system to provide desired results using evaluation sources that can vary (such as the opinions of people) for determining the amount of a quantity or quality of data or media.

Audio Quality

Audio quality is the ability of a speaker or audio transfer system to recreate the key characteristics of an original digital audio signal. Some of the measures of audio quality include fidelity, frequency response, total harmonic distortion, noise level and signal to noise ratio.

The type of audio coder that is used along with its compression parameters influences digital audio quality. Audio compression devices reduce the data transmission rate by approximating the audio signal and this may add distortion.

Packet loss and packet corruption errors will result in the distortion or muting of the audio signal. The compression type influences the amount of distortion that occurs with packet loss or bit errors. Audio coders that have high compression ratios (high efficiency) tend to be more sensitive to packet loss and errors. Even when small amounts of error occur in a speech coder, the result may be very different sounds (a "warble") due to the use of codebooks. Warbles are sounds that are produced during the decoding of a compressed digital audio signal that has been corrupted (has errors) during transmission. The warble sound results from the creation of different sounds than those originally sent. Muting is the process of inhibiting audio (squelching). Muting can be automatically performed when packet loss is detected.

Figure 1.21 shows some of the causes and effects of audio distortion in IP Television systems. This example shows that audio signals are digitized, compressed and error protection coded prior to transmis-

Figure 1.21, IP Audio Distortion

sion. During the transmission process, some packets are lost or corrupted. The loss of packets can result in the temporary muting of the audio signal. Because the data compression process represents sounds by different codes that represent the original audio signal, packet corruption results in the creation of a different altered sound than the sound that was previously transmitted. When there is significant data corruption, unusual artifact sounds ("Warble" sounds) can be created.

Audio Fidelity

Audio fidelity is the degree to which a system or a portion of a system accurately reproduces upon its output, the essential characteristics of the signal impressed upon its input. Audio fidelity can be determined by comparing an original (reference) audio signal with a received audio signal to determine the difference levels. The difference in signal levels represents the distortion that occurs between the source and receiver of the audio signal.

Figure 1.22 shows how to measure audio fidelity. This diagram shows that fidelity testing can identify the distortion that is added at various places in the recording, transmission and recreation of an audio signal. This example explains that the same reference test signal is applied to the input of the system and to a comparator. The comparator removes the original reference signal to show the amount of distortion that is added in the transmission and processing of the signal.

Frequency Response (FR)

Frequency response is a measure of system linearity or performance in reproducing signals across a specified bandwidth. Frequency response is expressed as a frequency range with a specified amplitude tolerance in decibels. Frequency response in digital audio systems is typically limited to one half the sampling frequency (Nyquist limit).

Figure 1.22, Audio Fidelity Testing

Total Harmonic Distortion (THD)

Total harmonic distortion is a ratio of the combined amplitudes of all signals related harmonically to the amplitude of a fundamental signal. THD is typically expressed as a percentage of signal level.

Noise Level

Noise level is a measure of the combined energy of unwanted signals. Noise level is commonly specified as a ratio (in decibels) of noise level on a given circuit as compared to decibels above reference noise level for an electrical system or decibels sound pressure level for an acoustical system.

Signal to Noise Ratio (SNR)

Signal to noise ratio is a comparison of the information-carrying signal power to the noise power in a system. For SNR testing, a connection is setup and a test audio signal is applied to the transmitter. The energy level at the receiving end is measured and recorded. The audio test signal is then removed and the energy level at the receiving end (the noise) is measured and recorded. The difference between these two levels (commonly converted to dB) is the signal to noise ratio.