Multiple sub-Nyquist sampling encoding

MUSE (Multiple sub-Nyquist Sampling Encoding), commercially known as Hi-Vision (a contraction of HIgh-definition teleVISION) was a Japanese analog high-definition television system, with design efforts going back to 1979.

It used dot-interlacing and digital video compression to deliver 1125 line, 60 field-per-second (1125i60) signals to the home. The system was standardized as ITU-R recommendation BO.786 and specified by SMPTE 260M, using a colorimetry matrix specified by SMPTE 240M. As with other analog systems, not all lines carry visible information. On MUSE there are 1035 active interlaced lines, therefore this system is sometimes also mentioned as 1035i. MUSE employed 2-dimensional filtering, dot-interlacing, motion-vector compensation and line-sequential color encoding with time compression to "fold" or compress an original 30 MHz bandwidth Hi-Vision source signal into just 8.1 MHz.

Japan began broadcasting wideband analog HDTV signals in December 1988, initially with an aspect ratio of 2:1. The Sony HDVS high-definition video system was used to create content for the MUSE system, but didn't record MUSE signals. It recorded Hi-Vision signals which are uncompressed. By the time of its commercial launch in 1991, digital HDTV was already under development in the United States. Hi-Vision MUSE was mainly broadcast by NHK through their BShi satellite TV channel, although other channels such as WOWOW, TV Asahi, Fuji Television, TBS Television, Nippon Television, and TV Tokyo also broadcast in MUSE.

On May 20, 1994, Panasonic released the first MUSE LaserDisc player. There were also a number of players available from other brands like Pioneer and Sony.

Hi-Vision continued broadcasting in analog by NHK until 2007. Other channels had stopped soon after December 1, 2000 as they transitioned to digital HD signals in ISDB, Japan's digital broadcast standard.

History

MUSE was developed by NHK Science & Technology Research Laboratories in the 1980s as a compression system for Hi-Vision HDTV signals.

• Japanese broadcast engineers immediately rejected conventional vestigial sideband broadcasting.
• It was decided early on that MUSE would be a satellite broadcast format as Japan economically supports satellite broadcasting. MUSE was transmitted at a frequency of 21 GHz or 12 GHz.

Modulation research

• Japanese broadcast engineers had been studying the various HDTV broadcast types for some time. It was initially thought that SHF, EHF or optic fiber would have to be used to transmit HDTV due to the high bandwidth of the signal, and HLO-PAL would be used for terrestrial broadcast. HLO-PAL is a conventionally constructed composite signal (based on $Y$ for luminance and $C$ for chroma like NTSC and PAL) and uses a phase alternating by line with half-line offset carrier encoding of the wideband/narrowband chroma components. Only the very lowest part of the wideband chroma component overlapped the high-frequency chroma. The narrowband chroma was completely separated from luminance. PAF, or phase alternating by field (like the first NTSC color system trial) was also experimented with, and it gave much better decoding results, but NHK abandoned all composite encoding systems. Because of the use of satellite transmission, Frequency modulation (FM) should be used with power-limitation problem. FM incurs triangular noise, so if a sub-carrierred composite signal is used with FM, demodulated chroma signal has more noise than luminance. Because of this, they looked at other options, and decided to use $Y/C$ component emission for satellite. At one point, it seemed that FCFE (Frame Conversion Fineness Enhanced), I/P conversion compression system, would be chosen, but MUSE was ultimately picked.
• Separate transmission of $Y$ and $C$ components was explored. The MUSE format which is transmitted today, uses separated component signalling. The improvement in picture quality was so great, that the original test systems were recalled.
• One more power saving tweak was made: lack of visual response to low frequency noise allows significant reduction in transponder power if the higher video frequencies are emphasised prior to modulation at the transmitter and de-emphasized at the receiver.

Technical specifications

MUSE's "1125 lines" are an analog measurement, which includes non-video scan lines taking place while a CRT's electron beam returns to the top of the screen to begin scanning the next field. Only 1035 lines have picture information. Digital signals count only the lines (rows of pixels) that have actual detail, so NTSC's 525 lines become 486i (rounded to 480 to be MPEG compatible), PAL's 625 lines become 576i, and MUSE would be 1035i. To convert the bandwidth of Hi-Vision MUSE into "conventional" lines-of-horizontal resolution (as is used in the NTSC world), multiply 29.9 lines per MHz of bandwidth. (NTSC and PAL/SECAM are 79.9 lines per MHz) - this calculation of 29.9 lines works for all current HD systems including Blu-ray and HD-DVD. So, for MUSE, during a still picture, the lines of resolution would be: 598-lines of luminance resolution per-picture-height. The chroma resolution is: 209-lines. The horizontal luminance measurement approximately matches the vertical resolution of a 1080 interlaced image when the Kell factor and interlace factor are taken into account. 1125 lines was selected as a compromise between the resolution in lines of NTSC and PAL and then doubling this number.

MUSE employs time-compression integration (TCI) which is another term for time-division multiplexing, which is used to carry luminance, chrominance, PCM audio and sync signals on one carrier signal/in one carrier frequency. However, TCI achieves multiplexing by compression of the contents in the time dimension, in other words transmitting frames of video that are divided into regions with chrominance compressed into the left of the frame and luminance compressed into the right of the frame, which must then be expanded and layered to create a visible image. This makes it different from NTSC which carries luminance, audio and chrominance simultaneously in several carrier frequencies. Hi-Vision signals are analog component video signals with 3 channels which were RGB initially, and later YP_bP_r. The Hi-Vision standard aims to work with both RGB and YP_bP_r signals.

Key features of the MUSE system:

• Scanlines (total/active): 1,125/1,035
• Pixels per line (fully interpolated): 1122 (still image)/748 (moving)
• Reference clock periods: 1920 per active line
• Interlaced ratio: 2:1
• Aspect ratio 16:9
• Refresh rate: 59.94 or 60 frames per second
• Sampling frequency for broadcast: 16.2 MHz
• Vector motion compensation: horizontal ± 16 samples (32.4 MHz clock) / frame, a vertical line ± 3 / Field
• Audio: "DANCE" discrete 2- or 4-channel digital audio system: 48 kHz/16 bit (2 channel stereo: 2 front channels)/32 kHz/12 bit (4 channel surround: 3 front channels + 1 back channel)
• DPCM Audio compression format: DPCM quasi-instantaneous companding
• Required bandwidth: 27 MHz Usable bandwidth is 1/3 of this, 9 Mhz due to the use of FM modulation for transmission.

Colorimetry

The MUSE luminance signal $Y$ encodes $YM$, specified as the following mix of the original RGB color channels:

${\begin{aligned}YM=0.294\,R+0.588\,G+0.118\,B\end{aligned}}$

The chrominance $C$ signal encodes $B-YM$ and $R-YM$ difference signals. By using these three signals ($YM$, $B-YM$ and $R-YM$), a MUSE receiver can retrieve the original RGB color components using the following matrix:

${\begin{aligned}{\begin{bmatrix}G\\B\\R\end{bmatrix}}\ =\ {\begin{bmatrix}1&-1/5&-1/2\\1&1&0\\1&0&1\end{bmatrix}}{\begin{bmatrix}1&0&0\\0&5/4&0\\0&0&1\end{bmatrix}}{\begin{bmatrix}YM\\B-YM\\R-YM\end{bmatrix}}\ =\ {\begin{bmatrix}1&-1/4&-1/2\\1&5/4&0\\1&0&1\end{bmatrix}}{\begin{bmatrix}YM\\B-YM\\R-YM\end{bmatrix}}\end{aligned}}$

The system used a colorimetry matrix specified by SMPTE 240M (with coefficients corresponding to the SMPTE RP 145 primaries, also known as SMPTE-C, in use at the time the standard was created). The chromaticity of the primary colors and white point are:

MUSE colorimetry (SMPTE 240M / SMPTE "C")

Primaries	CIE 1931 coordinates
	x	y
Red	0.630	0.340
Green	0.310	0.595
Blue	0.155	0.070
White Point D65	0.3127	0.3290

The luma ($EY$) function is specified as:

${\begin{aligned}EY=0.212\,ER+0.701\,EG+0.087\,EB\end{aligned}}$

The blue color difference ($EPB$) is amplitude-scaled ($EB-EY$), according to:

${\begin{aligned}EPB=1.826\,(EB-EY)\end{aligned}}$

The red color difference ($EPR$) is amplitude-scaled ($ER-EY$), according to:

${\begin{aligned}EPR=1.576\,(ER-EY)\end{aligned}}$

Signal and Transmission

MUSE is a 1125 line system (1035 visible), and is not pulse and sync compatible with the digital 1080 line system used by modern HDTV. Originally, it was a 1125 line, interlaced, 60 Hz, system with a 5:3 (1.66:1) aspect ratio and an optimal viewing distance of roughly 3.3H. In 1989 this was changed to a 16:9 aspect ratio.

For terrestrial MUSE transmission a bandwidth limited FM system was devised. A satellite transmission system uses uncompressed FM.

Before MUSE compression, the Hi- Vision signal bandwidth is reduced from 30 MHz for luminance and chrominance to a pre-compression bandwidth $Y$ of 20 MHz for luminance, and a pre-compression bandwidth for chrominance is a 7.425 MHz carrier.

The Japanese initially explored the idea of frequency modulation of a conventionally constructed composite signal. This would create a signal similar in structure to the $Y/C$ composite video NTSC signal - with the $Y$ (luminance) at the lower frequencies and the $C$ (chrominance) above. Approximately 3 kW of power would be required, in order to get 40 dB of signal to noise ratio for a composite FM signal in the 22 GHz band. This was incompatible with satellite broadcast techniques and bandwidth.

To overcome this limitation, it was decided to use a separate transmission of $Y$ and $C$. This reduces the effective frequency range and lowers the required power. Approximately 570 W (360 for $Y$ and 210 for $C$) would be needed in order to get a 40 dB of signal to noise ratio for a separate $Y/C$ FM signal in the 22 GHz satellite band. This was feasible.

There is one more power saving that appears from the character of the human eye. The lack of visual response to low frequency noise allows significant reduction in transponder power if the higher video frequencies are emphasized prior to modulation at the transmitter and then de-emphasized at the receiver. This method was adopted, with crossover frequencies for the emphasis/de-emphasis at 5.2 MHz for $Y$ and 1.6 MHz for $C$. With this in place, the power requirements drop to 260 W of power (190 for $Y$ and 69 for $C$).

Sampling systems and ratios

Main article: Chroma subsampling

The subsampling in a video system is usually expressed as a three part ratio. The three terms of the ratio are: the number of brightness (luma) $Y$ samples, followed by the number of samples of the two color (chroma) components $Cb$ and $Cr$, for each complete sample area. Traditionally the value for brightness is always 4, with the rest of the values scaled accordingly.

A sampling of 4:4:4 indicates that all three components are fully sampled. A sampling of 4:2:0, for example, indicated that the two chroma components are sampled at half the horizontal sample rate of luma - the horizontal chroma resolution is halved. This reduces the bandwidth of an uncompressed video signal by one-third.

MUSE implements a similar system as a means of reducing bandwidth, but instead of static sampling, the actual ratio varies according to the amount of motion on the screen. In practice, MUSE sampling will vary from approximately 4:2:1 to 4:0.5:0.25, depending on the amount of movement. Thus the red-green chroma component $Cr$ has between one-half and one-eighth the sampling resolution of the luma component $Y$, and the blue-yellow chroma $Cb$ has half the resolution of red-green.

Audio subsystem

MUSE had a discrete 2- or 4-channel digital audio system called "DANCE", which stood for Digital Audio Near-instantaneous Compression and Expansion.

It used differential audio transmission (differential pulse-code modulation) that was not psychoacoustics-based like MPEG-1 Layer II. It used a fixed transmission rate of 1350 kbp/s. Like the PAL NICAM stereo system, it used near-instantaneous companding (as opposed to Syllabic-companding like the dbx system uses) and non-linear 13-bit digital encoding at a 32 kHz sample rate.

It could also operate in a 48 kHz 16-bit mode. The DANCE system was well documented in numerous NHK technical papers and in a NHK-published book issued in the USA called Hi-Vision Technology.

The DANCE audio codec was superseded by Dolby AC-3 (a.k.a. Dolby Digital), DTS Coherent Acoustics (a.k.a. DTS Zeta 6x20 or ARTEC), MPEG-1 Layer III (a.k.a. MP3), MPEG-2 Layer I, MPEG-4 AAC and many other audio coders. The methods of this codec are described in the IEEE paper:

Real world performance issues

Unlike traditional, interlaced video where interlacing is done on a line by line basis, showing either odd or even lines of video at any one time, thus requiring 2 fields of video to complete a video frame, MUSE used a four-field dot-interlacing cycle, meaning it took four fields to complete a single MUSE frame, and dot interlacing is interlacing that was done on a pixel by pixel basis, dividing both horizontal and vertical resolution by half to create each field of video, and not in a line by line basis as in traditional interlaced video which reduces only the vertical resolution to create each video field. Thus, in MUSE, only stationary images were transmitted at full resolution. However, as MUSE lowers the horizontal and vertical resolution of material that varies greatly from frame to frame, moving images were blurred. Because MUSE used motion-compensation, whole camera pans maintained full resolution, but individual moving elements could be reduced to only a quarter of the full frame resolution. Because the mix between motion and non-motion was encoded on a pixel-by-pixel basis, it wasn't as visible as most would think. Later, NHK came up with backwards compatible methods of MUSE encoding/decoding that greatly increased resolution in moving areas of the image as well as increasing the chroma resolution during motion. This so-called MUSE-III system was used for broadcasts starting in 1995 and a very few of the last Hi-Vision MUSE LaserDiscs used it (A River Runs Through It is one Hi-Vision LD that used it). During early demonstrations of the MUSE system, complaints were common about the decoder's large size, which led to the creation of a miniaturized decoder.

Shadows and multipath still plague this analog frequency modulated transmission mode.

Japan has since switched to a digital HDTV system based on ISDB, but the original MUSE-based BS Satellite channel 9 (NHK BS Hi-vision) was broadcast until September 30, 2007.

Cultural and geopolitical impacts

Internal reasons inside Japan that led to the creation of Hi-Vision

• (1940s): The NTSC standard (as a 525 line monochrome system) was imposed by the US occupation forces.
• (1950s-1960s): Unlike Canada (that could have switched to PAL), Japan was stuck with the US TV transmission standard regardless of circumstances.
• (1960s-1970s): By the late 1960s many parts of the modern Japanese electronics industry had gotten their start by fixing the transmission and storage problems inherent with NTSC's design.
• (1970s-1980s): By the 1980s there was spare engineering talent available in Japan that could design a better television system.

MUSE, as the US public came to know it, was initially covered in the magazine Popular Science in the mid-1980s. The US television networks did not provide much coverage of MUSE until the late 1980s, as there were few public demonstrations of the system outside Japan.

Because Japan had its own domestic frequency allocation tables (that were more open to the deployment of MUSE) it became possible for this television system to be transmitted by Ku Band satellite technology by the end of the 1980s.

The US FCC in the late 1980s began to issue directives that would allow MUSE to be tested in the US, providing it could be fit into a 6 MHz System-M channel.

The Europeans (in the form of the European Broadcasting Union (EBU)) were impressed with MUSE, but could never adopt it because it is a 60 Hz TV system, not a 50 Hz system that is standard in Europe and the rest of the world (outside the Americas and Japan).

The EBU development and deployment of B-MAC, D-MAC and much later on HD-MAC were made possible by Hi-Vision's technical success. In many ways MAC transmission systems are better than MUSE because of the total separation of colour from brightness in the time domain within the MAC signal structure.

Like Hi-Vision, HD-MAC could not be transmitted in 8 MHz channels without substantial modification – and a severe loss of quality and frame rate. A 6 MHz version Hi-Vision was experimented with in the US, but it too had severe quality problems so the FCC never fully sanctioned its use as a domestic terrestrial television transmission standard.

The US ATSC working group that had led to the creation of NTSC in the 1950s was reactivated in the early 1990s because of Hi-Vision's success. Many aspects of the DVB standard are based on work done by the ATSC working group, however most of the impact is in support for 60 Hz (as well as 24 Hz for film transmission) and uniform sampling rates and interoperable screen sizes.

Device support for Hi-Vision

Hi-Vision LaserDiscs

On May 20, 1994, Panasonic released the first MUSE LaserDisc player. There were a number of MUSE LaserDisc players available in Japan: Pioneer HLD-XØ, HLD-X9, HLD-1000, HLD-V500, HLD-V700; Sony HIL-1000, HIL-C1 and HIL-C2EX; the last two of which have OEM versions made by Panasonic, LX-HD10 and LX-HD20. Players also supported standard NTSC LaserDiscs. Hi-Vision LaserDiscs are extremely rare and expensive.

The HDL-5800 Video Disc Recorder recorded both high definition still images and continuous video onto an optical disc and was part of the early analog wideband Sony HDVS high-definition video system which supported the MUSE system. Capable of recording HD still images and video onto either the WHD-3AL0 or the WHD-33A0 optical disc; WHD-3Al0 for CLV mode (up to 10 minute video or 18,000 still frames per side); WHD-33A0 for CAV mode (up to 3 minute video or 5400 still frames per side). These video discs were used for short video content such as advertisements and product demonstrations.

The HDL-2000 was a full band high definition video disc player.

Reel to reel VTRs

Analog VTRs

For recording Hi-Vision signals, Three reel to reel analog VTRs were released, among them are the Sony HDV-1000 part of their HDVS line, the NEC TT 8-1000 and the Toshiba TVR-1000.

These analog VTRs had a head drum angular speed of 3600 RPM and are similar to Type C VTRs. They output a video bandwidth of 30 MHz for luma and 7 MHz for both chroma channels each, with a signal to noise ratio of 41 dB. They accept luma and chroma signals with video bandwidths of up to 30 MHz for both. Video bandwidth is measured before FM modulation. Signals are recorded onto the tape using FM modulation.

Linear tape speed is 483.1 mm/s and writing speed at the heads is 25.9 m/s. The head drum is 134.5 mm wide and has 4 video record heads, 4 video playback heads and 1 video erasing head. It could record for 45 minutes on 10.5 inch reels. These machines, unlike conventional type C VTRs, are incapable of showing images while paused or playing the tape at low speeds. However they may be equipped with a frame store to capture images and display them while fast forwarding or rewinding the tape.

The video heads are made of Mn-Zn ferrite material, those used for recording have a gap of 0.7 microns and a width of 80 microns and those for playback have a gap of 0.35 microns and a width of 70 microns. It records audio on 3 linear tracks, and control signals on a linear track. Unlike conventional type C videotape recorders, Vertical Blanking Intervals are not recorded on the tape. Helical tracks have groups of 4 signals or channels, arranged side by side and length-wise with red chrominance, blue chrominance, and two green chrominance signals with luminance information. Two tracks for green chrominance plus luminance are used to increase the bandwidth of these signals that can be recorded on the tape. Each of these 4 signals have a video bandwidth of 10 MHz.

The VTR uses Iron metal oxide tape with cobalt for high coercivity, with capacity for 40 MHz of bandwidth at a head drum speed of 3600 RPM, which is sufficient for applying FM modulation to 10 MHz signals. To record 4 channels simultaneously in a single helical track, a separate, independent video head is required for each channel, and 4 video heads are grouped together which make a single helical track with 4 channels.

Digital VTRs

In 1987, technical standards for digital recording of Hi-Vision signals were released by NHK, and Sony developed the HDD-1000 VTR as part of their HDVS line, and Hitachi developed the HV-1200 digital reel to reel VTR. Audio is recorded digitally similarly to a DASH (Digital Audio Stationary Head) digital audio recorder, but several changes were made to synchronize the audio to the video. These digital VTRs can record 8 channels of digital audio on linear tracks (horizontally along the entire length of the tape).

According to the standards, these VTRs operate with a head drum speed of 7200 RPM to accommodate the higher signal bandwidths of digital signal modulation on the tape which is also accommodated with the use of metal alloy particle tape, have a bit rate of 148.5 Mbit/s per video head, a linear tape speed of 805.2 mm/s and a writing speed at the heads of 51.5 m/s, are similar to Type C VTRs, have a head drum 135mm wide, 8 video playback, 8 video recording and 2 video erase heads, with 37 micron wide helical tracks. Output signal bandwidth is 30 Mhz of video bandwidth for luma (Y) and 15 Mhz of video bandwidth for chroma (P_b, P_r).

Audio is recorded with a sampling rate of 48 kHz stored at 16 bits per sample in linear tape tracks, sampling rate for luma is 74.25 Mhz and 37.125 Mhz for chroma stored at 8 bits per sample. Signal to noise ratio is 56 dB for chroma and luma. Video fields are divided into 16 helical tracks on the tape. Total video bandwidth is 1.188 gigabits/s. Cue signals are recorded into 3 linear tape tracks. Video is recorded in groups of 4 tracks or channels, which are side by side length-wise within each helical track, to allow for parallelization: high total data rates with relatively low data rates per head, and reduce the linear tape speed.

Digital video signals are recorded line by line (1 row of pixels in every frame of video or 1 line of video at a time) with ECC (Error Correcting Code) at the end of each line and in between a number of vertical lines. Reed-Solomon code is used for ECC and each line also has an ID number for trick play such as slow motion and picture search/shuttle.

Displays

Hi-Vision requires a display capable of handling 30 Mhz of video bandwidth simultaneously for each of the component video channels: R, G, B or Y, P_b and P_r. It was displayed on direct view color CRTs and CRT projectors, and plasma displays and Talaria projectors were explored to determine their ability to display Hi-Vision images. Some TVs have built in MUSE decoders.

Cameras

Cameras based on Saticon tubes, Plumbicon tubes, Harpicon tubes and CCD image sensors were used to capture footage using the Hi-Vision format. A prototype based on Vidicon tubes was also created.

MUSE decoders

A MUSE decoder is required for receiving MUSE broadcasts from satellites, and for viewing content in the MUSE format. The decoder converts MUSE format signals into Hi-Vision component video signals that can then be shown in a display.

Video cassettes

W-VHS allowed home recording of Hi-Vision programmes.

UniHi

For recording Hi-Vision video signals, NHK and 10 Japanese companies ("NEC, Matsushita Electric Industrial, Toshiba, Sharp, Sony, Hitachi, Sanyo Electric, JVC, Mitsubishi Electric, Canon") in 1989 released UniHi, a professional videocassette format. Recorders for the format were manufactured by Panasonic, Sony, NEC, and Toshiba. These machines were less expensive than their Type C counterparts. Both studio and portable versions were made.

The head drum spins at 5400 RPM and uses tape that is 12.65 mm wide. It has a luminance (Y) bandwidth of 20 MHz and a chrominance (P_b, P_r) bandwidth of 7 MHz for video output. Video is recorded in analog form. The head drum is 76mm wide. It uses two video heads with azimuth recording and records each frame of video into 12 helical tracks; only 6 tracks are necessary for each video field if recording interlaced video. Audio is recorded digitally as a PCM signal, as a section on the helical tracks. Writing speed at the heads is 21.4 m/s.

The tape also has 3 linear tracks, one for audio, control and time code each. Signal to noise ratio for luminance is 41 dB and for chrominance it is 47 dB. The tape is wrapped 180° around the head drum. Development began in 1987. It uses metal particle tape. It could record video for 1 hour (63 minutes). Linear tape speed is 120 mm/s. The cassette measures 205mm (width) x 121mm (depth) x 25mm (height). Signals are recorded using time-compression integration, in groups of two signals length-wise on each helical track. Grouping is used to increase the bandwidth that can be recorded on the tape. The cassette is intented to be air-tight with two flaps in the cassette's opening to protect the tape.

This videocassette format was developed in order to reduce the size of HD recording equipment. The Sony version of the UniHi VTR, the HDV-10, had a price of over 90,000 US dollars.

See also

• Analog high-definition television system
• HD-MAC, a planned high-definition analog video standard in Europe
• Clear-Vision

The analog TV systems these systems were meant to replace:

• SECAM
• NTSC
• PAL

Related standards:

• NICAM-like audio coding is used in the HD-MAC system.
• Chroma subsampling in TV indicated as 4:2:2, 4:1:1 etc...
• ISDB

External links

• Example of an early MUSE LaserDisc player
• Hi-Vision, MUSE, and the Optical Disc