Featuring H.264

For full format paper (including table, picture, graphic etc) Please open at Featuring H.264

Now Featuring H.264
The Benefits of Standards
Confidence. Quite simply, standards build
confidence. Because of standards, you can be
sure that any CD plays in any CD player; any
television station can be viewed on any brand
of television; any DVD plays in any DVD player.
Innovation and new markets. Instead of
a world of small competing technology
fiefdoms, standards create the foundation
for widespread adoption of innovative new
consumer products. Satellite television is
based on the MPEG-2 standard developed
in 1994. Standard audio formats have made
MP3 players commonplace.
Economy. As standards are ratified, the
industry can focus on how to deploy them at
a lower cost instead of developing redundant
technologies. The adoption of the MPEG-2
standard allowed the broadcast and DVD
industries to focus their efforts on developing
innovative tools for creating and delivering
MPEG-2 rather than developing alternatives
to MPEG-2, ultimately lowering delivery costs.
Choice. Standards enable the builders of
media networks to select products from a
number of vendors and integrate them
into a single, scalable system. Competition
between vendors results in a broader choice
of products varying in cost, performance,
and features.
Reduced costs. Using standards, content
providers can eliminate the time-consuming
and costly process of encoding and managing
the same material in multiple formats.
Increased revenue. Content providers can
leverage standards to offer their content via
new platforms such as mobile networks, the
Internet, and digital television.
Technology Brief
QuickTime and MPEG-4
The most exciting phase in the evolution of MPEG-4 is about to begin. MPEG-4, the
newest in a long line of successful worldwide multimedia standards, now includes
H.264 video. Providing stunningly beautiful video in compact files, H.264 will revolutionize
virtually every application that uses video, from mobile multimedia to video
conferencing to broadcast and satellite television.
What Is MPEG-4?
MPEG-4 is a multimedia standard with audio and video at its core. It was defined
by the MPEG (Moving Picture Experts Group) committee, the working group in the
International Organization for Standardization (ISO) that specified the widely adopted,
Emmy Award–winning standards known as MPEG-1 and MPEG-2. MPEG-4 is the result
of an international effort involving hundreds of researchers and engineers. The initial
parts of MPEG-4, whose formal designation is ISO/IEC 14496, were finalized in October
1998 and became an international standard in early 1999.
MPEG-4 was created to ensure seamless delivery of high-quality audio and video
over the Internet, on IP-based networks, and to a new generation of consumer digital
media devices. Because these devices range from narrowband cell phones to broadband
set-top boxes to broadcast high definition (HD) televisions, MPEG-4 provides
high-quality audio and video across the entire bandwidth spectrum.
Based on a time-tested technology
The ISO could have chosen to base its MPEG-4 standard on any existing file format, or
it could have created an entirely new format. The ISO chose to use the QuickTime file
format because of its decade-long track record in the industry. Quite simply, QuickTime
works. Designed from the ground up to be cross-platform and media agnostic while
providing stability, extensibility, and scalability, QuickTime delivers the foundation
needed to encode, process, and play digital media on any MPEG-4–compliant device.
Since its inception in 1991, QuickTime has incorporated the best technology available
to deliver the highest-possible quality.
3D
VR
MIDI
Graphics
Text
and many more
Video
Audio
Metadata
User Data
Copyright Information
Media Data
Media Index
Types of Tracks
Compression Format
Edit Information
QuickTime File Format
Technology Brief 2
QuickTime and MPEG-4
The QuickTime file format is a “container” that can hold a variety of media types and
their respective data, such as audio, video, Macromedia Flash, animations, text, images,
and VR. QuickTime packages these media types as “tracks,” which are a key component
of what has made QuickTime so adaptable. New capabilities can be added simply by
creating new track types. What’s more, new versions of QuickTime maintain backward
compatibility, ensuring the continued viability of multimedia developed with earlier
versions. This unique combination of flexibility and backward compatibility made the
QuickTime file format the perfect choice for the basis of MPEG-4.
MPEG-4 Success
MPEG-4 is an extremely well-adopted format. It has been distributed in QuickTime 6
to hundreds of millions of Internet users around the world. It has been built into digital
video and still cameras from some of the industry’s largest companies. It is the format
used in the incredibly popular iPod and iTunes Music Store. And it has been built into
tens of millions of multimedia-enabled cell phones around the world.
In addition to the flexible MPEG-4 file format, the technologies at the heart of this
success to date are the MPEG-4 Part 2 video codec and the Advanced Audio Coding
(AAC) audio codec.
MPEG-4 Part 2 video delivers improved compression efficiency over previous video
standards by employing updated coding tools and techniques. For example, the codec
provides single-pass variable bit rate (VBR) control, meaning that the encoder can be
set to a target data rate that ensures playback at the appropriate rate for a particular
delivery mechanism. In addition, the QuickTime MPEG-4 Part 2 codec features rigorous
color management, a motion estimator optimized for both precision and speed, and
a high-performance quantizer, all of which work together to provide high-quality
video time after time. The decoder also provides an optimized post-processing stage
to remove coding artifacts. The encoder and decoder are optimized for the Velocity
Engine, with additional decoder optimizations implemented for G3 systems. Finally,
the QuickTime MPEG-4 Part 2 codec is compatible with numerous other MPEG-4 technologies,
as proven in a host of industrywide interoperability tests.
Technology Brief 3
QuickTime and MPEG-4
AAC audio provides much more efficient compression than older formats like MP3, yet
it delivers quality rivaling that of uncompressed CD audio. AAC was developed by the
MPEG group that includes Dolby, Fraunhofer (FhG), AT&T, Sony, and Nokia—the same
audio experts that created MP3 and AC-3. Taking full advantage of the many advances
in perceptual audio coding and compression achieved since the development of MP3
more than a decade ago, AAC delivers higher-quality output at lower data rates. And
with support for modern audio requirements such as multichannel audio at up to 48
full-frequency channels and high-resolution sampling rates up to 96kHz, AAC proves
itself worthy of replacing MP3 as the new standard for Internet audio. The AAC codec
in QuickTime builds on state-of-the-art signal processing technology from Dolby
Laboratories and offers both a constant bit rate mode for bandwidth-constrained scenarios
and a constant quality mode for a consistently high-quality listening experience.
An evolving standard
The beauty of the MPEG standards is that they don’t stand still. As evidenced by the
long and continually improving life of MPEG-2, standard formats can and do evolve
as technologies advance. MPEG-4 is no different. Over the last few years, hundreds
of the world’s finest video experts in hardware, software, video telephony, authoring,
streaming, cable, television, and DVD have been hard at work specifying, scrutinizing,
evaluating, verifying, and ratifying the next-generation standard for video. Working
jointly with the International Telecommunications Union (ITU), the MPEG committee
created ITU-T Recommendation H.264|ISO/IEC 14496-10 AVC, otherwise known as
MPEG-4 Part 10 or, more simply, H.264.
H.264: A Revolution in Quality and Coding Efficiency
H.264 is an intricate web of the latest innovations in video compression technologies
that together provide a vast improvement over previous generations of video codecs.
For example, H.264 delivers the same quality as MPEG-2 at a third to half the data rate.
When compared to MPEG-4 Part 2, H.264 provides up to four times the frame size at a
given data rate.
Thanks to its impressive efficiency, H.264 provides up to four times the resolution of
MPEG-4 Part 2 at the same quality and data rate.
Technology Brief 4
QuickTime and MPEG-4
In addition to exceptional efficiency, H.264 provides better image quality than past
standards when reaching its limits. Rather than breaking down into distinct blocks
and jagged objects, H.264 degrades gracefully, softening an image as compression
increases.
It’s important to keep in mind that H.264 is at the very beginning of its life cycle; users
can expect many years of improvement from this incredible foundation of compression
technologies. Just as today’s MPEG-2 video quality exceeds the highest expectations
of more than a decade ago when the MPEG-2 standard was ratified, H.264 will improve
with age.
Scalable from 3G to HD and beyond
With H.264, the ITU and MPEG groups set out to achieve the best-ever compression
efficiency for a broad range of applications—from broadcast and DVD, where MPEG-2
is traditionally used, to video conferencing, which has used H.263, to video on demand,
streaming and multimedia messaging, which have used MPEG-4 Part 2. Their efforts
were successful: H.264 delivers excellent quality across a wide operating range, from 3G
to HD and everything in between. Whether creating video for mobile phones, iChat AV,
the Internet, broadcast, or satellite delivery, H.264 provides exceptional video quality at
impressively low data rates.
H.264: Exceptional quality at any size
Scenario/Use Resolution and frame rate Example data rates
Mobile content (3G) 176 by 144, 10–24 fps 50–160 Kbps
Internet/Standard definition (SD) 640 by 480, 24 fps 1–2 Mbps
High definition (HD) 1280 by 720, 24p 5–6 Mbps
Full high definition (full HD) 1920 by 1080, 24p 7–8 Mbps
The impressive efficiency of H.264 makes state-of-the-art video more accessible to
more people, enabling anyone to experience the full quality of HD video on today’s
computers. With H.264, HD video plays back seamlessly on current desktop hardware.
Specifically, 1280-by-720 24p video plays perfectly on a single processor Power Mac G5,
while full 1920-by-1080 24p video plays beautifully on a dual processor Power Mac G5.
Technology Brief 5
QuickTime and MPEG-4
H.264 plays on today's computers.
Features of the H.264 standard
The H.264 standard fully specifies the H.264 bit stream and decoder, while more
generally specifying a set of encoding tools that can be used in creating H.264 content.
The H.264 encoding toolset incorporates the best ideas from past video standards,
improves on them, and adds many more new technologies. The list of tools and
technologies is quite extensive; here are a few of the most notable.
Advanced B-frame support. H.264 uses a more complex form of B-frame support, or
frame reordering, than previous standards like MPEG-2 to increase coding efficiency.
Frame reordering is based on the concept of describing a given frame using data from
other frames in the past and the future. The frame that depends on data from past
and future frames is called a B-frame, or bidirectionally predicted frame. Traditionally,
a B-frame could refer only to the frame immediately before or immediately after it
in the movie timeline. With H.264, a B-frame can refer to an entire stack of frames,
whether past or future, anywhere in the movie timeline. And unlike other codecs, a
B-frame can be used to describe another frame, meaning it can also be a reference
frame. Because H.264 uses data from many more frames anywhere in the movie
timeline to most efficiently represent a given frame, it delivers a truer representation
of the source material in a much lower data rate than other codecs.
P
B B
B B
B B B B
I I
P
P B
B
P
I
B
I
P
P
B
B
MPEG-1 and MPEG-2 Frame Dependencies H.264 Frame Dependencies
Display Time
Only in “open GOPs”
Display Time
Information Flow
Information Flow
I
Frame Dependency Comparison
The Benefits of H.264
Unprecedented video quality. H.264 uses
the latest innovations in video compression
technology to provide consistently crisp and
clear video for the best possible viewing
experience.
Ultra-efficient. H.264 delivers incredible
video quality at data rates one-fourth to
one-half the size of previous video formats.
Scalable from 3G to HD. Use a single codec
for all your delivery needs. H.264 delivers
great results for everything from mobile
multimedia to Internet to satellite and
beyond.
HD playback on today’s computers. High
definition H.264 video plays back seamlessly
on today’s G5 desktop hardware. With an
Apple Cinema HD Display and a dual Power
Mac G5, the home office is now the home
theater.
Ratified standard for industrywide
interoperability. H.264 content created with
QuickTime can play back on a broad range
of H.264 devices, including mobile phones,
set-top boxes, DVD players, and more.
Likewise, files created with these devices
can play back in QuickTime.
Technology Brief 6
QuickTime and MPEG-4
4x4 integer transform. H.264 is designed to operate on much smaller blocks of pixels
than other common codecs, which mitigates blocking, smearing, and ringing artifacts.
So H.264 video is crystal clear even in areas of fine detail. Because the transform is an
integer transform, it provides bit-precise reconstruction (that is, exact-match decoding)
rather than statistically generated reconstruction. As a result, there can be no drift
among various decoder implementations, so any compliant H.264 decoder will decode
the video exactly as the content author intended it to look.
Increased precision in motion estimation. H.264 also benefits from increased
precision in motion estimation, which is the process of simplifying redundant data
across a series of frames. By expressing information to 1/4-pixel resolution as opposed
to 1/2-pixel resolution like most other codecs, H.264 represents both fast- and slowmoving
scenes more precisely. So objects in motion are more crisply reconstructed
during decode, providing a better representation of the source material.
Flexible block sizes in motion estimation. During motion estimation, traditional
codecs commonly process frames at the macroblock level (16 pixels by 16 pixels).
H.264 can process on segments within a macroblock, ranging in size from the commonly
used 16x16 to as small as 4x4, which helps to code complex motion in areas
of high detail. The ability of H.264 to perform its processing on a variety of block
sizes means that scenes with complicated motion are more expressively described,
providing higher quality in lower data rates.
Intraframe prediction. H.264 is able to gain much of its efficiency by simplifying
redundant data not only across a series of frames, but also within a single frame, a
technique called intraframe prediction. The H.264 encoder uses intraframe prediction
with more ways to reference neighboring pixels, so it compresses details and gradients
better than previous codecs. Intraframe prediction is especially beneficial in highmotion
areas, which are traditionally difficult to encode. With H.264, high-motion
video can achieve stunning quality at much lower data rates.
Adaptively tuned deblocking filter. H.264 also features a robust deblocking filter,
which operates on 4x4 block boundaries to remove jagged blocking artifacts. Its
filtering is adaptively tuned per block boundary, making it a very effective smoothing
filter during the decoding of a finished bit stream. In addition to making smoother
pictures for display, this filter is used during the encoding process to provide a more
coherent reference picture for subsequent frames, which helps to improve image
quality. This advanced filter technology effectively eliminates blocking artifacts,
resulting in a smooth, clean picture.
Profiles and levels
Like any comprehensive standard, the H.264 standard defines a set of profiles and
levels to set points of conformance for various classes of applications and services.
In each profile, specific encoding tools are permitted to best meet the needs of the
intended scenario. H.264 includes four profiles:
• Baseline. Intended for low-latency applications such as video conferencing.
• Main. Intended for the majority of general uses, such as the Internet, mobile
multimedia, and stored content.
• Extended. Intended for streaming applications, where extra error resilience
technologies can be beneficial.
• Fidelity Range Extension (also known as High). Consists of four separate High
profiles (High, High 10, High 4:2:2, and High 4:4:4), intended for specific high-end
professional uses.
Technology Brief 7
QuickTime and MPEG-4
4:4:4
4:2:2
10 Bit
8x8
Sep. Cb, Cr
Monochrome
Stream
Error Resilience Switching
CABAC
B
I, P
CAVLC
4:2:0
6 Bit
Baseline
Extended
Main
High
High (FRExt)
High 10
High 4:2:2
High 4:4:4
For each profile, 16 levels can be applied, each specifying a typical frame size, frame
rate, and maximum data rate. The same 16 levels are used for each profile. This extensive
system of levels was designed to allow a large enough data rate in a given level
to produce 30 frames per second at the specified frame size.
H.264 levels
Level Typical picture size Typical frame rate Maximum bit rate
(non-FRExt)
1 QCIF 15 64 Kbps
1b QCIF 15 128 Kbps
1.1 CIF or QCIF 7.5 (CIF)/30 (QCIF) 192 Kbps
1.2 CIF 15 384 Kbps
1.3 CIF 30 768 Kbps
2 CIF 30 2 Mbps
2.1 HHR (480i or 576i) 30/25 4 Mbps
2.2 SD 15 4 Mbps
3 SD 30/25 10 Mbps
3.1 1280x720p 30 14 Mbps
3.2 1280x720p 60 20 Mbps
4 HD (720p or 1080i) 60p/30i 20 Mbps
4.1 HD (720p or 1080i) 60/30i 50 Mbps
4.2 1920x1080p 60p 50 Mbps
5 2kx1k 72 135 Mbps
5.1 2kx1k or 4kx2k 120/30 240 Mbps
H.264 Profiles
Technology Brief 8
QuickTime and MPEG-4
Apple’s H.264 Video Codec: Under the Hood
The H.264 standard specifies the decoder and bit stream syntax, but leaves the methodology
of encoding up to the individual implementation. Leveraging its many years
of experience in creating successful video coding technologies, Apple has created an
industry-leading H.264 implementation in QuickTime 7. Using a combination of specified
technologies and patent-pending techniques, the QuickTime H.264 video codec
produces phenomenal results.
Intelligent multipass encoding
Apple’s implementation of the H.264 encoder employs a patent-pending technology
for using the optimal number of passes to produce the best possible quality at the
desired bit rate. This technology produces superior quality and bit rate efficiency compared
with traditional dual-pass and single-pass encoders, and it takes the guesswork
out of the encoding process by intelligently determining the optimal number of
compression passes to perform.
H.264 encoding modes in QuickTime 7 Pro.
Single-pass encoding mode
When encoding speed is a higher priority than pristine quality, the QuickTime 7
H.264 encoder offers a single-pass encoding mode for a good result in a very short
time. Using patent-pending techniques, this accelerated encoding mode saves time
when the user wants to create “draft” versions of an encode or when a deadline is
quickly approaching.
Technology Brief 9
QuickTime and MPEG-4
Peak-constrained VBR options
QuickTime 7 allows the user to limit the upper boundary of the H.264 variable bit rate
encode for bit rate–constrained situations. The H.264 encoder offers peak-constraining
options tailored to two specific playback scenarios: streaming and CD-ROM/DVD. Apple’s
patent-pending technology for constraining peak data rate delivers better visual quality
for a given bit rate and ensures a smooth playback experience for narrowband connections
and devices such as CD-ROM and DVD players.
Peak-constrained VBR options for H.264 in QuickTime 7 Pro.
Conformance
The initial implementation of Apple’s H.264 decoder conforms to all levels of Main
profile in progressive format. Apple’s H.264 encoder can create MPEG-4 files that
interoperate with both Baseline and Main profile-conforming decoders.
Innovating to deliver exceptional quality
Because the set of encoding tools in the H.264 standard is specified but not explicitly
defined, Apple is able to create a superior H.264 video encoder using newly invented
techniques. The QuickTime H.264 encoder benefits from patent-pending technologies
in the areas of intelligent multipass encoding, single-pass encoding, constraining peak
data rate, bit allocation, and motion estimation to deliver unparalleled quality.
H.264 Throughout the Apple Product Line
Apple has a long history of using standards to support the adoption of innovative technologies,
and MPEG-4 is no exception. From USB and Bluetooth connections for peripherals,
to AirPort (802.11) wireless networking, to FireWire (IEEE 1394) high-speed digital
video transfer, Apple has championed technology standards that have been adopted
throughout the industry. Apple has worked closely with the MPEG committee to define
and ratify the MPEG-4 standard, including the latest addition to the standard, H.264.
Having spent years contributing to and then implementing the H.264 standard, Apple
is now ready to introduce this outstanding technology throughout its product line.
As the foundation on which other multimedia applications at Apple are built, QuickTime
houses H.264 as well as a multitude of other popular video, audio, and still image codecs.
Because H.264 is built into the QuickTime architecture, QuickTime-based applications
can take full advantage of this new video codec.
Technology Brief 10
QuickTime and MPEG-4
QuickTime is an essential component of the Mac OS X operating system. The latest
version of Mac OS X, v10.4 “Tiger,” leverages the H.264 support in QuickTime 7 to
deliver high-quality, multiperson video chatting in iChat AV, as well as to display
H.264 video in the Finder, Preview, Safari, and Mail.
QuickTime 7 also brings the benefits of H.264 to Apple’s consumer applications. Users
can enjoy stunning H.264 video playback in QuickTime Player and iTunes. They can
create high-quality H.264 versions of their iMovie projects for sharing with others
via email or the web. And users can create impressive presentations by incorporating
beautiful H.264 video in Keynote projects.
For video professionals, H.264 brings even more exciting possibilities. Thanks to the
H.264 support in QuickTime 7, Final Cut Studio users have the ability to deliver highquality
content at lower data rates for the web and DVD. Final Cut Studio includes
Compressor, which allows users to tap the power of networked Macintosh computers
to dramatically reduce H.264 encoding times. When Compressor is coupled with
Xsan, the SAN file system for Mac OS X, encoding from uncompressed high definition
video to H.264 is dramatically accelerated. DVD Studio Pro 4 leverages the quality and
efficiency of H.264 to deliver HD quality at SD data rates on DVD. DVD Studio Pro 4 is
the first commercially available DVD authoring software that lets users burn their HD
projects to high definition DVDs based on the latest HD DVD specifications. Created
using existing DVD burners and media, these discs can play back high definition content
in H.264 on a dual processor Power Mac G5 with the latest version of Apple’s DVD
Player, which also leverages H.264 support in QuickTime 7.
Technology Brief 11
QuickTime and MPEG-4
Widespread Industry Adoption of H.264
Apple is not the only company embracing H.264. Many standards organizations and
hundreds of companies are actively working to incorporate this new video standard
into their specifications and products. Standards bodies and industry consortia supporting
H.264 include the following.
Moving Picture Experts Group (MPEG). As mentioned earlier, H.264 has already been
ratified by the MPEG committee as part of the MPEG-4 standard, specifically called
MPEG-4 Part 10 or Advanced Video Coding (AVC).
International Telecommunication Union (ITU). The cocreator of H.264, ITU, has
ratified H.264 as its next video conferencing standard.
DVD Forum. H.264 has been selected by the DVD Forum as mandatory for its HD-DVD
specification, which defines one of two formats for next-generation, high definition
DVDs.
Blu-ray Disc Association. H.264 has also been chosen by the Blu-ray Disc Association
as mandatory for its Blu-ray Disc specification, which defines the other of the two
formats for next-generation, high definition DVDs.
Digital Video Broadcasting (DVB). This European-born consortium for creating digital
television standards has chosen H.264 as part of its specification for broadcast of both
SDTV and HDTV.
3rd Generation Partnership Project (3GPP). This group creates standards for mobile
multimedia on GSM-type mobile networks. It has chosen H.264 as the primary video
codec in its Release 6 specification.
Internet Streaming Media Alliance (ISMA). H.264 has been adopted by the ISMA,
which was formed to accelerate the adoption and deployment of open standards for
streaming rich media content over Internet protocols.
MPEG Industry Forum (MPEGIF). This group was created to further the adoption of
MPEG standards, including H.264, through promotion, interoperability testing, certification
programs, and other activities.
AVC Alliance. Formed to promote H.264 (sometimes called AVC) based products and
services that work together seamlessly, the AVC Alliance is made up of some of the
industry’s leading international companies.
H.264 is also under consideration for adoption by other standards groups, including
the 3rd Generation Partnership Project 2 (3GPP2), the group creating standards
for mobile multimedia on CDMA2000 networks; and Advanced Television Systems
Committee (ATSC), the U.S. group creating standards for digital television.
In addition to its broad standards support, numerous broadcast, cable, video
conferencing, and consumer electronics companies consider H.264 the video codec
of choice for their new products and services. Some of the most notable are Japan’s
top six broadcasting companies, DirecTV, Tandberg, and Philips.
Technology Brief 12
QuickTime and MPEG-4
MPEG-4, H.264 and QuickTime 7
The already successful MPEG-4 standard now features H.264, a video codec that
delivers unparalleled quality with even more impressive efficiency. Using the latest
innovations in video compression technology, H.264 delivers incredible results for the
entire range of applications, from mobile to Internet to satellite and beyond. Now that
H.264 is built into the QuickTime 7 architecture, QuickTime-based applications from
Apple and other companies can offer H.264 encoding and decoding capabilities. Best
of all, H.264 is a broadly supported standard. This adoption by a wide variety of open
standards means that any company in the world can create devices—mobile phones,
set-top boxes, DVD players, and more—that interoperate with QuickTime 7.
Get ready for QuickTime 7 with H.264 to change the digital video universe. Experience
this powerful addition to QuickTime 7 today.
For More Information
For more information about QuickTime
products and MPEG-4 technologies, visit
www.apple.com/quicktime.