Our mission is to identify and explain the
technologies and applications that allow television services to
be provided through Internet Protocol (IP) data networks.
Readers learn the options and the system to implement IPTV along
with new features and applications and business opportunities
that are available in the IPTV industry today.
Jake Everett, Head of Marketing, iPharro Media GmbH
Identifying ads that are received by specific
customers can add significant value to advertising programs and ensure
that unauthorized broadcast, alteration and usage of content does not
occur. The availability of broadband communication channels to end-user
devices has enabled ubiquitous media coverage with image, audio, and video
content. The increasing amount of multimedia content that is transmitted
globally has boosted the need for intelligent content management.
Similarly, broadcasters and market researchers want to know when and where
specific footage has been broadcast. Content monitoring, market trend
analysis, and copyright protection are emerging applications in the new
world of digital media.
The iPharro MediaSeeker technology is capable
of comparing digital footage, such as films, clips, and advertisements,
against digital media broadcasts from virtually any source. This enables
automatic and efficient supervision of digital content. The iPharro
MediaSeeker system is highly scalable, and uses superior computer vision
and signal processing technology for analyzing footage in video and audio
domains in real time.
Users can insert their reference content into
the iPharro MediaSeeker system, and nominate one or more media sources,
as different broadcast television channels
or digital video streams for monitoring. The system then generates
detailed statistics about the appearance of the reference content
within the monitored media sources and a copy of the broadcast footage
is retained for confirmation purposes. By minimizing the required
amount of manual intervention, virtually any source can be monitored
in a cost-effective way.
The system extracts the relevant
information from the video stream data itself and can therefore
efficiently monitor a nearly unlimited number of channels without
manual interaction. The iPharro MediaSeeker system computes digital
signatures (called fingerprints) from the reference content. The
fingerprints describe specific audiovisual aspects of the content,
such as color distribution, shapes, patterns, and the frequency
spectrum in the audio stream. Each piece of video has a unique
fingerprint that is basically a compact digital representation of its
unique audiovisual characteristics. The fingerprints of the reference
content are stored in a reference database along with all relevant
Figure 1 demonstrates how the video sources
to be monitored are buffered by signal acquisition units. Fingerprints
from these sources are extracted offline and then compared to the
fingerprints in the reference database.
Figure 1 Fingerprint Based Video Comparison
Figure 2 Fingerprinting, Segmentation and Key
Frame Selection Source iPharro
iPharro MediaSeeker uses a fast multi-stage
fingerprint comparison engine that reliably identifies any occurrence of
the reference content in the video data stream that is being monitored.
Figure 2 shows the fingerprinting process, in
which the system clusters similar frames that occur within close
proximity. This results in the temporal segmentation of the video into
small, visually coherent units called shots. For each shot, one
representative frame (key frame) is selected that will be used for
visualization in a storyboard. The fingerprints of the individual frames
are combined to form the video fingerprint for the entire clip. Based on
these fingerprints, iPharro MediaSeeker is able to identify if and when
reference content or parts thereof appear in one of the video streams
being monitored. Within the matching process, iPharro MediaSeeker analyzes
the footage to identify regions of interest (ROI). A region of interest
occurs, for example, when reference content is not shown full-screen, but
as a downsized version along with other content in a video. In such cases,
the analysis engine is able to identify the region in which the reference
content is shown, and disregards other content in subsequent processing
The media acquisition subsystem acquires the
video signal and records it as data chunks on the signal buffer units.
Depending on the use case, the buffer units may perform fingerprint
extraction as well.
This is useful in a remote capturing
scenario in which the very compact fingerprints are transmitted via
the Internet from a distant capturing site to a centralized content
The fingerprint for each data chunk is
stored in the media repository. Each data chunk becomes an analysis
task that is scheduled for processing by the controller. The
controller is primarily responsible for load balancing and the
distribution of jobs to individual nodes in the content analysis
cluster. The content analysis units fetch the recorded data chunks
from the signal buffer units directly and extract fingerprints prior
to the analysis. After processing several data chunks, the detection
results for these chunks are stored in the system database. The number
of signal buffer units and content analysis nodes may flexibly be
scaled to customize the system's capacity to specific use cases of any
Figure 3 depicts details of the iPharro
MediaSeeker system overview.
The complete iPharro MediaSeeker system
consists of many software components that may be combined and
configured to suit individual needs. Depending on the specific use
case, several components may be run on the same hardware;
alternatively, components are run on individual hardware for better
performance and improved fault tolerance.
The signal buffer units are designed to operate
around-the-clock without any user interaction. The continuous video data
stream is captured, divided into manageable chunks, and stored on the
internal hard disks. The hard disk space acts as a circular buffer as
older data chunks are moved to a separate long term storage unit for
archival. This guarantees uninterrupted signal availability over very long
periods of time. The controller will ensure the timely processing of all
data chunks so that no data is lost. The signal acquisition units are
designed to operate without any network connection, if required, to
increase the system's fault tolerance. The signal buffer units may
optionally perform fingerprint extraction and transcoding on the recorded
chunks, and store the fingerprints along with the data chunks. This
enables transmission of the very compact fingerprints including a
storyboard over limited-bandwidth networks, to avoid transmitting the full
With its GUI-based RecordingSelector tool,
iPharro MediaSeeker enables convenient configuration of the signal buffer
units. Through drag & drop, each capture card in the units can be assigned
which channels to record and individual priorities can be set to support
redundant signal acquisition.
The controller manages processing of the data
chunks recorded with the signal buffer units. All signal buffer units and
content analysis nodes are constantly monitored to perform load balancing.
The controller initiates processing of new data chunks by assigning
analysis jobs to the analysis nodes. The controller may automatically
restart individual analysis processes or entire analysis nodes, enabling
error recovery without user interaction. Through a graphical user
interface, the analysis subsystem can be configured and its status
monitored. The controller user interface is shown in Figure 5. To avoid a
single point of failure, the controller itself may be operated with full
redundancy using a secondary controller that is on hot stand-by during
normal operation. In this case, primary and secondary controllers maintain
and monitor a mutual heartbeat. In case the primary controller fails, the
secondary controller will automatically take over control of the system
and ensure uninterrupted operation. Manual switching of the primary and
secondary controller is supported as well, which allows the regular server
maintenance tasks to be conducted without the need to shut down the entire
The analysis cluster consists of one or more
analysis nodes which are the true workhorses of the iPharro MediaSeeker
Figure 4 Region of Interest (ROI) Detection
Each node independently processes the analysis
tasks that are assigned to it by the controller. This primarily includes
fetching the recorded data chunks, generating the video fingerprints, and
matching the fingerprints against the reference content. The resulting
data is stored in the media repository and in the database. The matching
process includes identification of regions in which reference content is
shown as a downsized version.
Figure 4 depicts a TV advertisement that is
shown in the upper left region of the screen while the rest of the screen
is occupied by program information. The iPharro MediaSeeker automatically
identifies the region in which the advertisement is shown - illustrated by
the blue border - to match it against the reference clip in the database.
This feature adapts to different sizes and aspect ratios, thus enabling
identification of reference content that is aired in split-screen or
Picture in Picture (PIP) formats. The analysis nodes may also operate as
reference clips ingestion nodes, backup nodes, or RetroMatch nodes in case
the system performs retrospective matching. However, all activity of the
analysis cluster is controlled and monitored by the controller.
iPharro MediaSeeker supports different SQLbased
relational database systems through its database access layer, such as
Oracle and Microsoft-SQL Server.
The system database acts as the central
repository for all metadata generated during operation, including
processing, configuration, and status information
The media repository is the main payload data
storage of the iPharro MediaSeeker system and holds the fingerprints, key
frames, and optionally a low quality version of the processed footage. The
media repository is normally implemented using one or more RAID systems
that can be accessed as a networked file system.
Many graphical user interfaces may be part of
an iPharro MediaSeeker system, such as the RecordingSelector or the
controller front-end. However, the main interface for operators, data
analysts, and other users is the iPharro MediaSeeker front-end. The
front-end enables users to review detentions, manage reference content,
edit clip metadata, play reference and detected footage, and perform
detailed comparisons between reference and detected content. Often,
published content differs slightly from the original reference content.
While this content will still be detected, the system will not report a
100% match. In these cases, users may examine the changes between
reference and detection in detail.
Figure 5 iPharro MediaSeeker Main Front-End
The reference content is shown in the upper
row, key frame by key frame, and compared to the detected content in the
lower row. Visual differences are highlighted using colored boxes.
Differences in the audio track are visualized as red and green bars above
the key frames and may be evaluated by playing both reference and detected
footage, side-by-side, through the user interface.
Figure 5 shows a screenshot from the iPharro
MediaSeeker main front-end. The iPharro MediaSeeker front-end is a
components-based graphical user interface that is customizable to satisfy
individual requirements. Functionality, such as user administration and
system monitoring may be added if required.
Web Front-end (Portal)
The portal is a web-based end-user interface to
the iPharro MediaSeeker system for offering on demand content detection as
a service. The portal targets customers with smaller amounts of reference
content, such as small to midsize advertising agencies, content owners, or
Clients can log in, upload their reference
content, and select the channels they wish to be monitored for a
designated time span. Detailed detection reports may be viewed online
at any time, and email notifications can be sent every time reference
content has been aired. In addition, the portal offers advanced
functionality such as RSS feeds, metadata access, download of
detection reports in Microsoft-Excel or XML format, and detailed clip
difference visualization, just like the GUI front-end
iPharro Media is on the
forefront of the digital asset management revolution, enabling
companies along every step of the media value chain to identify,
monetize and even protect video content.