Interactive advertising offers significant financial benefits to advertisers and broadcasters if they can get it to work! After communicating with Jeff Vinson from Audible Magic, he shared with IPTV Magazine that there are several key ways to use content recognition (CR) technology to improve and simplify interactive advertising. Advertisers look for better and more effective ways to advertise their products compared to other advertisers and being able to identify content to better match ads to viewer needs offers some major advertising advantages.

 Content Recognition (CR)

Content recognition can be performed by adding codes or unique data to the media (watermarking), by providing added data (such as simultaneous streaming channels) that identifies broadcast channels and media timing, or by identifying unique characteristics within the media (digital fingerprinting). 

Adding additional identification codes to program media requires the creation of codes and modification of media before or during distribution to the viewer. Content distribution agreements (television networks) may not allow the modification of program data stopping the watermarking option.

Providing additional channel information in a separate stream (such as through the Internet) can be an alternative to watermarking. Creating and distributing separate streaming information for multiple channels can be a complex process.

Content recognition that uses perceptual digital fingerprinting allows content to be identified without changes to the program media without the need to provide additional channel and timing information. All that is needed is a content analysis process and a database that can relate fingerprint information to a specific program.

Perceptual Fingerprinting

Perceptual fingerprinting uses fine-grained media characteristic measurements of the media content that can be compared to a reference database of fingerprints. This does not require modification to any of the broadcasted content 

which is important as broadcasters may require that content that is sent through their networks must remain unchanged. 

Using perceptual based advertising, smart devices can immediately control media display and interactive options based on program content or which ad being is being viewed regardless of its source or distribution channels.

For perceptual media to work, an analysis program must be installed in a smart viewing device. Smart devices are any media processing device such as a television, set top box, Blue-ray player, game consoles that can use small stored programs to analyze and identify characteristics of media programs they are playing. 

When a fingerprint is detected, a small window pops up at the corner of the screen display. It displays an image or text that provides an option to the viewer for interaction with the media program. This may be an offer for discount coupons or a prompt message that encourages the viewer to watch an additional media program that is related to the program or ad they are watching. When a user selects the offer with their TV remote, the current video can be squeezed back so the additional content can be seen. This information may be a helpful tip or it may be an offer such as a coupon or free sample

Figure 1 shows how an audio soundtrack can be analyzed to identify a unique set psycho-perceptual characteristics within a program audio track which is a digital fingerprint. This can be from programs that can be distributed on any broadcast or streaming system such as broadcast radio, Netflix, Roku, or DVR video. Devices can perform the same type of audio analysis and look up the characteristics it discovers in a reference database. When a match is found, the database information can be used to identify the program and select an appropriate ad to display to the viewer using content recognition

in advertising systems, advertisers are able to follow their audiences. When they do find an audience match, interactive events and offers can be made that have a very high viewer engagement rate. For example, during a televised Melissa Etheridge concert, nearly half the people viewing (46 percent) who saw an offer on their TV screens for a CDnow.com discount went to the website and clicked through to learn more. 

Advertisers can use content recognition systems to enable viewers to become part of an affinity program which rewards shoppers for viewing commercials. Each time they watch a commercial, their affinity card is credited with bonus viewing points. Their points can be displayed on the TV along with the brand icon as they watch the commercial increasing their perceived brand value. 

Fingerprint Matching Methods

The fingerprint matching system may analyze media and identify the programs (fingerprints) in different locations. Some of the fingerprint matching systems include cloud based fingerprinting, device based fingerprinting, and smart phone apps fingerprinting.

Cloud Based Fingerprinting

Cloud based fingerprint content recognition places the detailed analysis on a computer server that is connected to the Internet (located in the cloud). This allows it to work with many existing devices. Smart hardware such as connected TVs, Blu-Ray players, digital media adapters, such as Roku, can add software content recognition capability to recognize a program or advertisement that is played on the device. Cloud based fingerprinting can be an excellent solution for content recognition applications in devices that have with limited processing requirements (such as mobile devices). 

Using a cloud based fingerprint database enables the storing and sharing of a very large database that can contain millions of fingerprint references. They can be stored and processed by fast computer servers allowing for quick identification.

Figure 2 shows a connected viewing device can use a cloud database fingerprinting application to perform content recognition fingerprint matching. In this example, a connected TV has identified a digital audio fingerprint in a specific TV commercial sends it through the Internet where the fingerprint is identified. A coupon offer is sent to the display device which is displayed on the viewers screen. If the viewer selects to receive the coupon, it is sent to the device the viewer has setup to receive alerts and media (a mobile phone in this example).

Device Based Fingerprinting

Device based fingerprint content recognition places the analysis on a local device which uses a local database file to identify the program content. The internal cache of fingerprinting database records can be stored in devices such as a tablets or laptop computers. A lightweight (small size and processing requirements) application may be stored in the device which can quickly lookup fingerprints in the database without having to connect through the Internet.

 Figure 3 shows how a device can use an internal fingerprinting database to perform content recognition fingerprint matching. In this example, a fingerprint database is stored in the viewing device. The viewing device analyzes the media program to identify fingerprints. When it finds a fingerprint in its database that matches, it creates a small on-screen popup on the display with a coupon offer. If the viewer selects the offer using their remote, the information is sent to the merchant's web site allowing them to create the coupon and add it to the viewer's previously registered rewards card.

 Application Based Fingerprinting

Application based fingerprint content recognition places the fingerprint analysis on a smart device such as a mobile telephone that can run software applications (smart phones). 

The content identification process can be also combined with other information such as the location of the viewer to obtain ads that are better target to the viewer (localized ads). For example, if the viewer device detects a national commercial for a fast food chain, it can use the location information to create coupons for restaurants near the viewer's location.

Figure 4 shows a smart phone application that can use content recognition fingerprint matching. In this example, a smart phone application with audio fingerprinting detection capability has identified a digital audio fingerprint in a specific TV commercial. When the digital fingerprint is captured by a smart phone’s microphone and identified by the mobile phone app, a coupon can be created by the advertiser. The product coupon can be loaded into the smart phone’s memory in a storage locker. The consumer can check their coupon locker for special offers while shipping.