Project Title:
Advertisement Recognition from TV and Radio Broadcasts
Project Team Leader
Aljon Rey Aniban
Members:
Emmanuel Cagadas
Anna Mae Yap
Project Advisers:
Prospero Naval
Carlo Raquel
Significance of Project:
Advertising is a big part of the marketing industry. Advertisers sponsor both TV and radio programs. For companies, commercials are their medium to draw the people’s attention to their new products. These companies need to hire other companies just to verify that their commercials are really broadcasted.
At present, applications like this have already been developed in some well-developed countries. One example is Mediaguide.com which monitors music and advertising on over 2,700 college, non-commercial and commercial radio stations in 150 US markets; and over 3,500 internet stations in real-time, 24 hours per day, 7 days per week. They use the data & insight to provide business insight to record labels, broadcasters and advertisers; and to support a music discovery and advertising network that connects radio to digital outlets & mobile devices.
In the Philippines, most advertising companies monitor commercials manually. If we could automate commercial monitoring like in other countries, it will be a great help to Philippine advertising companies not only because computers perform more accurately compared to humans but also this companies will save a lot if they use Filipino made technology rather than availing it abroad.
Less Human Labor + Less Cost + Improve accuracy
Project Objectives:
· To determine the factors affecting accuracy
· To determine the factors affecting real-time implementability
· To determine how largely noise interruptions affect the performance of the system
· To determine the maximum number of advertisements that can be stored in the database while not affecting its real time performance.
RRL
· There are already existing commercialized applications on the internet that do advertisement monitoring both on the radio and television broadcasts. MediaGuide.com’s SeeSpotRun monitors advertisements broadcasted across the radio and internet radio.
· Audio-Based Radio and TV Broadcast Monitoring: The IBOPE Media radio and TV broadcast monitoring has already developed a scalable real-time audio fingerprinting system. The system extracts temporal feature from audio using the Short-time Fast Fourier Transform (STFT). When given an input stream to analyze, the system matches it against the database and automatically recognizes instances of the previously registered samples within the input stream. The algorithm exploits the temporal evolution of the signal frequency spectrum in order to identify patterns and produce the final classification. The database is clusterized in order to provide an efficient and scalable search strategy. The system has been assessed using a database containing 393 distinct commercials. A 1-hour audio stream from three different TV channels has been analysed in less than 3 hours, attaining a 95.4% recognition rate.
· A Highly Robust Audio Fingerprinting System: This system presents a highly robust fingerprint extraction method by extracting 32bit sub-fingerprint every 11.8ms via looking at energy differences along the frequency and time axes. It uses very efficient fingerprint search strategy, which enables searching a large fingerprint database with only limited computing resources.
· TV Advertisements Detection and Clustering based on Acoustic Information: This system detects individual commercials within a broadcast and groups together all repetitions of the same commercial over time. Detection is done in three steps, incrementally refining an initial course detection. First, the minimum energy points within the audio signal are found as hypothetical commercial start/end changes. Then validation of the candidates is performed by checking if there is an acoustic change at each point by acoustically comparing both sides for each candidate using the Bayesian Information Criterion (BIC) Algorithm. On step three, the proper selection of advertisements is made. To do so, first is necessary to find out precisely the boundaries of the connecting silences. This is done to eliminate the random amount of silence usually inserted between commercials. Afterwards, the distance between any two start-end marked point is compared with the set of allowed advertisement lengths, which for this study is 10, 20 and 30 seconds with a small error margin allowance. Clustering is later done over all previously detected commercials to find out how many times each commercial appears. Clustering uses three algorithms: Standard Dynamic Time Warping, a simplified DTW(DTW mod) algorithm and a Generalized Cross-Correlation comparison. This detection system achieves 82% precision and recall using only acoustic information.
Theoretical Framework:
The project will be having 2 major components: advertisement recognition module and user-interface module.
The advertisement recognition model would be divided into segmentation (detecting start and end points), windowing (dividing the audio file into narrow windows), extracting the feature vector coefficients (fingerprints) using Short-Time Fourier Transform, clustering of fingerprints in the database, and efficiently searching and matching of the queried audio file.
MySQL database would be used to store fingerprints. Java would be used to code this system.
Methodology
· Populate the database with pre-computed fingerprints of the commercial to be monitored
· Determine the start and end point of potential commercials in the audio input stream
· Save the commercial to the memory for matching purposes
· Segment the saved commercial into the appropriate length of frames
· Then perform fingerprint extraction on each frames
· Match the fingerprint to the database
· Determine the number of occurrences of each commercial
· Update the result
Major Activities/Work Plans
· Intensive research on signal processing
· Record a collection of audio input from TV and radios with commercials
· Database construction
· Implementation of methodology
Equipments Requirements
· TV tuner/Radio Tuner
· Java
· MySql
·
Literature Cited
1. A Comparison of Melodic Database Retrieval Techniques Using Sung Query, N. Hu and R. B. Dannenberg
2. A Highly Robust Audio Fingerprinting System, J. Haistma and T. Kalker
- A Similarity Measure for Automatic Audio Classification, J. Foote
4. An On-Line Algorithm for Real-Time Accompaniment, R. B. Dannenberg
5. Audio-Based Radio and TV Broadcast Monitoring, B. Oliveira, A. Crivellaro, and R. M. Ceasar Jr.
6. Automatic TV Advertisement Detection from MPEG Bitsream, D. A Sadlier, Dr. S. Marlow, Dr. N. O’Connor and Dr. N. Murphy
7. Automatic Recognition of Speech Sound by a Digital Computer, A. Iivonen
8. Automatic Music Monitoring and Boundary Detection for Broadcast using Audio Watermarking, T. Nakamura, R. Tachibana and S. Kobayashi
9. Automatic Audio Content Analysis, S. Pfeiffer, S. Fischer and W. Effelsberg
10. Finding Repeating Patterns In Acoustic Musical Signals: Applications for Audio Thumbnailing, J. Aucouturier and M. Sandler
11. Identification of Audio Titles on the Internet, H. Neuschmied, H. Mayer, and E. Batlle
12. Known-Audio Detection using Waveprint: Spectrogram Fingerprinting by Wavelength Hashing, M. Covell and S. Baluja
13. Mel Frequency Cepstral Coefficient for Music Modeling, B. Logan
14. Melody Matching Directly from Audio, D. Mazzoni and R. B. Dannenberg
15. Music Database Retrieval Based on Spectral Similarity, C. Yang
16. MUSART: Music Retrieval Via Aural Queries, W.P. Birmingham, R. B. Dannenberg, G.H. Wakefield, M. Bartsch, D. Bykowski, D. Mazzoni, C. Meek, M. Mellody, and W. Rand
17. On the Detection and Recognition of Television Commercials, R. Lienhart, C. Kuhmunch, and W. Effelsberg
18. Pattern Discovery Techniques for Music Audio, R.B Dannenberg and N. Hu
19. Robust Audio Hashing for Audio Identification, H. Ozer, B. Sankur, and N. Memon
- TV Advertisements Detection and Clustering based on Acoustic Information