INTERNATIONAL ORGANISATION FOR STANDARDISATION
ORGANISATION INTERNATIONALE DE NORMALISATION
ISO/IEC JTC1/SC29/WG11
CODING OF MOVING PICTURES AND AUDIO
ISO/IEC JTC1/SC29/WG11
MPEG2000/N3413
Geneva, CH
31 May – 2 June 2000
Title: / MPEG-7 Principal Concept List (0.9)Source: / MPEG-7 Multimedia Description Schemes (MDS) Group – Conceptual Modeling AHG
Authors: / John R. Smith (IBM T. J. Watson Research Center) on behalf of MPEG-7 Conceptual Modeling AHG
Status: / Approved
1 Introduction 2
2 Principal concept properties 2
3 References 3
4 Principal Concept List (V0.9) 5
1 Introduction
This document contains a list of principal concepts identified by the MPEG-7 Conceptual Modeling AHG ([1]) as being relevant to the development of MPEG-7. This list expands upon an earlier version given in [2]. Since July, 1999 ([14]) the Conceptual Modeling (CM) AHG has studied the principal concepts of MPEG-7 by:
(1) Examining MPEG sources of concepts such as the MPEG-7 Requirements [12] and MPEG-7 Applications documents [11]
(2) Examining MPEG-7 development [13] documents such as the various MPEG-7 XM and WD documents (in particular [4][5][6][7][8][9][10])
(3) Surveying proposals to MPEG-7 and tracking Core Experiments (i.e., [17][18][20][21])
(4) Monitoring discussions on MPEG-7 reflectors, and
(5) Surveying related multimedia research literature (including [22][23][24][25][31][32][36][37][38][39][40][41][42][43]).
This work is ongoing in that it provides a snapshot of the development of MPEG-7 and the MPEG-7 Principal Concept List shall be continuously revised and maintained to reflect ongoing progress.
2 Principal concept properties
The MPEG-7 Principal Concept List currently identifies and defines 183 principal concepts sorted alphabetically. Each principal concept entry gives the following information:
Property / Description /Principal Concept / Concept name
Definition / Definition in words
Model Construct / {Attribute, Entity, Function, Relationship, Type}
Domain / {Audio, Generic, Video}
Type / {Ancillary, Audio-visual data, Description, Feature, Language, Meta, Model, Process, Semantics, Structure, Syntax/ Semantics}
Source / {MPEG-7 Requirements, MPEG-7 Applications, MPEG-7 Generic DS (0.8), Proposal documents}
MPEG-7 MDS XM / {Y/N}
MPEG-7 MDS CE / {Y/N}
MPEG-7 MDS WD / {Y/N}
MPEG-7 Visual XM / {Y/N}
MPEG-7 Visual CE / {Y/N}
MPEG-7 Visual WD / {Y/N}
MPEG-7 Audio XM / {Y/N}
MPEG-7 Audio CE / {Y/N}
MPEG-7 Audio WD / {Y/N}
MPEG-7 Systems WD / {Y/N}
MPEG-7 DDL WD / {Y/N}
MPEG-7 Proposal / {Y/N}
MPEG-7 Development / {MDS XM, MDS CE, MDS WD, Visual XM, Visual CE, Visual WD, Audio XM, Audio CE, Audio WD, Systems WD, DDL WD}
MPEG-7 Construct (D or DS) / {D, DS}
MPEG-7 DDL Construct / {Element, Attribute, Type}
Description / Description of development as D or DS or systems
Related principal concepts / Related principal concepts
Related secondary concepts and terms / Related secondary concepts and terms
The list of principal concepts is given in Section 4.
3 References
[1] Ad Hoc Group on Conceptual Modeling, ISO/IEC JTC1/SC29/WG11 MPEG2000/N3268, Noordwijkerhout, NL, March 2000.
[2] MPEG-7 Principal Concept List (V0.7), ISO/IEC JTC1/SC29/WG11 MPEG2000/N3250, Noordwijkerhout, NL, March 2000.
[3] MPEG-7 Principal Concept List (V0.8), ISO/IEC JTC1/SC29/WG11 MPEG2000/M6020, Geneva, CH, May 2000.
[4] MPEG-7 Multimedia Description Schemes XM (Version 2.1), ISO/IEC JTC1/SC29/WG11 MPEG2000/M6007, Geneva, CH, May 2000.
[5] MPEG-7 Multimedia Description Schemes WD (Version 2.1), ISO/IEC JTC1/SC29/WG11 MPEG2000/M6008, Geneva, CH, May 2000.
[6] MPEG-7 visual part of XM 5.0, ISO/IEC JTC1/SC29/WG11 MPEG2000/N3321, Noordwijkerhout, NL, March 2000.
[7] Text of WD 2.0 of MPEG-7 Visual, ISO/IEC JTC1/SC29/WG11 MPEG2000/N3322, Noordwijkerhout, NL, March 2000.
[8] MPEG-7 Audio WD, ISO/IEC JTC1/SC29/WG11 MPEG2000/N3234, Noordwijkerhout, NL, March 2000.
[9] MPEG-7 DDL WD 2.0, ISO/IEC JTC1/SC29/WG11 MPEG2000/N3293, Noordwijkerhout, NL, March 2000.
[10] MPEG-7 Systems WD 0.3, ISO/IEC JTC1/SC29/WG11 MPEG2000/N3292, Noordwijkerhout, NL, March 2000.
[11] MPEG-7 Applications Document, ISO/IEC JTC1/SC29/WG11/N2860, MPEG99, Vancouver, BC, July 1999.
[12] MPEG-7 Requirements Document (V.10), ISO/IEC JTC1/SC29/WG11/N2996, MPEG99, Melbourne, Vic, October 1999.
[13] MPEG-7 Development Process, ISO/IEC JTC1/SC29/WG11/N3158, MPEG99, Maui, Hi, December 1999.
[14] Conceptual Modeling of MPEG-7 Description Schemes, ISO/IEC JTC1/SC29/WG11/M4775, MPEG99, Vancouver, BC July 99.
[15] Object-Based Scene Motion Description: Actions and Interactions, ISO/IEC JTC1/SC29/WG11 MPEG99/M4716
[16] Summary of MPEG-7 Audio Activities and Recommendations, ISO/IEC JTC1/SC29/WG11 MPEG99/N2677
[17] MPEG-7 Audio Core Experiment Methodology; MPEG-7 Description Schemes (V0.5) - Annex 1, ISO/IEC JTC1/SC29/WG11 MPEG99/N2796
[18] Description of Core Experiments for MPEG-7 Shape/Motion descriptors, ISO/IEC JTC1/SC29/WG11 MPEG99/N2818
[19] MPEG-7 Visual Part of XM Version 2, ISO/IEC JTC1/SC29/WG11 MPEG99/N2822
[20] Description of Core Experiments for MPEG-7 Color/Texture descriptors, ISO/IEC JTC1/SC29/WG11 MPEG99/N2819
[21] AHG on Core Experiments for Color/Texture Descriptors in MPEG-7, ISO/IEC JTC1/SC29/WG11 MPEG99/N2833
[22] A. B. Benitez, A. Jaimes, S.-F. Chang, J. R. Smith, C.-S. Li, “Fundamental Entity-Relation Models for the Generic Audio Visual DS”, ISO/IEC JTC1/SC29/WG11 MPEG99/M4754, Vancouver, Canada, July 1999.
[23] A. Jaimes and S.-F. Chang, “A conceptual Framework for Indexing Visual Information at Multiple Levels”, Submitted to IS&T/SPIE Internet Imaging 2000.
[24] U. Srinivasan, C. Lindley, and B. Simpson-Young, “A Multi-Model Framework for Video Information Systems”, Database Semantics - Semantic Issues in Multimedia Systems, Kluwer Academic Publishers, pp. 85-108, Jan. 1999.
[25] V. Oria, M. T. Ozsu, L. Liu, X. Li, J. Z. Li, Y. Niu, and P. J. Iglinski, “Modeling Images for Content-Based Queries: The DISIMA Approach”, Second International Conference on Visual Information Systems, pp. 339-346, San Diego, CA, Dec. 1997.
[26] G. Booch, Object-Oriented Analysis and Design with Applications, Second Edition, Benjamin/Cummings Publishing Co., 1994.
[27] C. Ghezzi, M. Jazayeri, D. Mandrioli, Fundamentals of Software Engineering, Prentice-Hall, 1991.
[28] T. J. Teory, D. Yang, J. P. Fry, “A Logical Design Methodology for Relational Databases Using the Entity-Relationship Model”, ACM Computing Surveys, Vol. 18, No. 2, June 1986.
[29] P. P.-S. Chen, “The Entity-Relationship Model – Towards a Unified View of Data”, ACM Trans. Database Systems, Vol. 1, No. 1, March, 1976, pp. 9 – 36.
[30] M. R. Blaha, W. J. Premerlani, J. E. Rumbaugh, “Relational Database Design Using an Object-Oriented Methodology”, Communications of ACM, Vol. 31, No. 4, April, 1988.
[31] M. E. S. Loomis, A. V. Shah, J. E. Rumbaugh, “An Object-Model Technique for Conceptual Design”, Proc. European Conf. on Object-Oriented Programming, June 15 – 17, 1987, Lecture Notes in Computer Science 276.
[32] Y. Lahlou, “Using an Object-Oriented Data Model as a Meta-Model for Information Retrieval”, First IEEE Metadata Conference, April 16-18, 1996
[33] R. Elmasri, S. B. Navanthe, Fundamentals of Database Systems, Benjamin.Cummings Publishing Co., New York, NY, 2nd Ed., 1994.
[34] A. Kemper, G. Moerkotte, Object-Oriented Database Management – Applications in Engineering and Computer Science, Prentice Hall, Englewood Cliffs, NJ, 1994.
[35] J. Martin, J. J. Odell, Object-Oriented Analysis and Design, Prentice Hall, Englewood Cliffs, NJ, 1992.
[36] D. Woelk, W. Kim, W. Luther, “An Object-Oriented Approach to Multimedia Databases”, ACM Proc. Of Conf. on Data Management, 1986, pp. 311 – 325,
[37] E. Oomoto, K. Tanaka, “OVID: Design and Implementation of a Video-Object Database System”, IEEE Trans. On Knowledge and Data Engineering, Vol. 5, No. 4, August, 1993, pp. 629 – 643.
[38] C. H. C. Leung, D. Hibler, N. Mwara, “Picture Retrieval by Content Description”, Journal of Information Science, 18, 1992, pp. 111 – 119.
[39] A. B. Benitez, A. Jaimes, S.-F. Chang, J. R. Smith, C.-S. Li, “Fundamental Entity-Relation Models for Generic Audio Visual DS”, ISO/IEC JTC1/SC29/WG11 m4754, Vancouver, Canada, July 1999.
[40] O. Lassila and R. R. Swick, “Resource Description Framework (RDF) Model and Syntax Specification”, W3C Recommendation REC-rdf-syntax-19990222, Feb. 22, 1999.
[41] J. Griffioen, R. Mehotra, R. Yavatkar, “An Object-Oriented Model for Image Information Representation”, Proc. ACM CIKM, Nov., 1993.
[42] A. Gupta, T. E. Weymouth, R. Jain, “Semantic Queries with Pictures: The VIMSYS Model”, 17th International Conference on Very Large Data Bases, Sept. 3-6, 1991, pp. 69-79.
[43] M. Davis, “Media Streams: An Iconic Language for Video Annotation”, Telektronikk 4.93: Cyberspace, Vol. 89, No. 4 (1993): 50-71.
4 Principal Concept List (V0.9)
(see below)
1
MPEG-7 Conceptual Model -- Principal Concepts List V0.9
Principal Concept / Definition / Model Construct / Domain / Type / Source / MPEG-7 Development / MPEG-7 Construct (D or DS) / MPEG-7 DDL Construct / Description / Related principal concepts / Related secondary concepts and terms /Abstraction level / The particular level of detail in which data is represented / Attribute / Generic / Audio-visual data / MPEG-7 Requirements / MDS XM / DS / Element / Multiresolution Pyramid DS (XM): specifies a hierarchy of views of data. Summarization DS (WD): is used to specify a set of summaries to enable rapid browsing, navigation, visualization and sonification of AV content. Each summary is an audio-visual abstract of the content. / Hierarchy, scalability
Acquisition / The process of acquiring audio or visual data from a source / Function / Generic / Process / MPEG-7 Applications / MDS WD / DS / Element / CreationMaterial (WD): describes the devices and instruments used for the creation of the content (e.g., types of device, lens, films, instruments, settings, etc.). / Instrument, camera, editing / Shooting, recording, source, filming, take
Action / A semantically identifiable behavior of an object or group of objects, e.g., soccer player kicking ball / Attribute / Visual / Semantics / MPEG-7 Description Schemes (V0.8) / MDS CE / DS / Element / Annotation DS (WD): contains the description tools (Ds and DSs) intended for a simple and structured description of persons, objects, events, etc; Semantic Relationships (XM): predicative semantic attributes refer to actions (events) or states among two or more elements. Examples of action relations are “To throw” and “To hit”. / Motion, object, event, animation / Interaction, action unit, interaction unit
Aggregation / Grouping of items such as objects, regions or audio-visual data / Relationship / Generic / Audio-visual data / MPEG-7 Description Schemes (V0.8) / MDS WD / DS / Element / Cluster DS (WD): describes the arbitrary grouping of audio-visual data items or syntactic elements. / Cluster / Set, collection, examples
Analytic model / Statistical model of a feature class or aggregation of descriptors / Entity / Generic / Model / MPEG-7 Description Schemes (V0.8) / MDS WD / DS / Element / Analytic Model DS (WD): describes feature classes and classifiers. Includes ClusterDS, ExamplesDS, ClassifierDS. / Probability model, classifier, cluster, aggregation / Examples, mean, variance
Animation / The motion and deformation of a 2D or 3D synthetic model or mesh / Attribute / Visual / Structure / MPEG-7 Requirements / Motion, deformation, parametric model, 3D model, mesh, motion / Movement, face animation (FAP) and body animation (MPEG-4)
Annotation / Textual meta data associated with audio-visual data / Attribute / Generic / Meta / MPEG-7 Description Schemes (V0.8) / MDS WD / DS / Element / Annotation DS (WD): contains the description tools (Ds and DSs) intended for a simple and structured description of persons, objects, events, etc / Meta information, text / Textual description, free text, language
Archive / A stored collection of audio-visual data / Entity / Generic / Meta / MPEG-7 Applications / MDS XM / DS / Element / Collection Structure DS (XM) / Cluster
Associated information / The non-audio and -video Information associated with the data / Entity / Generic / Meta / MPEG-7 Requirements / MDS WD / DS / Element / Related Material DS: contains the description tools (Ds and DSs) related to additional information about the AV content available in other materials.
Audio / Temporally-varying data or time signal intended for listening or hearing / Entity / Audio / Audio-visual data / MPEG-7 Requirements / Audio WD / DS / Element / Audio Segment DS: describes a segment of audio material. / Audio features, audio domain / Analog audio, digital audio
Audio domain / Semantic domain of audio material, e.g., music, speech, silence, special effects, soundtrack / Entity / Audio / Meta / MPEG-7 Requirements / MDS WD / DS / Element / Media Identification DS (WD): contains description tools (Ds and DSs) that are specific to the identification of the master media (the instances are described by the Media Instance DS). / Music, speech / Audio data classes, synthetic, symbolic
Audio features / Perceptual characteristics of audio data / Attribute / Audio / Feature / MPEG-7 Requirements / Audio WD / D / Element / Audio Sampled D (WD): all sampled audio descriptors are defined as subtypes of AudioSampledD / Audio, audio spectrum / Amplitude envelope, reverberation, sound surrounding perception, loudness
Audio object / An object that acts as a sound source for audio data / Entity / Audio / Semantics / MPEG-7 Requirements / MDS CE / DS / Element / Object DS (CE): represents a physical or abstract object that is present or is related to the multimedia document. Concept DS (CE): describes a template for objects or events. ConceptObject DS (CE): represents an object that participates in the definition of a concept. / Source, object / Surrounding, actual sound, atomic sound effect, commentative sound, bloop
Audio spectrum / The characteristics of the frequency content of audio data / Attribute / Audio / Structure / MPEG-7 Requirements / Audio WD / D / Element / Audio Independent Component DS (WD): describes the decomposition of a spectrogram into a collection of statistically independent spectral and temporal features. Audio Spectrum Envelope D (WD): describes the spectrum of the audio according to a logarithmic frequency scale. / Audio features / Frequency contour, frequency profile, fundamental frequency, spectral envelop, spectral shape, spectral tilt
Audio-visual data / The audio and visual data acquired from an audio-visual source / Entity / Generic / Audio-visual data / MPEG-7 Requirements / MDS WD / DS / Element / Media Information DS (WD): describes the format of the audio-visual data / Data, program, image, video, audio, audio-visual material
Audio-visual material / A product obtained from processing audio-visual data / Entity / Generic / Audio-visual data / MPEG-7 Requirements / MDS WD / DS / Element / Media Instance DS (WD): contains the description tools (Ds and DSs) that identify and locate the material instances whose description contains this Media Profile DS. / Program, image, video, audio, audio-visual data
Auralization / An aural summary of an audio program / Entity / Audio / Audio-visual data / MPEG-7 Description Schemes (V0.8) / MDS WD / DS / Element / Summarization DS (WD): describes a set of summaries to enable rapid browsing, navigation, visualization and sonification of AV content. Each summary is an audio-visual abstract of the content. / Summary, variation / Sonification, visualization, abstract