P.O. Box 140277
Irving, Texas 75014-0277
Fax: (888) 522-7313
GENTECH
Genealogical Data Model Phase 1
A Comprehensive Data Model for Genealogical Research and Analysis
May 29, 2000
Page 1
Time Entry System (TES) Requirements Draft 11/04/18
Title PageTitle:GENTECH Genealogical Data Model
Data Model Version:1.1
Document Date:May 29, 2000
Lexicon Phase 1 Participants
The members of the Phase 1 Lexicon Working Group include the following individuals.
Principal Members:Robert Charles Anderson
Paul Barkley
Robert Booth
Birdie Holsclaw
Robert Velke
John Vincent Wylie
Additional Contributors:Helen F. M. Leary
Beau Sharbrough
Table of Contents1.0 PROJECT OVERVIEW
1.1 The Origin and History of the Project
1.2 Sponsors of the Data Model
1.3 The Character of the Data Model
1.3.1 Fundamental Principles of the Data Model
1.3.2 Points to Note About the Data Model
1.3.3 Following the Data Model
1.3.4 Extensions to the Data Model
1.4 Frequently Asked Questions (FAQ)
1.4.1 Is this data model a replacement for GEDCOM?
1.4.2 Where are the GEDCOM tags?
1.4.3 Where is the citation, and where is the family?
1.4.4 Why is the model so complicated?
1.4.5 Could such a tedious model ever be used to build a real application?
1.4.6 What limitations does the model have because of computer technology?
1.4.7 How are primary and secondary sources handled in this data model?
1.4.8 Will GENTECH certify software that complies with this data model?
2.0 THE GENEALOGICAL RESEARCH PROCESS FLOW
3.0 THE UNIFIED THEORY OF GENEALOGICAL DATA
3.1 The Original Statement Types
3.1.1 Statement Type 1: Statements About Relationships
3.1.2 Statement Type 2: Statements About Events
3.1.3 Statement Type 3: Statements About Characteristics
3.2 Statement Types 1 to 3 Compared
3.3 The Super Statement Type
4.0 SCOPE AND REQUIREMENTS
4.1 SCOPE
4.2 REQUIREMENTS
4.2.1 Basic Research Requirements
4.2.2 Person Name Requirements
4.2.3 Place Name Requirements
4.2.4 Date Requirements
4.2.5 Attribution and Administrative Requirements
5.0 GENTECH GENEALOGICAL DATA MODEL
5.1 THE ENTIRE GENEALOGICAL DATA MODEL
5.1.1 Naming Conventions
5.1.2 Connectors
5.2 ADMINISTRATION SUBMODEL
5.3 EVIDENCE SUBMODEL
5.4 CONCLUSIONAL SUBMODEL
5.4.1 The General Concept of an ASSERTION
5.4.2 The Four Subject Types in ASSERTION
5.4.3 Characteristic Entities
5.4.4 Event Entities
5.4.5 Group Entities
5.4.6 Persona Entities
5.4.7 Place Entities
6.0 DATA DEFINITIONS
6.1 ACTIVITY
6.2 ADMINISTRATIVE-TASK
6.3 ASSERTION
6.4 ASSERTION-ASSERTION
6.5 CHARACTERISTIC
6.6 CHARACTERISTIC-PART
6.7 CHARACTERISTIC-PART-TYPE
6.8 CITATION-PART
6.9 CITATION-PART-TYPE
6.10 EVENT
6.11 EVENT-TYPE
6.12 EVENT-TYPE-ROLE
6.13 GROUP
6.14 GROUP-TYPE
6.15 GROUP-TYPE-ROLE
6.16 PERSONA
6.17 PLACE
6.18 PLACE-PART
6.19 PLACE-PART-TYPE
6.20 PROJECT
6.21 REPOSITORY
6.22 REPOSITORY-SOURCE
6.23 REPRESENTATION
6.24 REPRESENTATION-TYPE
6.25 RESEARCH-OBJECTIVE
6.26 RESEARCH-OBJECTIVE-ACTIVITY
6.27 RESEARCHER
6.28 RESEARCHER-PROJECT
6.29 SEARCH
6.30 SOURCE
6.31 SOURCE-GROUP
6.32 SOURCE-GROUP-SOURCE
6.33 SURETY-SCHEME
6.34 SURETY-SCHEME-PART
APPENDIX A: PRINCIPLES OF LOGICAL DATA MODELING
A.1 DATA MODELING AND THE RELATIONAL MODEL
A.2 THE RULES OF NORMALIZATION
A.2.1 First Normal Form: Eliminate Repeating Groups
A.2.2 Second Normal Form: Eliminate Redundant Data
A.2.3 Third Normal Form: Eliminate Columns Not Dependent on Key
A.2.4 Fourth Normal Form: Isolate Independent Multiple Relationships
A.2.5 Fifth Normal Form: Isolate Semantically Related Multiple Relationships
A.3 THE ENTITY RELATIONSHIP DIAGRAM
A.3.1 The Entity and the Attributes
A.3.2 Choosing Keys
A.3.3 Relationships
A.3.4 Entity and Attribute Definition
APPENDIX B: LOGICAL VIEWS OF THE DATA MODEL
B.1 RESEARCH PLAN AND TASK LIST
B.2 RESEARCH LOG
B.3 CITATION
APPENDIX C: DATA MODEL CONNECTIONS FOR EXPERT SYSTEMS
C.1 PERSON NAME EXPERT SYSTEM
C.2 PLACE NAME EXPERT SYSTEM
C.3 DATE EXPERT SYSTEM
C.4 EVENT EXPERT SYSTEM
C.5 OTHER EXPERT SYSTEMS
APPENDIX D: ADDITIONAL STATEMENT TYPES
D.1 Statement Type 4: Statements About Sequence
D.2 Statement Type 5: Statements About Rank In A Group
D.3 The Partially Combined Statement Type
GENTECH Genealogical Data ModelMay 29, 2000Page 1
GENTECH Genealogical Data Model
A Comprehensive Data Model for Genealogical Research and Analysis
1.0 PROJECT OVERVIEW
1.1 The Origin and History of the Project
The GENTECH Data Modeling Project is an extension of the work done by GENTECH members on the Lexicon Project, an attempt to define genealogical data for the purpose of facilitating data exchange among genealogists. After some work on the Lexicon, the group recognized that it is difficult to define genealogical data out of context because of the various ways people interpret common genealogical terms. The group decided that the effort would be better served by defining genealogical data in the context of a logical data model, which is a systems engineering methodology used to define data in an automated data processing system.
It is important to recognize, however, that the group is simply using this methodology to define genealogical data; the group is not designing software. The Lexicon group was careful to make sure the model has not been shaped or influenced by the limitations of current software and hardware. We used data modeling as a means to define genealogical data and the relationships between that data in an effort to bring greater understanding to the genealogical community about data issues. While this does not rule out software developers using the model to create new generations of genealogical software—and in fact the Lexicon group would be delighted if that happens—that was not our goal. As a practical matter, we expect this explicit definition of genealogical data to foster discussion of genealogical data and perhaps in the future to help the genealogical community better exchange data by understanding the limitations of various subsets of genealogical data that may be implemented in automated or manual systems.
APPENDIX A: PRINCIPLES OF LOGICAL DATA MODELING (page 80) contains a discussion of data modeling concepts for readers who may not be familiar with the terminology used in entity relationship diagrams. If you have never worked with data models, you may find it useful to read that section in order to understand both the terminology used in the model, and some of the basic principles that underlie the organizational structure that we used to prevent redundancy, among other things.
It is important to note that we created a logical data model, and not a physical data model. The logical model describes the relationships of genealogical data, but when that model is actually implemented by a developer, the developer may choose to alter the model using certain methodologically accurate transformations to create the physical data model. Typically, transformations are used to reduce the complexity of the code that must be written, or to increase the performance of the computer system. Clearly, since the purpose of the Lexicon group is to define data and not to build an actual system, the logical data model is the appropriate construct.
The group met in Rochester, New York for two days in August 1996 with a facilitator in an intensive working environment called a Joint Application Development (JAD) session, normally intended to bring subject matter experts and developers together. In this case, the developers present were primarily there to act as further subject matter experts, and to facilitate the group’s understanding of the data modeling methodology.
During this JAD session, however, it became apparent that although the group was not creating a data model as part of the specifications for a real, to-be-built, specific genealogical application, certain parts of the model could not be created without some understanding of how the data might actually be used in a real application. Thus, the group reluctantly agreed to write a few requirements so that those who study the model can understand the underlying direction. Further, this document attempts to capture some, but not all, of the reasoning behind various portions of the data model. We were hampered by not having a “recorder”, the person in JAD sessions responsible for continuously transcribing the results of the discussion. Because this was a volunteer effort and spanned several years, we did not feel that we could afford to have a person in this role; the facilitator brought back the flip charts from each session and transcribed those into this document, subject to group review.
The Lexicon Group met again for two days in January 1997 in Plano, Texas to continue work on the model, and a short meeting was called in May in Valley Forge, Pennsylvania to review our progress. In September 1997 the group met in Dallas, Texas for two days and brought the initial draft data model to closure, although a follow up meeting in Denver in May 1998 was required to complete the data definitions. At that time, it was apparent that the group not only would not finish the data definitions, but had some issues with the current draft as well. A final meeting was held in Silver Spring, Maryland in June 1998. This paper reflects the thinking of the group through that period. It is expected that the data model will be revisited after public comments are received; this document is the formal Request for Comments (RFC).
The statement that best characterized the iterative process that the group went through was finally articulated in Silver Spring: “Now that I look at it, I don’t like it.” The group continually revisited data model sections to test new ideas against previously agreed upon constructs. The result was frustrating at times as old work was re-opened, but the result was to continually refine the model, making it more general, more powerful, and unfortunately somewhat more abstract than the original concepts. We believe that this is an extremely powerful data model that will accommodate a wide variety of genealogical data, but as a reader of this document, you should compare the model against your own understanding of genealogical data and attempt to find places where data cannot be accommodated by the model.
In order to understand the logical genealogical data model and compare it against your experience, however, it’s necessary to not only understand the data modeling terms from systems engineering that are defined in Appendix A (page 80 as previously mentioned), it’s also critical to understand the genealogical research process as the members of the group understand it. A process flow diagram and a discussion of this are presented in Section 2.0 THE GENEALOGICAL RESEARCH PROCESS FLOW on page 11.
When all comments have been evaluated, it is the intention of the group to disband and to encourage the formation of a Lexicon II group to use the data model to define genealogical terms.
1.2 Sponsors of the Data Model
Although the Lexicon Project began as an initiative of GENTECH, other national genealogical organizations were instrumental in providing support for this project as the data model evolved. Those organizations, in the order that they were able to join with GENTECH in this project, are the following.
- GENTECH (Charter sponsor)
- Federation of Genealogical Societies (FGS) (Charter sponsor)
- New England Historic Genealogical Society (NEHGS)
- National Genealogical Society (NGS)
- American Society of Genealogists (ASG)
- The Association of Professional Genealogists (APG)
- The Board for Certification of Genealogists (BCG)
These societies are currently sponsoring the genealogical data modeling process and not necessarily the product.
1.3 The Character of the Data Model
1.3.1 Fundamental Principles of the Data Model
The intention of the group was to create a data model that would support the following four principles.
- The purpose of the genealogical data model is to support the genealogical research process.
- There is one and only one place to put each piece of data, and there exists a place for every piece of genealogical data.
- Some researchers will not produce all the data that rigorous pursuit of the process will produce.
- Actual software systems based on the data model should teach, encourage, remind, and assist users to follow the research process to create high quality genealogical research that can be communicated to others.
1.3.2 Points to Note About the Data Model
The four brief statements in the previous section can be expanded to the following points about the data model.
- The data model expresses our understanding, where possible, of all genealogical data, and it attempts to be completely comprehensive and all inclusive.
- The data model is intended to facilitate the understanding of data issues in the genealogical community, and although the model itself has been created using a systems engineering methodology, the model was not designed to be the platform for a particular piece of software.
- The data model is extensible from our current understanding of genealogical data by putting most kinds of data into tables where rows could be added for additional types of genealogical data that were not considered in the original model. As little as possible is “hard coded” in the model. Thus the model, by being data driven where possible, will accommodate data that we have not considered, but which is of a type that we already understand.
- The model should in no way require the genealogical researcher to force data into inappropriate fields simply because the data modelers failed to allow for unusual data. Where the Lexicon group has failed to identify an entire type of data, the model will be extended.
- The data model eventually may support the creation of genealogical software, but the model exists independently of any implementations and does not constrain future developers in their choice of language or hardware, other than to suggest that the relational model is a reasonable construct from which to understand the data.
- The model encourages and supports storing the reasoning behind the genealogical conclusions reached, along with all the evidence that led to those conclusions.
- If a genealogical conclusion is later disproved, the model allows the researcher to correct the conclusion by making a correcting entry, not just purging the originally incorrect conclusion, although it does not force the researcher to correct if they’d rather purge.
- The data model supports both the professional level researcher and the novice by allowing the novice to enter conclusional data without evidence as is currently the widespread practice in genealogical software. Although this is not specifically shown in the data model, the intention is that an actual implementation of this data model would simply fill in place holder records in the intervening entities as needed, in lieu of the novice actually entering the evidence that a more sophisticated user would enter. However, since the model is designed to strongly support evidence, it is anticipated that a sophisticated user interface in an actual software application would strongly encourage the user to enter the evidence, even if entering that data is optional.
- The data model prevents the mixing of other people’s data indiscriminately with the researcher’s own data. While the model certainly supports the importation of electronic data as it does bringing in more traditional sources, the model also supports the concept of attribution so that no data appears without an audit trail indicating its origin.
- The data model, at a macro level, supports the flow of data from evidence to conclusions through a process of analysis and transformation, and further, supports the continuing use of preliminary conclusions to build more advanced conclusions.
The group deliberately avoided creating lists of legal values for the various entities and attributes, such as surety values, standard repository lists and abbreviations, research objective keywords, and so forth because we felt that trying to set data value standards was beyond the scope of this phase of the Lexicon project.
1.3.3 Following the Data Model
As previously stated, it is our hope that the data model will serve to increase understanding of data issues throughout the genealogical community. Assuming that we have correctly modeled genealogical data, we expect the following.
- The core of the data model will be followed. This means that no implementation of the data model will compromise the current entities, attributes, and relationships.
- The data model will be extended by individuals and companies. This can be done by the following means.
- By adding attributes to existing entities.
- By adding entities to the data model.
- By adding relationships to the data model, particularly to the new entities.
- The core of the data model will not be compromised by extensions, such as removing attributes or entities, or by changing or removing relationships.
1.3.4 Extensions to the Data Model
In addition to the extensions to expert systems discussed in APPENDIX C: DATA MODEL CONNECTIONS FOR EXPERT SYSTEMS on page 94, the group carefully removed a number of entities and relationships after agreeing that they were not needed in the core model. The following are some logical extensions to the model.
- Research objectives can be controlled against a RESEARCH-OBJECTIVE-KEYWORD lookup table.
- Research objectives can be placed in a hierarchical structure so that some high level research objectives have two or more lower level objectives.
- People who appear in citations such as authors, editors, and compilers can be linked to people in the PERSONA entity so that searches that return people who are the subjects of assertions can also return those same people who are involved as authors or in other roles in SOURCEs.
- Both SEARCH and ADMIN –TASK could have attributes for Cost and Time to track expenses and effort. While this would be of interest to professional genealogists who bill for their services, it might be of interest to others as well.
1.4 Frequently Asked Questions (FAQ)
Before we discuss the genealogical research process and the comprehensive genealogical data model that supports the rigorous research process, we address several frequently asked questions (FAQ).