OData Extension for Temporal Data Version 4.0
Working Draft 01
20 28 November January 20143
Technical Committee:
OASIS Open Data Protocol (OData) TC
Chairs:
Barbara Hartel (), SAP AG
Ram Jeyaraman (), Microsoft
Editor:
Ralf Handl (), SAP AG
Hubert Heijkers (), IBM
Michael Pizzo (), Microsoft
Martin Zurmuehl (), SAP AG
Additional artifacts:
This prose specification is one component of a Work Product that consists of:
- OData Extension for Temporal Data Version 4.0 (this document)
- OData Temporal ABNF Construction Rules Version 4.0
- OData Temporal ABNF Test Cases
- OData Temporal Vocabulary
Related work:
This specification is related to:
- OData Version 4.0 Part 1: Protocol
- OData Version 4.0 Part 2: URL Conventions
- OData Version 4.0 Part 3: CSDL
- OData ABNF Construction Rules Version 4.0
- OData Core Vocabulary
- OData JSON Format Version 4.0
This specification replaces or supersedes:
- None
Declared XML namespaces:
- None
Abstract:
This specification defines how to represent and interact with temporal data using the Open Data Protocol (OData).
Status:
This Working Draft (WD) has been produced by one or more TC Members; it has not yet been voted on by the TC or approved as a Committee Draft (Committee Specification Draft or a Committee Note Draft). The OASIS document Approval Process begins officially with a TC vote to approve a WD as a Committee Draft. A TC may approve a Working Draft, revise it, and re-approve it any number of times as a Committee Draft.
Copyright © OASIS Open 2012. All Rights Reserved.
All capitalized terms in the following text have the meanings assigned to them in the OASIS Intellectual Property Rights Policy (the "OASIS IPR Policy"). The full Policy may be found at the OASIS website.
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published, and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this section are included on all such copies and derivative works. However, this document itself may not be modified in any way, including by removing the copyright notice or references to OASIS, except as needed for the purpose of developing any document or deliverable produced by an OASIS Technical Committee (in which case the rules applicable to copyrights, as set forth in the OASIS IPR Policy, must be followed) or as required to translate it into languages other than English.
The limited permissions granted above are perpetual and will not be revoked by OASIS or its successors or assigns.
This document and the information contained herein is provided on an "AS IS" basis and OASIS DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY OWNERSHIP RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
Table of Contents
1Introduction
1.1 Terminology
1.2 Normative References
1.3 Non-Normative References
1.4 Typographical Conventions
2Overview
2.1 Interacting with Temporal Data
2.1.1 Single Entities
2.1.2 Collections of Entities
2.1.3 Related Entities
2.1.4 Time as a Key Part
2.1.5 System Time
2.2 Definitions
2.2.1 Application Time
2.2.2 System Time
2.2.3 Bi-Temporal
2.2.4 Time-Travel Query
2.2.5 Time-Series Query
2.2.6 Temporal Entity
2.2.7 Time Slice
2.2.8 Time Series
2.3 Example Model
2.4 Example Data
2.5 Example Use Cases
3Temporal Query Options
3.1 Temporal Expression
3.2 Time-Travel Query Options
3.2.1 System Query Option $at
3.2.2 System Query Option $systemat
3.3 Time-Series Query Options
3.3.1 System Query Option $from
3.3.2 System Query Option $to
3.3.3 System Query Option $systemfrom
3.3.4 System Query Option $systemto
4Vocabulary for Temporal Entities
5Representing Temporal Entities
5.1 Time Slices
5.2 Time Series
6Temporal Requests
6.1 Querying Temporal Entities
6.1.1 Time-Travel Queries
6.1.2 Time-Series Queries
6.1.3 Interaction with Standard System Query Options
6.2 Modifying Temporal Entities
6.2.1 Create Entity
6.2.2 Update Entity
6.2.3 Delete Entity
6.2.4 Create Relation
6.2.5 Update Relation
6.2.6 Delete Relation
6.3 ETags
6.4 Actions and Functions
7REMOVE: F2F January 2013 Zürich Notes
7.1 Thursday
7.2 Wednesday
7.2.1 "Timeless" model supports point-in-time travel
7.2.2 Business-time-only model
7.2.3 System-time-only model
7.2.4 Bi-temporal model
7.2.5 Difference to previous approach
7.2.6 Experimental Syntax
7.2.7 Model differences
7.2.8 Further thoughts
8REMOVE: F2F July 2012 Redmond Presentation
8.1 Requirements
8.2 Open Questions, Issues, and Work Items
8.3 Additional Notes
9Conformance
Appendix A.Acknowledgments
Appendix B.Revision History
1Introduction
This specification adds the notion of time-dependency to the Open Data Protocol (OData) without changing any of the base principles of OData. It defines semantics and a representation for temporal data, especially:
- Semantics and operations for querying temporal data,
- Results format for queries containing temporal data,
- Vocabulary terms to annotate which data depends on time, and how.
1.1Terminology
The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in [RFC2119].
1.2Normative References
[OData-ABNF]OData ABNF Construction Rules Version 4.0.
See the link in "Related work" section on cover page.
[OData-Tmp-ABNF]OData Temporal ABNF Construction Rules Version 4.0.
See link in "Additional artifacts" section on cover page.
[OData-CSDL]OData Version 4.0 Part 3: CSDL.
See link in "Related work" section on cover page.
[OData-Protocol]OData Version 4.0 Part 1: Protocol.
See link in "Related work" section on cover page.
[OData-URL]OData Version 4.0 Part 2: URL Conventions.
See link in "Related work" section on cover page.
[OData-VocTemp]OData Temporal Vocabulary.
See link in "Additional artifacts" section on cover page.
[RFC2119]Bradner, S.,“Key words for use in RFCs to Indicate Requirement Levels”, BCP 14, RFC 2119, March 1997.
1.3Non-Normative References
[Fowler]Temporal Patterns, Martin Fowler,
[Kulkarni]Temporal Features in SQL standard, Krishna Kulkarni, May 13, 2011,
[Saracco]A matter of time: Temporal data management in DB2 10, IBM developerWorks, Cynthia M. Saracco, Matthias Nicola, Lenisha Gandhi, April 3, 2012,
[SQL:2011]ISO/IEC 9075-2:2011 Information technology - Database languages - SQL - Part 2: Foundation (SQL/Foundation).
1.4Typographical Conventions
Keywords defined by this specification use this monospaced font.
Normative source code uses this paragraph style.
Some sections of this specification are illustrated with non-normative examples.
Example 1: text describing an example uses this paragraph style
Non-normative examples use this paragraph style.
All examples in this document are non-normative and informative only.
All other text is normative unless otherwise labeled.
2Overview
When keeping track of time, the most important questions are:
- When did or will something happen?
- When did we learn that it happened?
This leads to two dimensions of time: application time, also called actual time, business time, or valid time, and system time, also called recording time or transaction time.
Keeping track of time is typically done by storing each fact together with the period of time for which this fact is valid, using separate periods for each time dimension, and the time periods are part of the logical key for “fact records”. See [SQL:2011] or [Kulkarni] on how this is done in the SQL standard.
A consumer's perspective on these facts can be different: even if a time dimension is tracked internally, it may or may not be visible in a consumer's perspective, and even if visible it is often not considered part of the logical entity key.For example an employee is still the same person even after switching to another department.
The goals of this extension are:
- Keep the models as simple as possibleby allowing to hide time, and
- Provide easy means for time-travelandtime-series queries even if time is hidden.
More sophisticated queries are still possible in models that make time visible, even if time is not made part of the entity keys.
2.1Interacting with Temporal Data
2.1.1Single Entities
Assume a simple model: employees will work in different roles and in different departments during their career, and sometimes they even change their name. So employees are assigned an immutable ID when first entering the company and all employee-related information will be recorded with a validity period attached:
Example 2: possible temporal database table foremployee data using the validity start date as part of the database key
If an employee gets a different job title, this information can already be entered into the system and stored as a new record:
Example 3: employee datastored with from-inclusive, to-exclusive validity period
Note that this also allows entering planned changes ahead of time, using a future validity start date.
For consumers only interested in the information as of now, the natural OData model uses the employee ID as the entity key and either omitsthe validity start and end dates, or represents them only as dependent information:
Example 4: OData model with time-independent entity key
The OData service will expose only the information from the one record valid at the time the request is processed.
Example 5: request processed before 2013-10-01
GET ~/Employees('E314')
results in
{
"@odata.context": "$metadata#Employees/$entity",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior"
}
Example 6: request processed at or after 2013-10-01
GET ~/Employees('E314')
results in
{
"@odata.context": "$metadata#Employees/$entity",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior"
}
Clients that are aware of the time-tracking capabilities underlying this seemingly timeless model may be interested in the information at a certain point in time. These time-traveling clients use the system query option $at to request data at a specific point in time:
Example 7: rRetrieve employee in the past
GET ~/Employees('E314')?$at=2012-01-01
results in
{
"@odata.context": "$metadata#Employees/$entity/$timeslice",
"@Temporal.From": "2011-01-01",
"@Temporal.To": "2013-10-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior"
}
Example 8: rRetrieve employee in the future
GET ~/Employees('E314')?$at=2025-01-01
results in
{
"@odata.context": "$metadata#Employees/$entity/$timeslice",
"@Temporal.From": "2013-10-01",
"@Temporal.To": null,
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior"
}
The returned results are called time slices, and the validity period of each time slice is automatically included in the response. A null value for the period start means the beginning of time, and a null value for the period end means the end of time. Time-period boundaries that are not declared properties of the requested entities are represented as instance annotations with terms from a vocabulary for temporal entities.
This is necessary for the next type of question time-tracking aware clients will want to ask: how did the information change over time:
Example 9: rRetrieve employee in a time period
GET ~/Employees('E314')?$from=2012-01-01&$to=2025-01-01
results in
{
"@odata.context": "$metadata#Employees/$entity/$timeseries",
"value":[
{
"@Temporal.From": "2011-01-01",
"@Temporal.To": "2013-10-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior"
},
{
"@Temporal.From": "2013-10-01",
"@Temporal.To": null,
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior"
}
]
}
This returns all time slices whose validity period overlaps with the requested time period.
2.1.2Collections of Entities
These temporal query options also work with resource paths identifying collections of entities. Requesting data at a specific point in time results in a collection of time slices:
Example 10: rRetrieve multiple employee in the past
GET ~/Employees?$filter=contains(Name,Jobtitle eq 'Junior')&$at=2012-01-01
results in one timeslice for each employee matching the filter at the specified point in time
{
"@odata.context":"$metadata#Employees/$timeslice",
"value": [
{
"@Temporal.From": "2011-01-01",
"@Temporal.To": "2013-10-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior"
},
...
]
}
Requesting data for a period of time results in a collection of time series, one for each entity. The time series for each entity is itself a collection of time slices for that entity:
Example 11: rRetrieve multiple employee in the past
GET ~/Employees?$filter=Jobtitle contains(Name,eq 'Junior')&$from=2012-01-01&$to=2025-01-01
results in one time series for each employee matching the filter at some point within the specified period, and returns the time slices within that series that match the filter. So the value is an array of arrays:
{
"@odata.context": "$metadata#Employees/$timeseries",
"value": [
[
{
"@Temporal.From": "2011-01-01",
"@Temporal.To": "2013-10-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior"
},
{
"@Temporal.From": "2013-10-01",
"@Temporal.To": null,
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior"
}
],
[
{
"@Temporal.From": "2012-03-01",
"@Temporal.To": null,
"ID": "E314E401",
"Name": "Gibson",
"Jobtitle": "Expert"
}
]
...
]
}
The interpretation of non-temporal system query options in combination with $at is straight-forward: the request is simply interpreted at this point in time.
The combination with $from and $to is less straight-forward and needs to be defined per standard system query option: a $filter is considered matching if the validity period of the time slice partially or fully overlaps with the request period and filter condition is true for the property values of the time slice at any point in time during the requested period, and only those time slices during that period matching the filter are part of the result.
For $orderby the last value in the resulting time series (after filtering) is used, assuming that typically the last value is of most interest. For details see section 6.1.3.
2.1.3Related Entities
Each employee is assigned to a department, and this assignment can change over time, as can properties of the department. Both types of changes are tracked, and again this need not be visible in the model.
Example 12: OData model with time-independent entity key and navigation
If the assigned department is internally stored per employee record, this can influence the visible representation; see next example. Reassigning an employee to a different department without changing any of the visible employee properties may result in time slices that do not differ other than in their time-period boundaries, see Example 14. Services may or may not “compress” the returned time series by joining adjacent time slices that do not differ in the selected properties, so the service could also have returned the response of Example 9. This can also happen if only a subset of the properties is requested using the system query option $select.
Example 13: employee data stored with foreign key to department
Example 14: rRetrieve employee in a time period during which a relationship changed
GET ~/Employees('McDevitt')?$from=2012-01-01&$to=2025-01-01
results in
{
"@odata.context": "$metadata#Employees/$entity/$timeseries",
"value": [
{
"@Temporal.From": "2011-01-01",
"@Temporal.To": "2013-10-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior"
},
{
"@Temporal.From": "2013-10-01",
"@Temporal.To": "2014-01-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior"
},
{
"@Temporal.From": "2014-01-01",
"@Temporal.To": null,
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior"
}
]
}
Expanding related entities in combination with $at is straight-forward: nested within the time slice of a base entity are those time slices of related entities that are valid at the requested point in time:
Example 15: rRetrieve employee in the past with expanded department
GET ~/Employees('E314')?$at=2012-01-01&$expand=Department
results in
{
"@odata.context": "$metadata#Employees/$entity/$timeslice",
"@Temporal.From": "2011-01-01",
"@Temporal.To": "2013-10-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior",
"Department": {
"@Temporal.From": "2010-01-01",
"@Temporal.To": "2012-06-01",
"ID": "D08",
"Name": "Support"
}
}
Example 16: rRetrieve department in the future with expanded employees
GET ~/Departments('D15')?$at=2025-01-01&$expand=Employees
results in
{
"@odata.context": "$metadata#Departments/$entity/$timeslice",
"@Temporal.From": "2010-01-01",
"@Temporal.To": null,
"ID": "D15",
"Name": "Services",
"Employees": [
{
"@Temporal.From": "2014-01-01",
"@Temporal.To": null,
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior"
},
{
"@Temporal.From": "2012-03-01",
"@Temporal.To": null,
"ID": "E401",
"Name": "Gibson",
"Jobtitle": "Expert"
},
...
]
}
The time-slice boundaries of the nested entities reflect the actual validity of the related entities and are independent of the time-slice boundaries of the base entity.
Expanding related entities in combination with $from and $toadds the challenge that for single-valued navigation properties the related entity needs to be represented as a time series, i.e. in JSON as an array instead of as an object.
Example 17: rRetrieve employee in a time period and expand the department
GET ~/Employees('McDevitt')?$from=2012-01-01&$to=2025-01-01&$expand=Department
results in
- An employee time slice with an array of expanded departments because the department (name) changed during the validity period of the employee time slice. Note that the validity periods of the department time slices completely cover the validity period of the employee time slice, but are not truncated to match the employee time slice.
- An employee time slice with a different job title. The expanded department time slice is a copy of the last time slice in the expanded department of the preceding employee time slice. If the request had specified a preference of odata.allow-entityreferences (see [OData-Protocol, section 8.2.8.1], this could have been expressed as an entity reference containing annotations for the validity period.[RH1]
- An employee time slice with a different expanded department.
{
"@odata.context": "$metadata#Employees/$entity/$timeseries",
"value": [
{
"@Temporal.From": "2011-01-01",
"@Temporal.To": "2013-10-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Junior",
"ext": "$metadata#Departments/$entity/$timeseries",[RH2]
"Department": [
{
"@Temporal.From": "2010-01-01",
"@Temporal.To": "2012-06-01",
"ID": "D08",
"Name": "Support"
},
{
"@Temporal.From": "2012-06-01",
"@Temporal.To": null,
"ID": "D08",
"Name": "1st Level Support"
}
]
},
{
"@Temporal.From": "2013-10-01",
"@Temporal.To": "2014-01-01",
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior",
"Department": [
{
"@Temporal.From": "2012-06-01",
"@Temporal.To": null,
"ID": "D08",
"Name": "1st Level Support"
}
]
},
{
"@Temporal.From": "2014-01-01",
"@Temporal.To": null,
"ID": "E314",
"Name": "McDevitt",
"Jobtitle": "Senior",
"Department": [
{
"@Temporal.From": "2010-01-01",
"@Temporal.To": null,
"ID": "D15",
"Name": "Services"
}
]
}
]
}
Describe the result:
-Department name changes
-Job title changes – same department, time-slice is repeated → could send entity/time-slice reference (separate example)
-Department changes
-Nested time-series always covers validity period of "outer" time-slice
-Nested validity periods not truncated to match "outer"
2.1.4Time as a Key Part
For some entities it may be desirable to make their time-dependency visible in the model, e.g. the budget allocated to a department in a certain period of time.