Working Document for Help Desk Items
Summary of Help Desk as of 11/29/2015
http://osgug.ucaiug.org/HelpDesk/Lists/servicerequests/GreenButton.aspx
[56] ESPI standard doesn't allow negative values [Tom Kring] 3
[57] Use of UUIDs for UsagePoint should be persistant across lifetime of association 3
[58] Guidelines are needed for proper usage of hrefs in the Atom representation 4
[60] Green Button file names contains PII 7
[61] ESPI lacks clarity for sending OAuth authorizations in batch 7
[62] Timestamps are Universal so where is timezone and DST info 8
[64] ESPI ElectricPowerUsageSummary needs usage total to go with last period 12
[65] How many Interval blocks should there be per entry 12
[66] Meaning of UsageSummary may need (direction) clarification 12
[67] Extend costAdditionalLastPeriod with optional details 12
[68] Add ESPI Use Case for Green Button data file download [Anne Hendry] 13
[69] CurrencyCode is UINT8 but needs to be UINT16 14
[70] Incorporate 61968-9 enumerations into schema 14
[71] Add tariff profile and service delivery point as optional addition to usage point 15
[72] Remark about test # FDFTP6 (related links matching self links) 15
[73] Green Button test for stability of UUIDs needed 15
[74] Add additional UsageSummary structures for Gas and Water 16
[75] Logical Information Model components should be part of model 16
[76] CSWG REQ 21 review - NISTIR 7628 References 16
[77] CSWG REQ 21 review - RFC 5849 OAuth 1.0 17
[78] CSWG REQ 21 review – TLS 17
[79] CSWG REQ 21 review – References 17
[80] DateTimeInterval should make attributes optional 17
[81] Modify UsagePoint/RoleFlags to match SEP2 18
[82] How to communicate ‘greenness’ of Energy During an Interval (Scott.Crowder) 18
[83] Add a PII element that is optional (jim.vezina, NREL) 19
[84] Scope Selection page on Data Custodian site vs 3rd Party Site 20
[85] Time of Use tier indicator alignment with SEP 2.0 20
[86] Proposal for Green Button Security/Privacy Enhancement 21
[88] Change title of ElectricPowerUsageSummary to UsageSummary and extend 21
[89] ESPI Schema extension: Revenue Quality value for Quality of Reading Flag 23
[90] Error in GB TestPlan: FB_07,08 AccumulcationBehavior should be 4 23
[91] Error in Test Plan: FB_04 23
[92] Need to include <link> reference to Certification ID 24
[93] Customers have requested access to the Green Button CMD API interface to access their own account information 24
[94] Inclusion of Read Cycle to Usage Point and Usage Summary 24
[95] Green Button Download My Data Gas Function Block [FB_10] only allows "therms" UOMType = 169 25
[96] Green Button Download My Data Gas Function Block [FB_11] only allows UOMType = 128 (US gl (US Gallons) 25
[97] Update UsagePoint for 61968-9 2nd edition elements 25
[xx] title 26
[56] ESPI standard doesn't allow negative values [Tom Kring]
We have run into a problem with our Green Button implementation that has led to a concern about one aspect of the ESPI standard. The <value> element, child of the <IntervalReading> element, is defined as Uint48 – an unsigned integer value.
With our current implementation, this will be a problem for net meters, since we are only outputting the net data, which could contain negative values. We could fix our implementation to output the delivered and received data separately - as long that’s what we receive from the meter. There are still meters out there that only record the net usage. Also, we’re wondering if there may be other use cases where the values may be either positive or negative – temperature data, for example.
The CIM Part 9 equivalent of this element defines the value as a string.
Comments:
steve.van ausdall (2/2/2012 4:27 PM): Should also change SummaryMeasurement.value to Int48.
steve.van ausdall (1/3/2012 5:57 PM): Suggest defining and changing type to Int48, derived from xs:long, Signed integer, max inclusive 140737488355328 (2^47), min inclusive -140737488355328
Marty: same argument applies to cost values …
Recommendation of OpenADE Task Force
Change data type of SummaryMeasurement.value, IntervalReading.value, IntervalReading.cost to Int48. [NAESB]
[57] Use of UUIDs for UsagePoint should be persistant across lifetime of association
UUIDs for UsagePoints should be recommended to be persistent so that they can be associated with actual UsagePoints in the field and updates can be recognized.
06/12: Anne: Usage point versus meter, etc.
No changes.
Comments:
Recommendation of OpenADE Task Force
Make UUID for UsagePoint retention required in Green Button test plans. The UUIDs should be constant over the lifetime of the association between the Retail Customer, the Data Custodian, and the Third Party. The UUIDs should be unique for each three-way association (or two-way association for DMD) and not reused. [NAESB]
[58] Guidelines are needed for proper usage of hrefs in the Atom representation
At the Grid-Interop 2011, we discussed the use of atom:link elements with rel="up" in the Green Button file format. These links are meant to point from an atom:entry (e.g., /UsagePoint/1/MeterReading/1) to its hierarchically superordinate atom:entry (e.g., /UsagePoint/1), and we came to the conclusion that they should use, e.g., href="/UsagePoint/1", whereas in the examples given in the NAESB draft they use href="/UsagePoint/1/MeterReading", i.e., they point to a collection, not to an entry.
On my flight home, I have thought about this question again, and I now consider our conclusion premature.
Although up-links that point to an entry make it easier to select the hierarchically superordinate atom:entry, they prevent the other direction, making it impossible to select the hierarchically subordinate entries. For example, an <atom:link rel="up" href="/UsagePoint/1"/> would not only be found in the atom:entries for the MeterReadings, but also in the atom:entries for the ElectricPowerUsageSummaries, and these two kinds of subordinate entries cannot be distinguished without decomposing the URLs in the hrefs, what we wanted to avoid in the first place.
On the other hand, up-links that point to collections make it possible to elegantly select (with the help of XPath expressions) both the superordinate and the subordinate elements. Given an atom:entry element, we can select
• the hierarchically superordinate element by putting the context on the given atom:entry's up-link and evaluating the XPath expression
../../atom:entry[atom:link[@rel="related"]/@href=current()/@href]
(we select the entry whose related-link points to the same collection as the current up-link)
• the hierarchically subordinate elements of a given type (e.g., the MeterReadings, but not the ElectricPowerUsageSummaries) by putting the context on the corresponding related-link and evaluating the XPath expression
../../atom:entry[atom:link[@rel="up"]/@href=current()/@href]
(we select the entries whose up-links point to the same collection as the current related-link).
These selections work for 1:n relationships between super- and subordinate elements, where both the related-link and the up-link have type="application/atom+xml;type=feed".
But there are also 1:1 relationships
• either hierarchically from a super- to a subordinate element (e.g., the ElectricPowerQualitySummary subordinate to a UsagePoint) where both links have type="application/atom+xml;type=entry"
• or associations independent of the hierarchy (e.g., the ReadingType related to a UsagePoint), here the related-link has type="application/atom+xml;type=entry" (but the up-link of the ReadingType points to the feed of ReadingTypes and has type="application/atom+xml;type=feed").
In the case of such 1:1 relationships, analogous XPath expressions to the ones shown above can be used, where [@rel="up"] is replaced by [@rel="self"] (and the parenthesised "we select" remarks should read "the same entry" instead of "the same collection").
To summarise: I now believe that the up-links given in the NAESB draft are perfectly valid.
Comments:
heiko.theißen (4/12/2012 11:03 AM): The way I would expect the atom:links to be interpreted can be expressed in the following XSLT templates:
xsl:template match="atom:link[@rel='related']">
xsl:variable name="target" select="ancestor::atom:feed/atom:entry/atom:link
[@rel!='related' and @href=current()/@href]"/>
xsl:choose
xsl:when test="$target/@rel='self' and
$target/../atom:link[@rel='up']/@href=../atom:link[@rel='self']/@href">
xsl:apply-templates select="$target/.." mode="hierarchical-relation-1-1"/>
</xsl:when
xsl:when test="$target/@rel='self'">
xsl:apply-templates select="$target/.." mode="non-hierarchical-relation-1-1"/>
</xsl:when
xsl:when test="$target">
xsl:apply-templates select="$target/.." mode="relation-1-n"/>
</xsl:when
</xsl:choose
</xsl:template
xsl:template match="atom:link[@rel='up']">
xsl:variable name="target" select="ancestor::atom:feed/atom:entry/atom:link
[@rel!='up' and @href=current()/@href]"/>
xsl:choose
xsl:when test="$target/@rel='self'">
xsl:apply-templates select="$target/.." mode="hierarchical-relation-1-1"/>
</xsl:when
xsl:when test="$target">
xsl:apply-templates select="$target/.." mode="relation-1-n"/>
</xsl:when
<!-- There are no up-links for non-hierarchical relations. -->
</xsl:choose
</xsl:template
heiko.theißen (4/12/2012 10:54 AM): I think that the usage of these links is not yet uniformly observed in all the Green Button examples that are "out there". The sample data by SDG&E http://www.greenbuttondata.org/greendevelop.html are correctly linked, they seem to have been copied from the original sample https://collaborate.nist.gov/twiki-sggrid/pub/SmartGrid/GreenButtonInitiative/GETBatchSampleEUI.xml with only the IntervalBlock contents replaced.
I found one other set of sample data by PGE http://pge.com/includes/docs/xml/shared/greenbutton/XML%20Sample%20Residential%20Electric%20Usage%2007072011%2012032011.zip, and here the atom:links are not used in the way I expect:
• The UsagePoint entry contains a related-link to a MeterReading entry whereas it should link to a MeterReading feed.
• The MeterReading entry should contain an up-link to the MeterReading feed /UsagePoint/xxx/MeterReading, but it does not.
• The MeterReading entry contains a related-link to an IntervalBlock entry whereas it should link to an IntervalBlock feed.
• The IntervalBlock entry should contain an up-link to the IntervalBlock feed /UsagePoint/xxx/MeterReading/yyy/IntervalBlock, but it does not. (There are no up-links at all in the sample.)
The hierarchial relationship between the entries in PGE's sample can only be derived from the internal structure of the hrefs, by splitting these URIs at the slashes. But in my opinion this should not be done, since these URIs are assumed to be opaque.
The usage of atom:links is probably irrelevant to "Green Button apps" that only visualize the IntervalReadings for one UsagePoint. But the issue should become relevant as more complex apps are developed. A common understanding about the links between producers of Green Button data (energy companies) and consumers (app developers) is perhaps not yet reached.
martin.burns (1/31/2012 2:25 PM):
Recommendation of OpenADE Task Force
For discussion white paper see : http://osgug.ucaiug.org/sgsystems/OpenADE/Shared%20Documents/Testing%20and%20Certification/GreenButtonTestPlan/referenceMaterial/GreenButtonAtomLinks.docx
Best Practices: [NAESB]
- Every entry shall have an uplink to its parent feeds.
- URLs are opaque and not designed to be parsed for navigation. They may reflect the internal storage or layout of the servers which will vary. Therefore clients should not make any assumptions about their structure.
- If a server supports retrieving feeds, the feeds shall have an uplink to its parent entry.
- When the multiplicity in the data model is not 1:1, the “related” link must point to a feed
5. When the multiplicity in the data model is 1:1, the “related” link must point to an entry
- Batch defines the set of atom entrys that pertain to a common single Authorized resource.
- Bulk defines a set of atom entrys that pertain to multiple authorized resources theoretically with separate individual or Batch authorizations.
- A Subscription pertains to any single resource. However, Subscription is probably always a feed.
- Use of BatchItemInfo. If you want to support multiple requests (e.g. http requests) in a single call, you would need BatchItemInfo to distinguish the results. This probably needs to be elaborated further in the standard. *** should result in a new Help desk item
- All collections of atom entrys are returned as a feed.
- entry/@base or feed/@base may have a common URL base after which all relative link URIs are appended. e.g. feed/@base = http://my.web.com/DataCustodian/. If there is a conflict between base and a resource link, use the resource link.
[60] Green Button file names contains PII
Are the early implementations of Green Button encoding PII in the exported filenames? One of the primary tenets of ESPI is the "separation of privacy" between Data Custodian and 3rd Party; encoding PII in the filename allows that information to leak between the Custodian and 3rd Party.
Comments:
Recommendation of OpenADE Task Force
Add inspection test for Green Button Data in the test plans.
[61] ESPI lacks clarity for sending OAuth authorizations in batch
If it is possible to send updates for resources requiring multiple distinct authorizations, the method for doing so needs to be defined. One possible way would be to add signature_method, signature, and nonce to Authorization and allow it to be included with BatchItemInfo. Otherwise, if Data Custodians are to verify valid authorization within batch POSTs, and reject submissions associated with resources to which the Authorized Thrid Party does not have authorization, that should be specified.
06/12: Can we clarification from Steve Van Ausdall on this? Is he proposing that there be facilities for multiple tokens in a batch response?
Comments:
donald.coffin (8/17/2012 4:52 PM): Assuming the definition of "bulk" and "batch" as described in John Teeter's definition. The current OAuth flow should be capable of handling the "batch" situation, since it requires the Data Custodian to only verify the authorization of a single user. The "bulk" situation requires the Data Custodian to verify the user has authorization to access the data of multiple Retail Customers. One method of achieving this would be for the Data Custodian to establish a "unique" UserID for the "bulk" user account which would then allow the "bulk" transfer transaction to utilize the same OAuth process as all other users. For the "bulk" user to gain access to various different Retail Customer's data, the Data Custodian would then be required to establish a technique that a Retail Customer can use to "grant" permission to the "bulk" user to access their data. The definition of this "granting" process is beyond the scope of the ESPI standard and therefore is NOT covered.
john.teeter1 (8/14/2012 3:31 PM): We had a brief discussion about "Batch" on OpenESPI call. While we didn't resolve the OAuth question directly, we figure there were two categories of things: "Batch" and "Bulk" where: Batch is the delivery of multiple resources under the auspices of a single Retail Customer's authorization. Bulk is the delivery of multiple resources under the auspices of 1 or more Retail Customer's authorization. I could do a "batch" request for multiple subscriptions owned by one user; or I might do a "bulk" request for multiple subscriptions each of which might be owned by different Retail Customers. The OAuth implications of the two (batch.vs.bulk) seem to be unique.