Department of Finance and Deregulation
Reliance Framework
2 September 2011
Contents
1.Our Approach
2.Assumptions
3. Glossary
4.Requirements traceability
5.Stakeholders interviewed
6.Data Book
7.Reference tables
8.Potential for Ready Application
9.Service interaction maps
10. Statistics
11. Mutually Exclusive Service Interactions
1.Our Approach
Ernst and Young have applied a rigorous approach to the capture of data and to the analysis conducted. This section outlines the control mechanisms we applied to the capture of data, and the method we applied to produce the outputs for the project.
1.1Controls in the capture of data
Data capture, in particular that associated with manual interpretation and entry of complex, unstructured data, is prone to accidental error. We have designed our process to be conscious of this risk and then applied a structured approach to check for the presence of quality issues. This approach includes layers of quality checking across the process that is then subjected to verification by team members not directly involved in the data capture and entry.
To minimise the likelihood of data capture and entry errors we built the following mechanisms into our data capture process:
►A relational database, using MS Access, was built for data entry. This combined enforcement of referential integrity in data captured with a strict lockdown on the functionality end-users could access. Users were only able to access data through pre-configured forms rather than directly to the raw tables. The data capture forms only allowed users to select values from master tables (e.g. agency table or form table) and users were not granted access to add entries to master tables.
►The team undertaking the analysis was physically seated together in a single room. This allowed discussion on data elements where some interpretation was required, and in more complex cases this was discussed with the broader project team. The outcomes of these discussions were captured to minimise the likelihood of inconsistent interpretation.
►The team was overseen by an Executive Director from our Melbourne office who has considerable experience in data analytics and information management
A Quality Assurance process was applied that sampled the data captured to check for errors in the actual data captured, and for the absence of data that should have been captured. This quality assurance was conducted by an independent person with experience in data audit.
1.2Identification of service interactions
The definition of a Service Interaction was agreed to be a unique interaction that an individual has with the Commonwealth to use a specific service.
The service interactions were identified through a:
►Review of agency websites, annual reports and Commonwealth publications, and
►Series of meetings with agency representatives.
Once identified, the service interactions and forms were:
►Provided to the agencies for review and confirmation. Specifically, agencies were provided with a spreadsheet and asked to confirm the list and to provide volume data for the identified service interactions, frequency of transactions and channels of form submission. Some agencies were able to confirm the service interactions and provide data requested for a portion of service interactions. The majority of service interactions however, were not confirmed and only a small amount of volume data was received.
►A decision was taken to proceed with the analysis due to the time criticality of the assignment. This was achieved using the “Life Events” categories from as a filter to limit the interactions.This formed the basis of the first service interaction ‘Cluster’. Later in the project a number of other clusters were investigated with the Department of Finance and Deregulation.
►The list of services was further rationalised by removing mutually exclusive interactions as defined by DoFD and DEEWR. An example of a mutually exclusive pair of service interactions are “Claim new start allowance” and “Apply for youth allowance“.
Each data element was assigned into a hierarchy from level 1 (highest level) to level 5 (most granular level of information).
An MSAccess database was designed and developed to allow capture of whether the data element was present on the form. Users of the database manually reviewed each form and determined whether the data element existed on that form. If the data item existed then an entry was made onto the form. Where assumptions were required to be made to determine the data element’s appropriateness, these have been captured in the database to allow for consistency between different members of the data evaluation and entry team. The team worked on this in a room together so open discussion on data could be had.
A screen shot from the data entry form is shown below.
Diagram 1: MSAccess database
Results and Limitations
The result of this exercise provided the number of forms individuals could complete at different times. It also showed the number of times (as a raw count) same or similar information could be provided by individuals.
No weighting of the importance of the data element was applied, nor were the business rules for the use of that data considered. This analysis also did not consider any privacy or legislative constraints of sharing the data – it was an exercise in mapping and counting the number of times the same or similar data elements appeared in the set of forms.
1.3Grouping of service interactions into clusters
A cluster has been defined as a series of transactions that may occur for a particular population group or customer type; for instance, services and obligations that apply on the birth of a child or apply in an ongoing way to adults employed full-time.
A number of clusters were identified and then service interactions were allocated to each (and agreed with DoFD).
The clusters were separated into two views, namely a Commonwealth view and an Individual view as illustrated below:
Diagram 2: Clusters
Both clusters and cluster elements were developed through consultation and discussion with DoFD as logical combinations of service interactions. Each of the clusters deal with different groups of individuals, agencies or events as shown below:
Table 1: Groupings within Clusters
The forms (for each service interaction), were then mapped against each of the clusters to identify how they would be used (i.e. the service interactions). For example “Claim baby bonus” is mapped to the “starting a family” cluster element. A breakdown of these is provided in Section 7.
This mapping was then represented on the service interaction maps as provided in Section 9.
Results and Limitations
There is a range of ways that the clusters and cluster elements could be defined. The list of cluster groups (and their elements) is not exhaustive but is believed to be sufficient for the purposes of the Scoping Study.
The approach to defining clusters and their elements was undertaken following discussions with DOFD and reviewing Commonwealth documents and websites, such as
Once defined, the clusters (and their elements) were then investigated for their value in illustrating the potential benefits (i.e. in terms of sharable information). It was determined that the two most valuable clusters were:
►Customer Type
►Life Events
Each of these clusters were subsequently investigated to define sharing potential, or to otherwise determine the potential for the scope of the Reliance Framework.
Finally, with agreement from DoFD and after discussion with Deputy Secretaries, it was determined that the ‘Customer Types’ and ‘Life events’ clusters would be analysed in detail.
1.4Potential benefits to Individuals and Commonwealth
To identify the potential benefits to individuals and the potential benefits to the Commonwealth, the following activities were undertaken:
►A list of potential benefits were created, by reviewing those benefits identified by DoFD in the RFQ and through stakeholder conversations.
►The benefits categories were then validated and agreed with DoFD.
The resultant potential benefit areas are illustrated below:
Diagram 3: Benefit categories
Results and Limitations
There is a range of benefit categories that could be defined. For the purposes of this report we have analysed only the benefits marked with ticks, based on direction provided by the Department of Finance and Deregulation and limitations in the information available.
Once the potential benefits areas were identified, the cluster elements were assessed against them to derive a priority (based on potential benefit magnitude) of enablement through the Reliance Framework.
1.5Development of benefit heat maps
To identify the Life Events and the Customer Types that will provide the highest potential benefit to Individuals and to the Commonwealth:
►Each of the benefit categories were comparatively rated as high, medium or low
►The ‘Individual benefit potential’ rating was determined by calculating the mean of the individual benefit categories
►The ‘Commonwealth benefit potential’ rating was determined by calculating the mean of the government benefit categories
►The ‘Combined benefit potential’ rating was determined by calculating the mean of both the ‘overall individual benefit potential’ and ‘overall Commonwealth potential’
Table 2 below provides the high, medium and low rating for each benefit category.
Table 3 below provides the high, medium and low Individual and Commonwealth and combined total benefit ratings.
INDIVIDUAL BENEFITS / High / Medium / LowNumber of occurences ('000) / 14300 ≤ 400 / 350 ≤ 120 / 120 ≤ 0
Number of agency interactions / Life Events: 7 ≤ 5
Customer Types: 8 ≤ 6 / Life Events: 5 ≤ 3
Customer Types: 6 ≤ 3 / Life Events: 3 ≤ 0
Customer Types: 3 ≤ 0
Number of service interactions / Life Events: 19 ≤ 12
Customer Types: 40 ≤ 26 / Life Events: 12 ≤ 6
Customer Types: 26 ≤ 13 / Life Events: 6 ≤ 0
Customer Types: 13 ≤ 0
Count of data elements / Life Events: 585 ≤ 390
Customer Types: 1305 ≤ 870 / Life Events: 390 ≤ 195
Customer Types: 870 ≤ 435 / Life Events: 195 ≤ 0
Customer Types: 435 ≤ 0
Level of pain / High / Medium / Low
COMMONWEALTH BENEFITS / High / Medium / Low
Number of occurences ('000) / 14300 ≤ 400 / 350 ≤ 120 / 120 ≤ 0
Number of agency interactions / Life Events: 7 ≤ 5
Customer Types: 8 ≤ 6 / Life Events: 5 ≤ 3
Customer Types: 6 ≤ 3 / Life Events: 3 ≤ 0
Customer Types: 3 ≤ 0
Number of service interactions / Life Events: 19 ≤ 12
Customer Types: 40 ≤ 26 / Life Events: 12 ≤ 6
Customer Types: 26 ≤ 13 / Life Events: 6 ≤ 0
Customer Types: 13 ≤ 0
Count of data elements / Life Events: 585 ≤ 390
Customer Types: 1305 ≤ 870 / Life Events: 390 ≤ 195
Customer Types: 870 ≤ 435 / Life Events: 195 ≤ 0
Customer Types: 435 ≤ 0
Table 2: Rules for high, medium and low benefit ratings
BENEFIT TOTALS / High / Medium / LowINDIVIDUAL BENEFIT POTENTIAL / 150 ≤ 100 / 100 ≤ 50 / 50 ≤ 0
COMMONWEALTH BENEFIT POTENTIAL / 150 ≤ 100 / 100 ≤ 50 / 50 ≤ 0
COMBINED BENEFIT POTENTIAL / 150 ≤ 100 / 100 ≤ 50 / 50 ≤ 0
Table 3: Rules for high, medium and low total benefit ratings
Results and Limitations
This analysis does not take into consideration the number of times the same Individual repeats the occurrence.
1.6Ready application
A sample of service interactions and related forms were selected to conduct detailed analysis to determine the potential for ready application within a Reliance Framework in the immediate, near term and long term. The sample size chosen was approximately 13% of the total of identified forms.
As set out in the table 4below, each form was reviewed to determine:
►Identical information, for potential immediate application within a Reliance Framework where the question and answer field on the forms are identical
►The same information (yet collected differently), for potential near term application within a Reliance Framework where the question on the forms is the same, but the answer field is different
►The information where the underlying concept is similar but not identical, for potential long term application within a Reliance Framework. The only concept that was selected for this category was identification number (specifically excluded from either of the above two categories) as the concept is the same (i.e. the ID number is used to identify individuals) but not identical across agencies.
►Unmatched information where the information collected from the forms did not fit into any of the above categories
Data element group / Identical information / Same information collected differently / Underlying concept is similar but not identical / TOTAL / Unmatched informationIndividual details / 96 / 9 / 103 / 208 / 173
Partner/Carer details / 23 / 2 / 36 / 61 / 54
Dependent's details / 7 / 0 / 7 / 14 / 20
Children's details / 8 / 0 / 9 / 17 / 33
Housing status / 3 / 0 / 0 / 3 / 29
Employment status / 2 / 0 / 0 / 2 / 22
Disability status / 0 / 0 / 0 / 0 / 14
Assistance to access service / 2 / 0 / 0 / 2 / 5
Financial details / 22 / 1 / 0 / 23 / 219
Total / 163 / 12 / 155 / 330 / 569
Table 4: Ready application with a Reliance Framework
2.Assumptions
The table below provides the assumptions that underlie our analysis:
Reference / Assumption1 / The scope of the analysis excluded Government to Business, and Government to Third Party interactions (however, DEEWR forms which include Third Party to Government have been included in this analysis at the request of DOFD)
2 / Positive verification of the service interactions for agencies was only provided by two agencies. As such, in order to meet the timeframes our analysis continued on the assumption that the identified services interactions for the remaining agencies were correct
3 / The higher the number of agencies involved in a service interaction, the higher the relevance of that interaction to the analysis
4 / The lower the level of occurrence of an interaction, the lower the level of benefits to the individual and the Government
5 / Most stressful events of life occur when individuals experience death, divorce, major personal injury or illness, marriage or gain new family members[1]
6 / For individuals the benefit category of ‘provide same data less often’ is determined by:
- Number of occurrences
- Number of agency interactions
- Number of service interactions
- Count of data elements
7 / For individual the benefit category of ‘less pain’ is determined by the stress impact of Life Events
8 / For the Commonwealth the benefit category of ‘save time and effort’ is determined by:
- Number of occurrences
- Number of agency interactions
- Number of service interactions
- Count of data elements
9 / The 'name' and 'address' data elements have been treated as constructs that can be shared, requesting the same data in different ways. 'Name' consists of current and previous name and 'address' consists of residential and postal address.
10 / Forms associated with Life Events or Customer Types that were available online only have not been coded.
11 / In the 'income' category, the same constructs that are requested over different financial years are treated as one.
12 / The 'employment status' and ‘student status’ categories only look at current status, not previous (e.g. 'employment status at the time of termination').
This assumptions table should be read in conjunction with the data book (Section 6).
3. Glossary
The table below provides a glossary of terms in this report.
Term / DefinitionIndividual / A person who is eligible to use Commonwealth services.
Service interaction / A Service Interaction is defined as a unique interaction that an individual has with the Commonwealth to use a specific service.
NB: the definition of service interaction has been re-defined to the above in agreement with DOFD from the contract definition of:
‘the provision by an individual (or their agent) of a single piece of data, such as provision of their name, of their fortnightly earnings, or of the date of birth of their youngest child’
Cluster / A cluster is a series of transactions that may occur for a particular population group or customer type; for instance, services and obligations that apply on the birth of a child or apply in an ongoing way to adults employed full-time
Cluster element / A cluster element is a specific population group experiencing an event(e.g. ‘Becoming independent’) or customer type (e.g. ‘Youth’). Each Cluster is composed of multiple cluster elements
Life events / Is determined by the events people experience in their lives (
Customer types / Is determined by how people identify when they present (i.e. complete a form)
Number of occurrence / The volume of transactions for a unique individual within a year, for the life events and customer types clusters, provided by the Department of Finance and Deregulation. Refer to Section10 Statistics for details
Note: Change of details may not constitute transactions unique individuals transactions
Data element / The lowest level of data requested on a form
4.Requirements traceability
The table below shows the section of either the Findings and Conclusions, the Method or the Data Book (both of which are the Analysis Approach) in which the requirement is addressed. Where a requirement is addressed by a document holistically, a Tick is provided as indication of this.
Ref. / Text / Findings and Conclusions / Analysis Approach
Method / Data book
a. / Investigate the viability of a Reliance Framework as described / √
b. / provide Government advice identifying those Commonwealth government services or transactions which require action by more than one agency / √
c. / identify where the ability for people to share their personal information in the manner outlined above could improve the convenience and responsiveness of government service delivery / √
1. / Identify and document those service interactions where the same or similar information is sought from individuals by more than one Australian Government agency
NB: the definition of service interaction has been re-defined to the above in agreement with DOFD from the contract definition of:
‘the provision by an individual (or their agent) of a single piece of data, such as provision of their name, of their fortnightly earnings, or of the date of birth of their youngest child’ / 4 / 1.2 / 5.1
2. / Classify those service interactions identified in item (1) according to: / √
2A. / The potential benefits to individuals and the potentialbenefits to Government if the information was able to be reused within a Reliance Framework - based on a rigorous,documented set of criteria including but not limited to considerations of volume reductions, time saved, financial cost reductions, accuracy improvements and security improvements / 3 / 1.4
1.5 / 6.1
2B. / Their potential for ready application within a Reliance Framework - from immediate potential where the information is identical, to near-term potential where the same information is collected differently, through to longer term potential, where the underlying concept is similar but not identical and legislation would be required / 5 / 1.6 / 6.3
3. / Where practical, group the service interactions into clusters that, taken together, could in the short, medium or longer term provide a material benefit to a specific population group of customer type if the information provided by individuals could be reused, and could also provide benefits to Government, as classified in items (2a) and (2b) / 6 / 1.3 / 6.4
6.6
Email 19 July
EY to DoFD / 1. A summary of the findings for report 3 supported by the evidence, including:
- An updated version of the heat map findings in report 3, including the following changes:
a) Including agency touch points as a static item
b) Remove the reduction in service interactions
c) analysis in point 2 below) could provide the greatest benefit for ready application in the reliance framework
- Removal of sections / information / diagrams that are not set out in the contract / √
2. Extend the analysis to include the following four additional data elements:
a) Disability (e.g. details of their disability)
b) Housing (e.g. where do you live)
c) Employment / student status (e.g. nature of your employment status or student status)
d) Relationships (e.g. relationship details)
This analysis will be provided to the same level, in the same format, as provided for the ‘income’ and ‘children / dependents’ deep dive (highlighted in yellow) in the spreadsheet attached. This analysis will include:
a) Coding the forms that include the data elements above (refer to 1a. 1b. 1c. 1d)
b) Analysing the data elements to determine whether:
- The information collected is identical
- The same information is collected differently
- The underlying concept is similar but not identical
c) Report findings in the updated version of the report 3 / √
3. Provide an appendix to report 3 that includes our methodology (referenced within the report) assumptions, qualifications and limitations of the data / √
Email 20 July DoFD to EY (Response) / We need an estimate of the error rate so that we can state the accuracy / √[2]
Although there is no need to redo reports 1 and 2, would you please flow through the data described below to the summary table from Report 1, which appears on page 18 / √
5.Stakeholders interviewed