Testing & Evaluation Teleconference
30 January 2014
Roll Call

Kristen Barrera (Cuda)711th HWP/RHAS (TE Lead)

Gary AnayaONR RC (TE Co-Lead)

Subhashini GanapathyWright State University

Jacob HodgesLMITT

Introduction

Discussion:

-This teleconference was called to kick-off the development of the Testing & Evaluation (TE) activities between the US participants before seeking inputs from the international participants

-Cuda provided an agenda to participants that identified four areas of discussion; specifically, (1) Update from Subhashini Ganapathy and Dwayne Eldridge’s (DISAM) meeting on the Systems Architecture; (2) T&E Questions (i.e., demographics, DMAP Update, and types of data to be collected); (3) T&E Process; and (4) Schedule the next TE Teleconference

Systems Architecture Meeting

/ Subhashini Ganapathy

Discussion:

-Subhashini Ganapathy briefed the team on the Systems Requirements discussions held at DISAM. She said the discussions focused on (1) the systems development and how the architectural environment would look, (2) the hosting of the environment, and (3) how it would be used for data collection and evaluation.

-Subhashini Ganapathy mentioned that the DISAM content will be in four formats (i.e., video, e-publication, mobile devices and web-based). The confirmation of mobile devices also led into the need to collect transparency data as part of the data analytics process.

  • Transparency Data Collection – Gary Anaya and Jacob Hodges both commented on the process used during the Mobile Learning Environment (MoLE) Project and briefings that were presented by Asi DeGani (Tribal) and Gary Anaya (ONR RC) on this area. In order to provide additional information for TE Planning, a copy of these briefings will be provided to the TE team.
  • Assignment of PIN Numbers and Welcome Message – Jacob Hodges stated that there is a need to ensure that the Security Cooperation Offices (SCO) Interim Evaluation and the Concept Evaluation (CE) needs to use the same process (i.e., compile a spreadsheet that will identify (1) email address of the volunteer, (2) Name of the international delegate who contacted the volunteer, (3) Email address of the Country Delegate, (4) A personal identification number (PIN) for the respective volunteer, (5) alpha spelling of the PIN number, and a (6) hyperlink to the SCO Interim Evaluation/ Concept Evaluation Registration Portal). This approach will help identify any potential problems during the beta test (i.e., SCO Interim Evaluation) and support the Research Protocol (RP) process. A copy of the files associated with the aforementioned RP process will be provided to the T&E team.

Testing & Evaluation Questions

/ Cuda

Discussion:

-Cuda stated off the discussions by providing the team with a read-ahead of the Demographics that has been compiled based on the input from the MLS Kick-Off Meeting.

  • Demographics Review – the TE team discussed the questions compiled by Cuda from the MLS Kick-Off Meeting. It was agreed that the total number of demographic questions should be limited to 9 or 10. It was agreed that the demographic questions should be uploaded to the Google Docs site for review by the international participants to ensure an equal understanding of terminology and understanding of what is to be achieved.
  • Cuda marked-up the [Draft] Demographics document by annotating areas the need to be reviewed as well as questions that could possibly be used as open-ended questions for closing the survey.
  • Data Mapping & Analysis Process (DMAP) – Cuda provided a draft copy that she is working with Subhashini Ganapathy to complete. The DMAP provides a picture of the Measures of Effectiveness (MoE) that will be used in the data collection process as well as the analysis tools that will be used to evaluate the data. The draft DMAP will be provided NTL 12 Feb 2014 via Google Drive and a note placed in the MLS LinkedIn page.
  • Testing & Evaluation (TE) Process – A brief discussion was held on the end-to-end TE process with respect to how it needs to be aligned with the Research Protocol Requirements. Jacob Hodges provided the team with a copy of the MLS Research Protocol Process, a copy of the TE process documents utilized as part of the Mobile Learning Environment (MoLE) Project were provided to the TE Team.
  • Schedule of next TE Teleconference – the next TE Teleconference will be held on Friday, 7 February 2014.

Action Items

Action items: / POC: / Deadline:
Provide the T&E Team with a copy of the presentation that discussed Transparency Data Collection / Jacob Hodges / Completed
Provide the T&E Team with a copy of the MoLE Registration Excel Spreadsheet that was used to allocate Personal Identification Number (PIN) / Jacob Hodges / Completed
Provide the T&E Team with a copy of the ‘MoLE Welcome’ email to show how the information from the Excel Spreadsheet is collated into the ‘welcome’ message / Jacob Hodges / Completed
Provide the T&E Team with a copy of the Research Protocol Process Flowchart / Jacob Hodges / Completed
Schedule the next T&E Teleconference / Jacob Hodges / Completed
Provide the T&E Team with a copy of the Data Collection Pivot Table that was used for analysis / Jacob Hodges / Completed
Populate the Data Management Analysis Tool (DMAT) to reflect MLS TE Data Collection Strategy / Kristen Barrera / Subhashini Ganapathy / Ongoing
Develop a spreadsheet for seeking international input to Demographics Questions / Gary Anaya/ Kristen Barrera / 7 Feb 2014
Post the recommended Demographics Questionnaire to the Google Docs for international participant review/concurrence / Gary Anaya / 7 Feb 2014

Testing & Evaluation Conference Call Minutes – 30 January 2014Page 1