MANAGEMENT TEAM WORKSHOP PROCEEDINGS

Mobile Learning Environment (MoLE)

FinalWorkshop

Location:

Office of Naval Research Global (ONRG) London

Blenheim Crescent, West Ruislip

11-12 September 2012

Principal Investigator:

Jacob Hodges, Q&P Ltd

1. Introduction

On 11-12 September 2012, the final Mobile Learning Environment (MoLE) Management Meeting was held at the Office of Naval Research Global (ONRG) in West Ruislip for the Office of the Under Secretary of Defence (Acquisition, Technology & Logistics) International Cooperation Office (Coalition Warfare Program) and the MoLE Project Managers, the Commander, U.S. Naval Forces Europe (CNE)/Commander, Naval Forces Africa (CNA) Deputy Director of Training/Readiness. The purpose of the meeting was toprepare for the close-out of the Mobile Learning Environment (MoLE) Project by having each of the working group leads present project-related activities that occurred in FY12 and FY13, present the results from the MoLE Proof of Concept and to conduct Science & Technology (S&T) Project Reviews on four mobile learning research grants that contributed to the success of the project.

The Mobile Learning Environment (MoLE)Management Meeting consisted of representatives from the Office of the Under Secretary of Defence (Acquisition, Technology & Logistics) International Cooperation Office (Coalition Warfare Program); Commander, U.S. Naval Forces Europe (CNE)/Commander, Naval Forces Africa (CNA) Deputy Director of Training/Readiness; Deputy Director for Joint Staff J7 Joint and Coalition Warfighting (DD J7 JCW JKO); Deputy Director for Joint Staff J4 Health Services Support Division; Advanced Distributed Learning (ADL) Initiative; U.S. Army Medical Research and Material Command, Telemedicine and Advanced Technology Research Center (TATRC); U.S. Air Force 711th Human Performance Wing (HPW)/Human Effectiveness Directorate (RHAS); Naval Medical Research Unit THREE (NAMRU-3); Office of Naval Research (ONR) Program 38; UK Ministry of Defence (MoD) Defence Centre Training Support; Defence Academy of the United Kingdom (DAUK); Q&P, Limited; and Tribal LTD.

A copy of all proceedings and the relevant documents, research studies and presentations related to the Mobile Learning Environment (MoLE) Project will be transferred, by 30 November 2012, to the All Pacific Area Network (APAN) at

2. Background

Commander, Naval Forces Europe (CNE)/Commander, Naval Forces Africa (CNA)/Commander, SIXTH Fleet (C6F) Deputy Director for Training and Readiness (N7) was the Lead and Project Manager for the Mobile Learning Environment (MoLE) Project. This project was approved for funding by the Coalition Warfare Program in May 2010. For the last two years, MoLE has focused on providing the foundation for conducting a proof of concept that would evaluate mobile learning (m-Learning) and information exchange capabilities using the Deputy Director, Joint Staff (J7) Joint and Coalition Warfighting (DD J7 JCW) Joint Knowledge On-Line (JKO) capability.

The project consisted of three working groups (i.e., Medical Content, Technology & Transition, and Testing & Evaluation) that would support the development and deployment of a mobile learning solution for meeting the emergent requirements for training to partner countries within the CNE/CNA/C6F theatre of operations, the DD J7 JCW Joint Knowledge Online (JKO technological sphere of influence with a cooperative and coordinated effort between U.S. governmental services and agencies in countries around the globe using mobile devices in the Advanced Distributed Learning (ADL) environment, and the need to support the sharing of educational content between US and multi-national partners in support of medical stability operations.

MoLE used Commercial Off-the-Shelf (COTS) mobile devices to assess the mobile medical content developed by the Medical Content Working Group. The Technology and Transition Working Group focused on providing a sustained capability once the MoLE Proof of Concept (PoC) had concluded and lessons learned were identified. The Testing and Evaluation Team was responsible for integrating the needs and requirements of the other working groups in order to focus on identifying and developing an effective strategy for evaluating the usability, utility, suitability and effectiveness’ of mobile technologies as a training and education medium.

As a result of the MoLE PoC, the project teamsaddressed many interface issues related to the development and deployment of future m-Learning capabilities based on a plethora of operational requirements as well as develop a transition strategy to ensure the project met the key performance indicators required for a Program-of-Record (POR) (e.g., Coalition Warfare Program, DD J7 JCW and Medical Community).

3.Presentations/Discussions

11 September 2012

  1. Herbert (Duke) Boutwell (Commander, Naval Forces Europe (CNE)/Commander, Naval Forces Africa (CNA)/Commander, SIXTH Fleet (C6F) Deputy Director for Training and Readiness (N7)), welcomed participants that were attending in person as well those attending as via the video teleconference (VTC) and teleconferencing.
  1. Duke Boutwell presented the “Mobile Learning Environment (MoLE) Coalition Warfare Program (CWP) Final Management Meeting” brief that recapped the two-year project. He provided an introduction of MoLE, overview of project goals and objectives, project end state, overview of the contracts and research grants funded by the Coalition Warfare Program (CWP), international participants involved in the project, and an overview of each of the working groups (i.e., Medical Content, Technology & Transition, and Testing & Evaluation). Jacob Hodges, Q&P Limited, closed-out the briefing by highlighting each of the seven research grants that played an important part of the MoLE Project.
  1. Cynthia Barrigan, Medical Content Lead and the Portfolio Manager for Global Health Engagement at the U.S. Army’s Telemedicine and Advanced Technology Research Center (TATRC) presented a review of the purpose and scope for the Medical Content Working Group. She discussed the goals and objectives of the Medical Content Working Group, sponsorship and funding that supported the mobile content, the teams’ roles and responsibilities, major activities/milestones, and the working groups accomplishments based on use-cases, strategic communications, organizational framework, standards, content collaboration, the tools and content. Cynthia Barrigan then discussed the strengths of the working group, the challenges faced in developing a medical app that represented the US/International partners, lessons learned and the transition strategy.
  1. Gary Anaya, Testing and Evaluation Working Group Lead from ONR’s Program 38, lead the discussion on the MoLE Testing & Evaluation (T&E) Results with other members of the T&E Team. Gary Anaya started the presentation by identifying the primary objectives of the T&E Working Group and both FY11 and FY12 activities before discussing the MoLE Proof of Concept (PoC).

For the PoC, he presented the six-step Testing and Evaluation Process used by the volunteers, the “Welcome email” and “Reminder Email” that was provided to the PoC volunteers at the start of the PoC and the bi-weekly emails that were sent to those that had not yet registered. With the assistance of Jacob Hodges, he discussed the MoLE PoC Informed Consent and the key point that it was translated into French and Spanish by the international partners. Gary Anaya then discussed the Demographics questions that were asked when MoLE PoC participants registered.

In discussing the MoLE PoC Analysis, Gary Anaya started by providing an overview of the participant totals and the demographics. He introduced the Testing & Evaluation Approach (i.e., evaluative questions as well as the category of questions). Adam Abrams and Phong Le then discussed the results from the PoC and the responses to the questions including a look at how all users responded in addition to responses related to android and iPhone users. The discussion then focused on the Category Analysis of how the volunteers rated the accessibility, self-efficacy, usefulness and utility of using mobile technologies for training.

Piers MacLean, from the Defence Academy of the United Kingdom (DAUK), then discussed the results from the qualitative questions. His discussions included the results from synonym analysis and a matching of similar terms using high frequency analysis on volunteer descriptions of their experience, what they liked most and least about mobile device learning and additional comments.

Gary Anaya completed the T&E discussion by discussing the User Feedback, User Suggestions and Conclusions based on the results of the MoLE PoC.

  1. Mark Willmann, the Technology & Transition Lead from the Deputy Director Joint Staff (J7) for Joint and Coalition Warfighting (DD J7 JCW) presented the Mobile Technology and Transition briefing. He reviewed the primary objectives of the Technology & Transition Working Group and FY11 and FY12 Technical Milestones and Deliverables. Mark Willmann elaborated further on the MoLE Application access (i.e., Apple iOS 5.0 and Android 2.3), transition strategy, JKO Mobile approach and student authentication process, tracking and reporting

Mark Willmann discussed the JKO Architecture as of 11 September 2012, and additional JKO Mobile enhancements that are forthcoming. This included the user experience as it related to entry into the JKO Mobile area, public and private views, roles-based access, graphic user interface (GUI), a detailed description of the JKO Mobile Enhancements, the Mobile Application Usability, Managing Users/Access Control, and the m-Learning Suite Reporting.

  1. Day 1 ended with Duke Boutwell leading open discussions on the data findings from the Testing & Evaluation Working Group and discussions surrounding the release of the data files to support other research initiatives.

12 September 2012

  1. Day 2 opened with a presentation of the MoLE Testing & Evaluation Results that was focused on the comments discussed during Day 1. Jacob Hodges presented an overview of day 1 Comments, a Statistical Discussion which consisted of the projections of Testing & Evaluation Team, review of participant totals, IRB records requirements, Conclusions and Recommendations.
  1. LtCol Tammy Savoie, J-4 Logistics and the MoLE S&T Coordinator, and Todd Holdren conducted a review of ONRG Grant # N62909-11-1-7025, entitled “Mobile Learning Environment (MoLE) Technology & Transition”. The review, provided by Tribal’s principal investigators, focused on project information (periodicity, value and participants), research goals and objectives, breakout of the research goals and how each was accomplished, the Mobile Applications and the m-Learning Suite, and post-MoLE Transition Activities. Todd Holdren represented the Deputy Director Joint Staff (J7) for Joint and Coalition Warfighting (DD J7 JCW) since the CWP-funded grant was to support the JKO Transition m-Learning Strategy.
  1. Dr. Steve Steffensen, U.S. Army Medical Research and Material Command, Telemedicine and Advanced Technology Research Center (TATRC) Principal Lead; Medical Content Lead and the Portfolio Manager for Global Health Engagement at the U.S. Army’s Telemedicine and Advanced Technology Research Center (TATRC) and Lt Col Tammy Savoie conducted a review of ONRG Grant # N62929-11-1-7026, entitled “Mobile App Infrastructure for Cross-Platform Deployment. The review, provided by Tribal’s principal investigators, focused on project information (periodicity, value and participants), S&T research goals, progress timelines, deliverables, technical reports and the wider successes of the project.
  1. Dr. Steve Steffensen, U.S. Army Medical Research and Material Command, Telemedicine and Advanced Technology Research Center (TATRC) Principal Lead; Medical Content Lead and the Portfolio Manager for Global Health Engagement at the U.S. Army’s Telemedicine and Advanced Technology Research Center (TATRC) and Lt Col Tammy Savoie conducted a review of ONRG Grant # N62929-11-1-7082, entitled “Mobile Content for Medial Stability Operations”. The review, provided by Tribal’s principal investigators, focused on project information (periodicity, value and participants), content progress to date, key successes, deliverables by research strand, S&T research goals, post-Proof of Concept (PoC) refinements to the MedAid App and transition of the capability.

This work relates to Department of the Navy Grant # N62909-11-1-1124 issued by the Office of Naval Research Global (ONRG).