Map the System 2018 – Screening Checklist

To evaluate the submissions for Map the System, we will review them through two lenses. The first is a pass/fail review of requirements: Does the submission meet the criteria? (Yes/No). If any of the answers in that section are NO, the submission will not be considered further. If all the answers in the Requirements Test are yes, the proposalwill be reviewed using the “Evaluation Criteria” section. The final question is an overall view of whether the submission should proceed to the next stage of the competition.

Team Name______Name of Reviewer ______

Institution Name ______

Step 1: Requirements Test

No. / Criteria / Yes/No
1 / Social or Environmental Issue/Problem
●Does the submission examine a social or environmental issue?
●Has the problem been articulated clearly in a concise and descriptive way?
2 / Team Composition - Team Size
●Maximum of 5 team members; and
Team Composition - Student Status
●At least one member of the team is either a current studentof the host institution (must be a registered student as at the date of the final submission deadline) or agraduate of the institution (having completed studies within 12 months of the date they registered for the competition).
3 / Submission Components
Are all of the following elements included in the final submission?
●Visual map/chart
●Summary of research findings (Word; PDF or Prezi - max 2,000 words); and
●Bibliography
**If all of the above are “yes”, move on to the main evaluation criteria. If no, reject the application.

Step 2: Evaluation Criteria

Criteria
Each of the following 5 criteria will be scored individually (points range 1 to 4). The questions under each criterion provide examples of some of the areas to consider in scoring the criteria. It is not intended that every submission is able to satisfy all of the questions in order to get a strong rating as some of them may not be relevant to the research topic selected. / Score of 1:
The submission is sub-par for this criteria, i.e. insufficient information provided; inadequate detail or obvious inaccuracies. / Score of 2:
The submission is average or just meets this criteria i.e. minimum required level of detail provided. / Score of 3:
The submission is strongly rated for these criteria, i.e. above average in terms of the level of detail provided. / Score of 4:
The submission excels or is ‘best in class’ for this criterion, i.e. information clearly demonstrates strong depth and breadth of research and/or demonstrates strong insight and self-reflection.
Criteria 1: Understanding of the problem
  1. Does the submission present a macro view of the sector (including an outline of the scale of the problem) with justification for the geography selected?
  2. Does the submission demonstrate an understanding of the depth, complexity and key risks (including critical government, country or economic risks) of the problem identified?
  3. Does the submission address the key underlying drivers of the problem?Does it address any complexity of tangential or related problems?
  4. Does the submission clearly demonstrate an understanding of all key stakeholders affected by the problem (directly and indirectly)?
  5. Does the submission outline a hypothesis as to why and when the problem started occurring and why it persists?
/ The problem is discussed only at the surface level. The root causes are overlooked or unexplored and/or presented in a way that is insufficient to demonstrate the depth of understanding expected from the timeline of this project. Opinions or assumptions are presented as facts with little or no evidence. / The applicants’ own opinions might be shared backed up by the facts of their research, yet those opinions are extrapolated to make unjust implications about the problem as a whole. The problem is explored in detail, root causes are identified, but some of the complexity is overlooked. For example, little or no explanation is given for why the problem persists and/or key risks. / The problem is well defined, data is cited, complexity is addressed, and reasons for the persistence of the problem are explored. Some additional detail is missing which would have made the report excellent. / The submission shares a thorough mapping of the problem, an understanding of the root causes, and a thorough portrayal of what is holding the status quo in place. There is a nuanced look at the impacts of the problem,including expert opinions and/or well-referenced insighton its future trajectory. Outstanding research questions are identified in areas where the applicant/s would have liked to go deeper.
Score awarded for Criteria 1:
Score of 1 / Score of 2 / Score of 3 / Score of 4
Criteria 2: Understanding of the solutions landscape
  1. Does the submission clearly articulate the overall solutions landscape in a concise and descriptive way, including acknowledgement of a diverse range of existing players, resources and networks?
  2. Does the submission demonstrate a deeper understanding of the nature and diversity of the existing solutions, including an understanding of what has worked, what hasn’t and the change needed?
  3. Does the submission explore some of the models for change being tried and what distinguishes the approaches?
  4. Does the submission look between the organizations by not only listing specific organizations but also looking at what joins up some of the efforts?
/ Very few nodes on the solutions landscape are identified, or many are identified yet they are not explored in any depth (simply a list or groupings of logos without exploring their work). / Many solutions efforts are listed and some are explored in depth. Some models are identified, yet the depth of analysis is not very robust. Little or no effort is focused on the space “between” the efforts (i.e. government support, knowledge sharing, etc). / A thorough look at different solution models is included, plus a comparison of their efforts. Different types of efforts (business, government, non-profit, etc) are explored and tangential efforts, which could be learned from, are discussed. / The submission is exceptional in its analysis of the solutions landscape. It not only includes a wide look at the efforts both locally and globally from which lessons can be drawn, but also thoroughly examines the space between the efforts and the links between different solutions.
Score awarded for Criteria 2:
Score of 1 / Score of 2 / Score of 3 / Score of 4
Criteria 3: Identification of Gaps & Levers of Change
  1. Does the submission clearly identify any potential gaps, market opportunities or what is missing from the solutions landscape? Does the submission clearly outline key levers of change that could contribute to the solutions landscape as well as actionable responses?
  2. Does the submission look beyond singular organizational ideas or new start-ups and identify efforts by government, non-profits, educational institutions, and/or researchers or other groups whose efforts might contribute to landscape of solutions?
  3. Are the gaps explained in enough detail to demonstrate an understanding of what it would take to positively contribute to solving this problem?
  4. Does the submission identify possible collaborations (private or public) to provide a more holistic solution? E.g. knowledge sharing opportunities, regulation, or a range of ideas apart from one singular “solution”?
/ The applicant/s present “their” solution to the problem (i.e. one idea for a business or activity designed to solve the problem) rather than identifying a range of missing pieces, and/or the applicant/s provide only a surface level list of missed opportunities which are not explored in detail.
Note: If the presentation focuses in great detail on THEIR idea for a solution, with very little detail on other possible paths to impact, they should receive a 1 in this section. / Many ideas for improvement and impact are explored, yet most of them are not explored in detail. / A thorough look at what is missing, including not only new ventures or new initiatives but also how the current efforts could be joined up or improved. Government, business, non-profits, researchers, and other actors are all considered. / The submission not only includes all of the efforts listed in the previous column but also goes into more detail on what would be needed for those efforts to succeed, linking up the lessons they learned in the problem analysis and demonstrating the insights they gathered from their interviews and research.
Score awarded for Criteria 3:
Score of 1 / Score of 2 / Score of 3 / Score of 4
Criteria 4: Lessons Learned
  1. Does the submission demonstrate a deeper awareness and self-reflection of key lessons and insights derived from the research?
  2. Do the applicants link the key lessons they learned when exploring the underlying drivers of the problem to the proposed levers of change?
  3. Does the submission include key lessons learned which anyone looking to take action on this issue could learn from or build upon when designing their solution?
  4. Do the applicants acknowledge any assumptions they tested or theories they disproved in their own learning journey about their perception of the problem/solutions landscape and the reality?
/ Very few lessons are drawn from the applicants’ research. Research seems like a repetition of facts rather than their own analysis and comparison across various perspectives. / Some lessons are shared, but there is little effort to link the problem analysis with these lessons. Assumptions are proposed for how to best move forward, but the data behind those assumptions – or even a recognition that they are assumptions – is not shared. / The lessons shared are very valuable, deep and include self-reflection on any assumptions tested or opinions changed. More detail would be helpful to consider the insights exceptional. / The lessons shared are so insightful that the reader feels they have acquired a valuable education on the topic. The link between the problem analysis and the lessons learned is strong. The lessons shared would be useful and insightful for anyone working in this area to take action.
Score awarded for Criteria 4:
Score of 1 / Score of 2 / Score of 3 / Score of 4
Criteria 5: Research Approach & Presentation of Deliverables
  1. Does the submission demonstrate a diverse range of research sources (i.e. not only just desktop research but practitioner interviews too)?
  2. Does the submission demonstrate a pro-active and creative research methodology?
  3. Do the visual ecosystem map/s submitted adequately reflect the necessary level of detail in a creative and insightful way?
  4. Does the research analysis report submitted provide sufficient detail to understand the research approach and methodology of the research undertaken, including a summary of key insights?
  5. Has the submission been adequately cited/referenced in the bibliography?
/ No interviews were conducted. Information is poorly referenced. Data is poorly presented. / Most research was desktop research, but information is cited and the range of data was adequate to present an average submission. / Some interviews were conducted but most research was desktop. The data is well presented.
Note, if the data is presented creatively but little to no interviews were conducted, they should still receive a 3 rather than a 4. / Many sources, including interviews, are referenced in the data collection. The data is presented in a compelling and audience-friendly way.
Score awarded for Criteria 5:
Total Score:

Step 3

Overall Application View
  • This application should progress further to the next stage of the competition
/ Yes/No
Comments
If selected to proceed, further questions to ask the applicant/s during the next stage: