Common Mistakes in Preparing the Mini-Portfolio

Page 1

Revised 6/15/07

1. Introduction

This document lists common mistakes that faculty have made in preparing their mini-portfolios for the Fulbright & Jaworski Faculty Excellence Award. They are discussed here to help faculty avoid them in the future.

Use this document as a supplement to the general instructions and the specific examples for each award category.

Like NIH grants, if your application is not found to meet the standard, you will receive feedback to help you prepare for re-submission.

2. Common Mistakes of a General Nature

2.1. Bulk of evidence is better suited for a different category than the one in which it was included.

 This mistake has most frequently happened when faculty submit a portfolio for the leadership category that mostly contains information about teaching.

 Reviewers can only use the published standards (i.e., examples) for the category for which you submit the portfolio. Consequently evidence that does not fit within the submitted category will have to be disregarded in the review process, unless the applicant explicitly justifies how it applies.

 Keep in mind that the exact same evidence used to substantiate the quantity of your work in one portfolio cannot be used in any other portfolio—even one in a different category. This means that if you reference specific teaching activities in a leadership portfolio, you cannot then refer to those same teaching activities in a subsequent teaching portfolio.

2.2. While most of the evidence included in a portfolio fits the category being reviewed in the given cycle, some of the evidence does not fit the category and there is no explanation as to why it was included.

 It can be advantageous to include some evidence in a portfolio in one category that would better fit another category (e.g., teaching information in the leadership portfolio), if the mismatched evidence is only a minor part of the overall evidence and if it is explicitly linked somehow to strengthening the evidence matching the primary category (e.g., evidence from teaching relates to teaching in a course that faculty member leads).

 Evidence better suited in another category "distracts" the reviewers from focusing properly on the evidence that matches the primary category. Such evidence makes it harder to compare the portfolio to the published standard.

 If you choose to include such evidence, explicitly acknowledge that it relates to another award category and give a brief explanation explaining why you included the evidence. (i.e., teaching within course gives you insight to help you be a better course leader).

2.3. Portfolio lacks information about actual outcomes of your educational contributions.

 Your goal in preparing a portfolio is to make a convincing case that the sum of your evidence of quality, quantity, and breadth matches or exceeds the published standards (i.e. examples). One of the strongest forms of evidence of quality (i.e., evidence that counts even more than learner evaluations) is information about actual learning outcomes (i.e., how well students performed after you taught them).

 Because information about outcomes can be difficult to obtain, it is not absolutely essential. However, whenever such evidence can be included, it tends to be highly regarded and positively received.

2.4. Portfolio lacks information needed for reviewers to judge all six criteria of scholarship.

 The review panel will use six nationally recognized criteria of scholarship to judge the quality of your educational contributions (see review criteria). Your goal is to provide explicit evidence for each of these criteria. Much of this evidence is best presented in your personal statement (e.g., goals as educator, personal preparation; how you have shared the methods and results of your educational activities, including lessons learned, to peer community; and ongoing self-critique).

 We suggest that you become familiar with the six criteria before writing your personal statement or preparing your structured summary.

2.5. Portfolio lacks evidence of peer review.

 To establish one's reputation in research, a researcher constantly submits his or her work to peer review (to get grants, papers published). To verify the quality of one's work in education and to gain insight about ways to continually improve, peer review is also important for educators. Your portfolio should include evidence that you have solicited and used peer input. Increasingly, the review panel is expecting to find evidence of peer mentoring and review in submitted portfolios (e.g., report from peer coach following an in-situ observation), since such evidence is found in the examples used as the standard.

2.6. Order of materials does not match the published examples.

 Reviewers will have limited time to review your portfolio. You can make their job much easier and help them use their time most effectively if you sequence the pieces of your portfolio in the same order as used in the published examples. This way, reviewers can more readily navigate through your portfolio and find specific types of information.

 The correct order is:

o Letter of Transmittal

  • Match to Standard Setting Examples
  • Structured Summary
  • Personal Statement
  • Structured Abstract - Educational Research and the Development of Enduring Materials only
  • Table of Appendices
  • Supporting Documentation arranged in Appendices
  • CV – in BCM format

3. Common Mistakes Related To the Personal Statement

3.1. The personal statement is a narrative summary of evidence, not a statement of goals, preparation, ongoing reflection and improvement, etc.

 Your structured summary is the best place to include evidence of quantity, or an organized catalog of what you've done. The personal statement, however, is the best place to include the information reviewers need to judge your work, using the criteria of scholarship they have been given, especially your goals, personal preparation, how you've shared your work with your peers so that they have benefited from your experience, and your ongoing self-critique and improvement.

3.2. The personal statement is too long.

 In writing your personal statement "less can be more." If your personal statement is much longer than a single page (maximum is two), reviewers will likely not read it as carefully as they should and may miss important information. Remember, you want to help reviewers to pick up certain important points that they need to judge the quality of your work (see 3.1). To ensure that this happens you may have to limit your statement to the most important points omitting all but the essentials.

4. Common Mistakes Related To the Structured Summary

4.1. Too much jargon or reference to unfamiliar things.

 Avoid using jargon or terms that judges are not familiar with (e.g., abbreviations for names of funding sources, awards, etc.). Remember that reviewers are most likely not familiar with many aspects unique to your specialty, department, etc. Unless you provide brief explanations when needed, they may undervalue important evidence (e.g.. not recognize the significance of a particular teaching award).

4.2. Some evidence of quantity in the CV is not clearly included in the structured summary.

 The CV is used as a reference. Evidence in your CV that you have not explicitly included in your structured summary will not be included in reviewers' consideration of your portfolio. We ask that you include the CV and documentation so that the reviewers can "audit" the assertions of quantity and quality you make in the structured summary.

4.3. The structured summary is not sufficiently condensed.

 This is a very common mistake, especially in the teaching category. Unlike the CV where long lists of specific accomplishments are expected, the structured summary is just that, a summary. In the structured summary, you should group related items as much as possible and then summarize the contents of each group with appropriate statistics. Use the published example(s) best matched to you own accomplishments as a guide.

4.4. For faculty relatively new to Baylor, the portfolio lacks sufficient evidence of quantity.

 Careful inspection of the examples indicates that faculty need around 5 years of experience at Baylor to qualify for the award. While this is not a hard and fast rule, faculty with less than five years need to explicitly make the case that they have done an exceptional amount each year so that the total roughly matches the standard-setting examples.

4.5. Information about quantity is ambiguous.

 Reviewers are a diverse group of individuals, including basic science researchers, clinicians, administrators, and educators. Many will not be familiar with the specific time demands of the different types of educational endeavors you include in your portfolio. Consequently, it is important to provide clear statistics or descriptive information so that reviewers can more easily equate the amount or your activities (quantity of time and effort) with the amount of activity included in the examples. One way to maximize reviewers’ understanding is to model the language you use after the example(s) most closely matched to your accomplishments. (For example, when presenting lectures, helpful information can include: duration of the lectures in minutes, frequency with which the lecture is given each year or across multiple years, and the typical number of learners. You can include information about preparation time, if there were special circumstances/demands that you want reviewers to take into account.) Whenever possible, follow the format used in the standard-setting examples and group many individual events and then describe the group as a whole. This will reduce the size of the portfolio and enhance reviewers’ ability to aggregate your activities into a more cohesive big picture.

5. Common Mistakes Related To Documentation (i.e., Appendix)

5.1. Confidential information is not masked or removed.

 Frequently, faculty include reports of learner evaluations that contain the names and results of multiple faculty, including themselves. It is important that the names of other faculty be masked or removed to protect their confidentiality. Furthermore, in most cases, individual ratings forms should not be included, but should be summarized into a table.

5.2. Portfolio includes more documentation than necessary.

 Remember that the purpose of documentation is to substantiate the assertions of quality and quantity you make in the personal statement and structured summary. You want the reviewers to trust that you are not exaggerating your evidence. Furthermore, the purpose of your portfolio, including the documentation, is to demonstrate that your educational accomplishments for the given category meet the published standard. Documentation beyond what's listed in the standard simply wastes paper and the reviewer's time. To help you avoid unnecessary documentation, limits in the number of letters and number of pages were implemented in the summer of 2007. It is important to adhere to these limits.

5.3. Letters of support or recommendation don't appear to have influenced self-reflection and/or improvement.

 Letter of support or recommendation can serve several purposes. For the purposes of promoting educational scholarship, the ideal letter provides feedback and insight which helps the recipient think about and improve their work. Some letters, such as those written for academic promotions, often fall short of this goal because they contain only "summative" feedback (general information about overall quality). We encourage faculty to also solicit input, or "formative" feedback (information to help make improvements), from their peers on an ongoing basis and then to include evidence of this feedback and its impact in the portfolio.

5.4. Copies of learner evaluations are not summarized and/or synthesized.

 To help reviewers form general conclusions about the quality of your work, we encourage you to resist the temptation to include individual copies of all your reports of learner evaluations, especially when you have them from different courses across many different years. (See 5.2.)

o We suggest instead that you create a table summarizing the results from the separate reports.

o In addition to quantitative data (e.g., evaluation averages), it is also appropriate to include representative qualitative data (e.g., comments from learners).