Running head: EVALUATION REPORT1
Evaluation Report
Tasha M. Brown
The University of Alabama
Evaluation Report1
Evaluation Report
Peer Review Results
My peers (i.e. classmates) provided essential feedback on my training session before deploying it to my target audience. While designing the training in Blackboard Learn, I decided to include the peer evaluation in the LMS as well. The peer evaluation was initially set up as a survey to allow for anonymous feedback. However, during class I learned that the results needed to be attached to the person. To meet that need, I attached my peer evaluation as a Word document in an email to my peers. Immediately after class, I realized that I could have change the survey in Blackboard to a test. Nonetheless, my option to email the document worked too.
The peer evaluators provided great feedback on the training sessions. Most of the evaluators wrote comments on how they recognize where the different principles were implemented in the trainings. Some of the feedback that appeared amongst all of the evaluations included the use of the avatar, sequence of the training sessions, and instructional handouts level of information and ease of use. The evaluators stated the avatars were personalized and followed the personalization principle. They also stated that the pretraining principle was implemented in the sequencing of the topics and activities. Finally, they stated the multimedia principle was evident in the instructional resources.
There were also various suggestions on how to improve the training sessions. Most of the suggestions are easy to address and will be implemented before the training is deployed. For the suggestions that would not be addressed, I provide a thorough reason as to why.
One of the suggestions for the project was to add trainings for students. At the beginning of this semester there were an abundant amount of resources for students, but there weren’t anything for faculty members to reference. Therefore, my focus for this project was on the professors as other departments on campus have developed instructional resources for students. I may consider linking to student resources on the “About the Training” page. This will allow the professors to email the students the link to where to find information on how to use Blackboard.
Another evaluator suggested adding session numbers to each training session. I toiled with including a session number as I developed the trainings. My rationale for not including one was if I build upon this project, I didn’t want use valuable time modifying the session numbers in the course and on all instructional resources. I was hoping that the main “Training Sessions” page would provide the learner (i.e. professors) with the correct sequence of the sessions. However, since one of the evaluators have suggested including a session number, it may be an important value for the professors. Therefore, before deploying the training sessions, I may reconsider adding a session number.
A couple of evaluators wondered how I would assess the objectives since the professors were not provided an opportunity to create an assignment within the training course. This is a huge limitation to the project. Unless the professors enroll me in their course, there is no way for me to assess if learning occurred and provide meaningful feedback. As one of the evaluators commented, she wasn’t sure if this could be technically accomplished in the Blackboard training course. Theoretically it could. However, to maintain the integrity and structure of the training session, it would be best for the professors to implement these steps either in their original Blackboard course or in a sandbox “play” course which they can request from the FRC. I do believe, though, that the students who use the course designed by the professor to submit assignments, discussions, etc. will serve as a good indicator as to how the professors have set up the tools in the course. To offer a way to provide some feedback, I may add a statement to the “About the Training” page on enrolling me in their course to provide feedback on the tools they have used in the course.
A couple of evaluators also suggested modifying the layout of the session pages. This includes increasing the size of the home page, reducing the size of the avatar, and breaking up the information on the pages. The home page along with the other pages in the LMS is set to a standard size so that it is viewed correctly in all Internet browsers.
I was concerned about the size of the avatar and the Did You Know fact learning object. Unfortunately, I had resized the learning object to the smallest size available without cutting off the side of the objects. The size of the learning object help determined the size of the avatar. When designing the project, I asked other instructional designers about their opinion and best practices for the size of the avatar. We concluded that the avatar should appear to be the same size as the learning object so the training page doesn’t appear to be disproportional. However, I may reduce the size of the avatar since this was a concern shared by a couple of the peer evaluators.
I’m sure there was a lot of information on each page to process. However, I’m unsure about breaking up the information on the page. This training session will be used mainly by professors who I work with to develop online courses. These professors are very familiar with our course and module layout. The training session mirrors this course and module layout for that very reason. So instead of breaking up the information on several pages, I may reduce the size of the avatar in hope of changing the appearance.
From the reviews, it appeared that the training sessions offered useful information that will allow the professors to successfully design their own course with the tools in Blackboard. I will take into consideration the comments the evaluators have proposed on improving the training and will implement the necessary changes before making it available to the professors.
eLearning Principles Evaluation
This course design presented many opportunities to incorporate some of the instructional design principles. While I couldn’t include all of them, I strived to apply as many principles as I could that would benefit the participant’s learning.
During the first stage of development, I knew the multimedia component would be very important to the overall design and implementation of the training sessions. The multimedia principle was incorporated in the instructional handouts. Instead of just including the instructions on how to perform a task, I included representational graphics (screen captures) that illustrated to the professors what they should see on their screen. Not only did this give the professors a visual representation, but it was also helpful in the professors fully understanding how to complete a task.
To provide the professors with essential tips about Blackboard Learn 9.1, I included a Did You Know fact in each training session. To ensure the contiguity principle was applied, I included the question and answer in the same window. The professors were allowed to view the question first and when Show Answer was clicked, the answer would appear. The contiguity principle can also been seen in the instructional handouts. A part of the screen captures is an arrow labeling the corresponding text/step to the particular area. I also arranged the handouts so that the screen captures appear on the same page as the instructions. This small step makes learning as seamless as possible for the professors.
Instructions in my training sessions were enhanced by the instructional videos. Instead of only providing instructional handouts, I created instructional videos that further explained the steps in the handouts. I used Camtasia Studio 7 to record the videos for each training session. To further benefit the learner’s understanding of the topics, I incorporated an avatar. The avatar was used to enhance each training session overview. It built on the information listed in the overview and discussed some of the objectives. So the learning experience was heighted by the application of the modality principle.
In my project, redundancy principle 2 was followed on the title slide. In addition to the video title, the activities that will be covered in the video were also listed on the title slide. Since there were no graphics or instructions being given, the narration of the objectives should not add to the professor’s processing demand nor will it hinder learning.
For my training sessions, I wanted to actively engage the professors with the materials. However, I had to be mindful of the purpose of the training sessions and the audience. Because this training is to increase one’s knowledge of how to use Blackboard, I refrain from including extra materials such as additional learning objects and graphics. I thought the additional materials would be distracting, overwhelming, and the professors would not take full advantage of them as they are there to only learn how to perform a task. Because of the coherence principle, it was important that my instructions (both instructional handouts and videos), avatars, Did You Know facts, and all other materials were simple, clear, and concise.
Personalizing the training sessions were a rather easy task for me. To convey the personalization principle, I used a conversational style (first and second person) in the instructional videos and the avatars. In addition to using the avatars as describe above, the avatars were also used to introduce and welcome the professors to the training. Because this was the first thing the professors encountered, they are immediately able to relate to a component of the training and thus improving the learning outcome. These simple personalization techniques shift the training session to a learner-centered approach, encourage the professors, and make them feel a part of the learning process.
The biggest concern for me was identifying how to organize the trainings and the content so that it was easily understood and transferablefor the professors. To help with segmenting the content, I viewed the topics and how one will proceed with completing that item within Blackboard. Then for each topic, I listed the main tasks. The tasks were usually included in the order of completion.Under the main tasks were subtasks. I used the main tasks and subtasks to segment the information. For instance, one of the main topics was Creating and Retrieving Assignments. One of the main tasks was to create an assignment. A subtask (and the second step) for this activity was to attach files to the assignment. Once I outlined the tasks, I wrote the instructions in the order. Chunking information this way allows 1) a better understanding of complex information, 2) easy access to only the necessary information at a given time, 3) easier encoding and transferability, and 4) the ability to navigate at one’s own pace. Thus, the steps build upon each other and walked the professors through step-by-step how to complete the main task. The segmenting principle had valuable application to my project.
For some activities, pretraining was very important. Before explaining how to complete the task at hand, it was vital to provide important information and concepts. Pretraining was best exemplified in the Creating Discussion Prompts training session. Before demonstrating how to create a discussion prompt, I thoroughly explained the difference between a discussion forum and thread. At the beginning of the instructional handout, I provided a diagram of how a discussion is setup. To maintain the context of the new concepts, the diagram was referenced at the beginning of the instructional video and during the demonstration of creating forums and threads.
After a professor has completed a training session, he/she can view sample examples in the course. The professors are encouraged to view the sample examples at the conclusion of the activities section of most training sessions. Each of the examples provides the professors with a view of the finished product. At the same token, the professor catches a glimpse of how the tool works from the student perspective. My project was limited to these examples as this training session is only to provide the professors with the necessary knowledge to use Blackboard Learn 9.1. The professors will slowly transition to creating activities as they continue to practice and follow the instructions and tips.
All the professors are encouraged to practice creating an activity once they have completed a training. Beyond that, there are few opportunities for the professors to practice creating an activity and receive feedback. While the professors can request a sandbox course (which will allow them to practice creating activities), many of them will create the activity in the real course where students are enrolled. Therefore, the professors did not practice creating the activities in the training session or receive effective feedback. However, the effectiveness will be measured in the real course as students attempt to complete activities. While the training may have violated some aspects of the practice principle, it did mirror the things (practice principle 2) the professors will be required to complete in their individual courses. From the examples and the instructional materials, the professors will be able to gradually practice and create activities within the real course.
To promote active learning, there is a FAQ wiki. The wiki contains a list of FAQ for each of the six topics. The professors also have the ability to edit the wiki; therefore, they can add their own questions and answers so that their peers can learn. This type of collaboration fosters meaningful learning, uses computer-supported collaborative learning tool, and permits the professors to experience the new tools/technologies Blackboard has to offer.
This is a learner-centered training and the professors have total control over how they progress through it. Because of the navigational controls, they can easily pace and advance through each of the training sessions and resources (learner control principle 4). To assist the professors through this process, techniques such as the Course Tools Menu on the left side and the guided tour videos on the homepage are incorporated. The trainings also include a limited number of links. This reduces the labor-intensity of learning for the novice users.
In addition to supplying the professors with the necessary knowledge to use Blackboard, this training session promotes creative thinking. Because of the new features of the LMS, the professors are afforded many opportunities to be creative and integrate new tools in their courses. The creative thinking leads to engaging activities that encourage collaborative learning. The training sessions were also designed from the whole-task perspective and used guided learning as the main feature. Instead of only providing the content in small segments, the training sessions build upon a real activity. For instance, the Creating Discussion training begins talking about the difference between forums and threads. In that initial discussion, an authentic discussion activity is created. Throughout the session, the necessary knowledge and skills are incorporated and the professors create a discussion forum and thread.
To make learning more effective and engaging, the training sessions include a very simple game. While I would have liked to include more interactive games in the training, I had to focus mainly on the purpose and the audience. Even though I was limited, I did incorporate a Did You Know fact in each session. The Did You Know fact,which was created with SoftChalk 7, was simple, relevant, and easy to use, but provided a significant piece of information for each training session. By aligning the game with the learning goals, the professors were able to have a meaningful learning experience.
As I developed the training session, my ultimate goal was to design a course that aligned with the eLearning principles as identified by the textbook authors. Incorporating some of them prove to be a minor challenge as I wanted to develop a quality project that would benefit others. A few of the challenges were determining how to appropriately fulfill some of the principles such as including games, providing opportunities for practice and offering feedback, and learning together virtually. These were particular challenges since I knew most professors don’t have a lot of time to exert to learning a new LMS. So, I had to determine how to include each principle while providing enough content and verifying I don’t include nonessential information (coherence principle).
The principles prove to be a huge benefit to the design and implementation of the project. Once I knew the content of my project, I developed it around the principles. I viewed each principle and wrote notes as to how I would meet (or if necessary, violate) that principle. Even though some of the professors may be reluctant to the new LMS, I think the design of this training will help the learning process and reduce the learning curve for them. I hope the correct application of these principles will contribute to this success.