1. Two Models for Evaluation
Flashlight Triad Model
Virginia Tech uses the Flashlight Triad Model of evaluation for technology oriented evaluations. The Flashlight Model comprises five steps:
1. Overview and Confronting the Blog: This step involves brainstorming to create a comprehensive list of elements that “feed into and flow” from a lesson or course. The examples that were given were: faculty development and preparation, student prerequisites, budget, assistance with course materials development, and student attitudes and satisfaction.
2. From Blob to Issue: Consider the purpose of the evaluation. Two questions should be answered:
1) What are the three most crucial things you fear might happen as a result of using the technology?
2) What are the three most crucial things you hope will occur?
One or two top issues should be selected from the answers and become the focus of the evaluation.
3. From Issue to Triad: For each issue that was selected, several triads are created. “A triad consists of the type of technology you might employ (T), a specific activity that the technology enables (A), and outcomes expected from that type of activity (O).” After several triads are created, the ones that will be focused on are selected.
4. From Triad to Data: For each triad that was selected, questions are generated to gather data through surveys, focus groups, interviews, etc. At least five types of questions can be generated: 1) Technology (related to technical infrastructure, training, function, etc.), 2) Interaction of Technology and Activity (related to technical systems and their ability to support the specific activity), 3) Activity (related to the nature of the student tasks), 4) Interaction of Activity and Outcomes (related to the capability for technology-based activities to generate predicted outcomes), and 5) Outcomes (related to student learning, satisfaction, and student suggestions)
5. From Data to Next Steps: Based on the evaluation data, decisions will be made about modification and improvement for the course/lesson.
My Implementation of the Flashlight Triad Model: Since technology implementation is such an important part of instruction, this is an excellent tool to evaluate the use of those technologies in the learning process. I have just started to use blogging in my classroom as a means for students to express their ideas and opinions on specific topics. In my evaluation of the use of blogging using this model, I would first focus on student attitudes and satisfaction. For the issues, I would select student motivation and active learning. The triad I would use is as follows:
Expression of ideas, thoughts, and reflection to improve writing; sharing ideas to support different perspectives
Students demonstrate more ownership in their writing; are more aware of checking their writing for grammatical and spelling errors before posting; gain different perspectives of ideas through reading other blogs.
From this triad, I would generate survey questions to ask students through an online survey. Based on the data that is collected, I can make a decision as to whether blogging is being beneficial for students in my classroom.
Dessinger-Moseley Full Scope Evaluation Model
The Dessinger-Moseley (1998) model of evaluation integrates evaluation throughout the ISD model. It includes six integrated evaluation activities:
|Recreation of Figure 1-1 from Confirmative Evaluation: Practical Strategies for|
Valuing Continuous Improvement by Dessinger and Moseley (2004)
- Formative Evaluation focuses on the need for training, instructional design and development processes and products, reaction during and after training, and accomplishment during training.
- Summative Evaluation focuses on the immediate results of program implementation: reaction during and after training, accomplishment during training, and the self-reported expectation that new knowledge, skills, and attitudes with transfer to the job and affect workplace performance.
- Confirmative Evaluation focuses on the program’s continuing impact and value, as well as the long-term transfer of knowledge and skills to the job and workplace.
- Meta Evaluation focuses on all the evaluation types—their inputs, outputs, outcomes, and focus.
Source: Dessinger, J., & Moseley, J. (2004). Confirmative Evaluation: Practical Strategies for Valuing Continuous Improvement. Pfeiffer & Co.
My Implementation of the Dessinger-Moseley Full Scope Evaluation Model:
|Project Team Communication Pattern|
Created using PowerPoint Diagrams
Phase 2: As we continue to meet to plan, organize, and develop the professional development sessions, the team should build confidence and begin to “buy-in” to the goals of the team. In this phase, I will assume a more coaching leadership style by explaining, advising, and supporting decisions made by the team.
Phase 3: In this phase, focusing on results is critical. My style of leadership must move to being that of a supporting motivator—encouraging the team and rewarding them for effort and production. I would ask the client for team members to have additional time during the school day to work together as well as provide lunch for team members.
Phase 4: The team at this point should feel comfortable in working together. As the manager, I should now be able to step back and take the role of a monitor. The decision making and problem solving aspects of finalizing the project can now be proficiently completed by the team members.