Section 3

Chapter 10 discusses evaluation in instructional design and provides you with two evaluation models, the CIPP and Kirkpatrick models for evaluation. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction.

The Input/Process/Output Model was developed in 1990 by David S. Bushnell. It is considered to be a process-driven training evaluation model, related to the CIPP model. It was adopted in 2000 by IBM as their corporate education strategy. It first looks at input factors, “such as trainee qualifications, program design, instructor quality and qualifications, materials quality, facilities, and equipment”. 

It then examines training planning, development and delivery. It then looks at the outcomes, both short-term and long-term.

The Training Validation Systems Model was developed in 1994 by Jac Fitz-Enz. Its four components are Situation, Intervention, Impact and Value. 

Situation: Ascertain current levels of performance and define desired future levels.

Intervention: Examine the reasons for the gap between current and future levels of performance and determine if training is the solution

Impact: Evaluate the difference between pre-training and post-training performance.

Value: “measuring differences in quality, productivity, service, or sales, all of which can be expressed in terms of dollars.”

Resources:

Russ-Eft, D., Bober, M., de la Teja, I., Foxon, M., Koszalka, T. (2008). Evaluator Competencies: Standards for the Practice of Evaluation in Organizations. Retrieved February 9, 2011 from http://media.wiley.com/product_data/excerpt/91/07879959/0787995991.pdf. 

Note: Several different practice-oriented and process-driven training evaluation models are highlighted in this excerpt.

Eseryel, D.. (2002). Approaches to Evaluation of Training: Theory & Practice. In Educational Technology & Society 5 (2) 2002 Retrieved February 9, 2011, from http://www.ifets.info/journals/5_2/eseryel.html.

Note: This is a scholarly work comparing Kirkpatrick, CIPP, IPO and TVS models. Has a helpful table showing the four side by side.

Grice, R. (2010). Flow chart of input-output approach to training evaluation. In How to measure training effectiveness. Retrieved February 9, 2011, from http://www.rogergrice.co.uk/Measuring%20training%20effectiveness.pdf.

Note: The bottom of Page 3 has a flow chart illustrating the IPO model.

I don't have a lot of experience in designing or evaluating my instruction yet. But I think the gist of what both these models boils down is similar to planning a road trip (which I've done many times!) You have to figure out where you (and your students are), where you want to go, and what's the best way to get there. Then afterwards, you assess to figure out whether you got there in a timely manner, and if not, why not? Was there construction, unexpected weather, route too boring? Then I think you go back and figure out a better route for next time, and if maybe you should take the van instead of the compact car. :)

Think about a technological innovation within your social system that was recently introduced (social system = place of work, home, etc.). Any innovation has what is known as perceived attributes - relative advantage, compatibility, complexity, trialability, and observability. Describe these attributes of the innovation and which of the attributes assisted and/or hindered adoption of the innovation. For example, was it not compatible with the existing beliefs of the system or was it overly complex to use or understand?

Our district recently transitioned (and is actually still in the process of transitioning) from using Novell Groupwise (a network-based email system) to using Google Mail (a web-based email system). The biggest hurdle I encountered as a tech support person in the district is that users expected the system to be hard and unfriendly to use, and the transition to be difficult and filled with technical nightmares. Once people were trained and actually began to use the system, they were pleasantly surprised and relieved. However, I had many users who actually insisted that I sit with them while they attempted to sign in and retrieve their e-mail or send an e-mail (not because they had tried and failed, but because they were afraid to try it without backup tech support). The culture of the district and past experience created an expectation that this was going to be difficult and unwieldy, when in fact, the new system is easier and less complicated for users, who were pleasantly surprised.

Chapters 12 & 13 focus on project management and how to manage projects when resources are scarce. You have been assigned to develop a series of professional development sessions focusing on technology use in the classroom for teachers. How will you use Situational Leadership to facilitate this project?

Three Factors affecting Situational Leadership

1. The amount of guidance and direction a leader gives

2. The amount of socio-emotional support a leader provides

3. The readiness level followers exhibit in performing a specific task, function or objective.

Four Phases of Situational Leadership

1. Directive, detailed and supervisory without being overbearing or demanding

2. Explain and clarify decisions and reward improvements (secure buy-in of process and product)

3. Focusing on results and making sure your team is rewarded for effort and production

4. Less as director and more as monitor. Step back and let team work.

Two things that I think are very important pieces of Situational Leadership are actually Phases 2 and 3. Phase 2 is about securing buy-in of the process and product. I think for any learners, but especially with teachers who would be participating in a professional development session, it is important for them to see WHY it is worth their effort to learn or use the technology, and HOW it is going to help them professionally (or help them help their students). When it comes to Phase 3, sometimes “simple bribery” will get you everywhere and rewards participants for their efforts. I have worked very closely with the Instructional Technologists in our district, and I know sometimes they will provide coffee and snacks or a little giveaways, such as a flash drive or package of software templates, to those who come and participate in the training sessions. That helps the training session be more enjoyable, and that little bit seems to motivate teachers to come and to have a good attitude. That is important when many of them are already overwhelmed and overworked.