Read Chapter 10, Designing and Conducting Formative Evaluations, from Dick, Carey & Carey.

Since you are not developing your instructional materials in this course, we will only be planning for possible formative evaluations that we may use to evaluate the materials when they are developed.

Martin Tessmer, in his book "Planning and Conducting Formative Evaluations", defines formative evaluation as "a judgment of the strengths and weaknesses of instruction in its developing stages, for purposes of revising the instruction to improve its effectiveness and appeal" (pg. 11). In a formative evaluation, evidence of an instructional program's worth is gathered for use in making decisions about how to revise the program while it is being developed. This is why it is called "formative" evaluation, because the instruction is in its developmental stages and is not yet "finalized ". It could be considered a prototype of the final materials. In a formative evaluation we want to determine if your newly developed course works at teaching the objectives you need to teach to the learners who need to learn them, before you present it to your target audience.

In any given formative evaluation, you can find out how to make your instruction more effective, efficient, interesting, motivating, usable, and even acceptable. This is done by carrying out procedures that will provide you with evidence as to these qualities in your instruction. The emphasis is on collecting data and revising the instruction. This revision process uses the results of your formative evaluation activities and can affect any of your previous steps.

Many believe that formative evaluation is the key to instructional success. Conducting a formative evaluation allows you to try out your materials to find out how they work and how you might make them better before you present them to your target audience. In many cases, however, evaluation isn't done because of cost and timing issues. It's difficult to get the design and development work done, and often by that time most people involved are ready to be finished. Robert Mager, in his book "Making Instruction Work", seeks to remind us of the importance of evaluation through these scenarios:

"Has this plane been tested?"
"What?"
"Has this plane gone through tryouts to make sure it works?"
"We don't have time for that. We consider the maiden flights the tryouts."
"Oh."
Or,
"Has this medicine been FDA approved?"
"What?"
"Has this medicine been tested and approved safe to use?"
"No need for that. We find out on the first few patients who die from it."
"Oh."

No matter how good you might be at instructional design, there's still no way to find out how well it works until you try it out. Since nothing is perfect, everything can be improved.

Expert Review
In this evaluation, experts review a rough version of the instruction with or without the evaluator present to determine its strengths and weaknesses. They can be content experts, technical experts, designers, or instructors.

One-to-One Evaluation
In this evaluation, one learner at a time reviews the instruction with the evaluator. The evaluator observes the learner using the instruction, notes the learner's comments, and questions the learner during and after the instruction.

Small Group Evaluation
In this evaluation, the evaluator tries out the unfinished instruction with a group of learners and records their performance and comments.

Field Test
In this evaluation, the evaluator observes the instruction being tried out in a realistic environment with a group of learners.

Generally, these four phases are carried out in order, although Expert Review and One-to-One can be carried out at the same time. Based on the data you collect in each stage of the evaluation, decisions should be made regarding how you will revise the instruction.

If you recall, in the last lesson you completed your plans for the development of the instructional materials you designed. You also considered your role in the development and delivery of the instruction. Now it's time to give some thought to the formative evaluation process.

It may not be realistic to think that you will be able to perform each type of formative evaluation with your instructional materials. Even so, we would like you to think about how you might perform each type of evaluation. There may be ways for you to fit in an expert review or one-to- one evaluation. Perhaps you can find a small group that will try out your instruction. No matter what you decide, the goal is to find ways in which to gather as much information as possible about the instruction so that it can be appropriately revised. Anything is preferable to "going public" with your first draft.

1. Describe how you might conduct an Expert Review of your instructional materials. What subject matter expert(s) would you consult with, and what specific types of information would you hope to collect?
   
   
2. Describe how you might conduct a One-to-One evaluation of your instruction. How would you go about selecting a few learners to participate who are representative of your target population? What factors would you take into account in the selection process? What specific information would you want to gather about your instruction, and what tools and methods might you use to gather that information (e.g., checklists, questionnaires, interviews, etc.)?
   
   
3. Describe how you might conduct a Small Group evaluation of your instruction. How would you go about selecting a small group of learners to participate who are representative of your target population? What specific information would you want to gather about your instruction, and what tools and methods might you use to gather that information?
   
   
4. Describe how you might conduct a Field Test of your instruction. What instructional setting would you use for the field test? How would you go about selecting a group of learners to participate who are representative of your target population? What information would you want to gather about your instruction that was not already gathered during the small group evaluation, and what tools and methods might you use to gather that information? If your situation makes it impossible to perform a true field test, then you may want to consider your first formal presentation of the instruction as a field test. However, if you do this, leave yourself some time to revise the materials after this initial instructional session. Don't set everything in concrete.

When you have addressed each of these questions, write a summary describing your proposed formative evaluation procedures.

This assignment should be produced using Microsoft Word. Append this assignment to the one you saved in the previous lesson (developments.doc). After you have saved your file, go to the student interface and submit your assignment for grading. Click here if you need additional information regarding submission of your assignment.

Points: 5

Grading Criteria

- Describes proposed Expert Review evaluation procedures, including types of subject matter experts they would consult with, and the information they would hope to collect. (.5)
- Describes proposed One-to-One evaluation procedures, including how they would select learners to participate, the specific information they would be looking for, and the tools and methods they would use to collect it. (.5)
- Describes proposed Small Group evaluation procedures, including how they would select learners to participate, the specific information they would be looking for, and the tools and methods they would use to collect it. (.5)
- Describes proposed Field Test procedures, including the setting they would use, how they would select learners to participate, the information they would be looking for beyond what they Small Group evaluation provided, and the tools and methods they would use to collect it. (.5)
- Summary of proposed formative evaluation procedures. (3)