Step Four—Collecting Your DataWith a specific assessment question in hand, look back at the mapping exercise you completed in Step Two and identify the course or courses that are pivotal for investigation of the program goals you are focusing on. Often, already existing assignments or examinations can be used for assessment (that is, your assessment will be “embedded” in your course). This is an efficient approach, even if you have to modify an existing assignment a bit so it asks more directly for student work related to your program goal of interest. You may be able to write a scoring rubric that the instructors for that course would be able to use for grading; if not, then you can just save copies of the student work and re-evaluate it later according to programmatic criteria. In that case, the scoring can be done later, perhaps in early summer, or whenever the flow of faculty work allows. Remember, any kind of student performance can serve as a work sample. This includes:Written Work Portfolios Presentations (individual or group, videotaped or represented by an observation sheet) Projects (individual or group) Reports Tests (essay or multiple choice) Exhibitions Performances Any other observable student work product Choose your methods and design In many departments, faculty have research training that they can rely on in their assessment work. Whatever methods you choose, your central challenges will be the same—how to collect the strongest evidence possible that your students are learning what you claim that they will learn in your program. The two most powerful elements of evidence are the pre/post design and the examination of student work samples. If you do post-instructional measurements only, you will have evidence of performance but not of learning—maybe they came to you already being able to do what they can do at the end of your program. If you rely on indirect measures of learning, such as student self-report, your evidence of performance or learning is weaker. Most assessment plans intentionally mix methods—for example, gathering student work samples of different kinds from ongoing courses, circulating surveys to solicit students’ subjective impressions of their learning experience, and inviting graduates of the program to participate in focus groups to determine if they felt properly prepared for the workplace. Student self-report about learning is considered useful in assessment work, but alone it is absolutely not sufficient. A good assessment plan will include some examinations of actual student work, perhaps focused on the major question or questions raised through a survey or a mapping exercise. Working with student work samples is labor intensive, so it is best reserved for the really central questions. If you do not have appropriate research expertise in your department, the CAE will be more than happy to help you choose methods appropriate to your assessment questions. If you do have relevant research expertise in your department, you may want to ask us to come and review your plan, anyway. We may be able to help you to avoid common assessment pitfalls, make you aware of any services we can provide to you, or to put you in touch with colleagues in other departments who are working on similar issues. In any case, we will be very interested to learn about what you are doing. Collect and analyze your data Again, if your faculty possesses relevant research skills, you will depend on this as you collect and analyze your data. If you need support in this area, CAE staff can help you with either quantitative or qualitative data collection and analysis. Another way we can help is to design computer-based data entry and analysis systems for your assessment project. In this way, we can enable you to use the technology we all have on our desk-top computers to simplify dealing with your data and save you time. |
 |
