Step Three—Developing Your Plan
Focus on questions that faculty really care about If you have allowed for plenty of discussion and reflection in Steps One and Two, genuine questions about what your students are actually learning have probably surfaced. Those are the questions you should focus your assessment work on. Do not allow yourselves to be pulled off these questions by externally imposed ideas of what you should be assessing, or by a mechanistic impulse to assess each of your objectives in turn. You may come up with more than one major question for assessment. If so, we strongly suggest that you consider investigating these questions one at a time. Taking two years to answer one important question well—with the first year devoted to data collection and the second to analysis and any needed curricular redesign—is preferable to collecting piles of data in one year, being overwhelmed with a huge data entry and analysis task that no one has time to do, and becoming discouraged about sustaining your assessment efforts. Sound familiar? If you are able to, write a five-year plan. Your plan will probably change over a five-year period, but five years is a good period of time to plan for. Include some analysis of actual student work samples The basic question that we are held accountable to in assessment of student learning is: Did our students learn what we said they would learn in our program? The surest way to answer this question is to look at what your students actually do—that is, to look at actual samples of student work. No program assessment plan is complete without this. Fortunately, it is something that can be done in any program. Do you claim that your students learn how to write? How to sing opera? How to teach elementary school children? Then papers, concerts, or teaching are the student work samples you will want to assess. You are assessing this student work already when you grade. Often, all you need to do for program assessment is to create a system for scoring that same work on program objectives of interest, in a way that is intelligible to other people. Using student work samples for assessment often means developing a scoring rubric. This raises the specter of scoring for ineffable qualities—a project that seems disrespectful, reductionist, and oxymoronic to some of us. On the surface, scoring for ineffables shouldn’t feel like a problem to us—we do it all the time when we grade. This objection, however, hides a more fundamental objection, which is that we resent the dictate to translate our professional judgments to external audiences. As we mature in our professions and disciplines, we develop a sense of what constitutes excellence. We share that sense with colleagues in our field, and it is part of our identities to be within disciplinary communities of understanding about what constitutes high quality work. Being required to do assessment means being required to translate our processes of judgment and show them to others. Unless we are genuinely focused on questions that we ourselves actually want to know the answers to, this feels invasive and insulting to our shared departmental or programmatic sense of professional expertise. If we are really finding useful answers to interesting questions, however, then these issues regarding accountability are a sideline. In any case, an assessment plan that does not include any direct
examination of student work samples cannot make a prima facie argument
that our students can do what we claim we teach them to do. Fit the time and resources available to you There is little point in writing an assessment plan that you cannot accomplish. Focus on one question at a time, and give yourself time to design your assessment, collect your data, analyze your data, discuss the results, and design and implement any curricular changes that you agree on. It can easily take two years to complete one such project, given the constraints that we all face on our time. It is far better to see one project through to actual programmatic change than it is to collect a lot of data that no one knows what to do with or has the time to do anything with. As you draft your five-year plan, consider questions such as the following.
The Hypothetical Studies Department, for instance, requires its majors to take a two-term, 300-level course sequence. Both courses in this sequence are designed to be writing intensive. Faculty who teach the 400-level theory course, however, are not satisfied with the quality of student writing they are seeing, and suspect that the 300-level courses are not preparing the students well. A review of available records show, however, that the 300-level courses are bottlenecks and that about half of the department’s majors take the 400-level course first. Do students who take the 300-level course sequence first write better in the 400-level course? Do they do better in other ways? In their most recent five-year assessment plan, this is the first student learning issue that the Hypothetical Studies Department will investigate. They are planning to base their assessment on examination of student writing samples from the 400-level course. They will design their assessment in spring term of Year 1, collect data from both of the 400-level course sections offered during Year 2, analyze that data during fall and winter of Year 3, and spend spring of Year 3 on any needed course redesign. That may seem like a long time to invest in answering one question. But it’s an issue they really care about, it’s a pace of work they can really manage, and they will end up with good, solid evidence of what their students can do. |
 |
